How to deal with new things added to Prelude in the future?

[RyanGlScott]

A thought occurred to me when reading this thread about MonadFail. In upcoming releases of GHC, new things are going to be added to the Prelude which haven't existed in GHC for very long. MonadFail, for instance, will be added in GHC 8.8, and I believe Semigroup will also make it in at some point.

This poses an interesting design question for base-compat, because unlike other typeclasses which have been migrated to Prelude (e.g., Applicative via the AMP), MonadFail et al. are not available in base prior to 4.9. It seems that we'd have to do one of the following:

1. Make base-compat depend on semigroups/fail/whatnot so that Semigroup/MonadFail/etc. can be backported from Prelude properly
2. Don't give base-compat any new dependencies. Instead, conditionally export things from Prelude based on what is available from a given base release
3. Some sort of hybrid approach

What are people's thought on this?

[RyanGlScott]

One possibility (which complements option 2 above) is to release a separate package called base-compat-extras which, unlike base-compat, does not have a strict no-dependencies policy. That way, it could safely depend on the semigroups package for old GHCs, and it could have a module called, say, Prelude.Compat.Extra that exports Semigroup on all versions of GHC.

[RyanGlScott]

This is now an issue in practice. For GHC 8.2, I'm going to be adding Data.Proxy.Compat which, because of the strict dependency rules that base-compat has, can only backport asProxyTypeOf back to GHC 7.6, when Data.Proxy (and thus the Proxy type) was first introduced. This is pretty annoying, since that means if you want to use asProxyTypeOf before GHC 7.6, you'll have to conditionally use Data.Proxy from the tagged package, and that's a whole lotta bleugh (see also #35).

I'm pretty tempted now to start base-compat-extras, especially since its usefulness will increase even more once Semigroup/MonadFail become a part of the Prelude and base-compat won't be able to backport them "properly".

[RyanGlScott]

cc'ing @phadej and @hvr, who might be opinionated on this topic.

[sol]

@RyanGlScott My preferred solution would be to:

1. Start migrating data types and classes over to base-orphans
2. Add base-orphans as a dependency to base-compat

@ekmett already indicated that he could be "pretty easy to convince".

[RyanGlScott]

I'm not sure we're talking about the same thing here. base-orphans is intended to be a canonical home for orphan instances from base, yes? If so, it doesn't really make sense to migrate them to base-orphans.

[hvr]

When I discussed with @ekmett to split e.g. semigroups into a lightweight semigroup-class and semigroups class in order to reduce the amount of conditionally exposed instances (as conditional APIs can cause quite some headaches for Hackage Trustees; and the main reason package authors were reluctant to depend upon semigroups was its non-trivial dependency footprint), this would have required us to instead start retroactively encode mutual exclusion between the old semigroups and the new semigroup-class via special Hackage revisions (which certain people would hold against us as usual), as well as incur the penalty of using orphan instances for the legacy instances in semigroups (which currently aren't orphans). IOW, it would have been messy to do this properly; due to this and other reasons we didn't pursue the semigroups-class/semigroup split. However, moving the canonical not-yet-in-base Semigroups or MonadFail into base-compat (it makes little sense to move them into base-orphans as they wouldn't be orphans) would result in the same dilemma as with the semigroups/semigroup-class split.

Unfortunately, I don't have any solution to offer yet... :-/

[RyanGlScott]

@hvr summarizes my thoughts on the matter much better than I could. For those reasons, I'd prefer to keep the task of backporting Semigroup to the semigroups package.

Given that base-compat is intended to be at the bottom of the dependency chain, I'd favor a separate package built on top of base-compat which has a more lenient dependency policy, which in turn could depend on semigroups, fail, and other compat packages as necessary.

[phadej]

I'm 👍 for a new package, which would re-export semigroups, fail, nats, tagged, bifunctors (Data.Proxy), void, transformers (Data.Functor.Identity!) things (I probably forget something!). The short mission statement would be.

backport base as much as possible, 0-dependencies with latest base, additional dependencies are used to backport types and classes for older base versions.

The clear difference to base-compat, is that base-compat has base (and unix) as only dependencies.

I'd use that package across "my" packages, e.g. quickcheck-instances, these and servant, as I don't care about increased dependency footprint for older GHCs (lens depends on all of that anyway). Others care, e.g. nick8325/quickcheck#158

[ekmett]

That makes for a pretty clear mission statement and role in the community.

[phadej]

@RyanGlScott asked me to write this long time ago, I apologize it took so long:

I see two options to proceeed:

• base-compat (as current) + base-compat-batteries
• base-compat + base-compat-lite (current base-compat)

This is naming issue, i.e. do we want to have base-compat to be the small
package depending on base, or bigger package depending on other
compat-packages (NB: on old GHC only).

I personally prefer the base-compat + base-compat-lite version, as
virtually every package I maintain currently have if !impl(ghc >= 8.0)\n build-depends: semigroups. This write up is to ask others, @ekmett and @sol,
in particular, what use is more important to you. The more popular use
should take the base-compat name.

We have brainstormed the names for alternatives with Ryan, and I think
that those are very good ones.

base-compat + base-compat-batteries

• base-compat stays as currently
• base-compat-batteries is the backport base as much as possible,
  0-dependencies with latest base, additional dependencies are used to
  backport types and classes for older base versions.

base-compat + base-compat-lite

• base-compat is the backport base as much as possible,
  0-dependencies with latest base, additional dependencies are used to
  backport types and classes for older base versions.
• base-compat-lite will be what base-compat is currently, i.e.
  package depending only on base and not backporting data types or type
  classes.

So what you think, how we should the new package be?

[phadej]

Notes about batteries package:

Whatever will be decided above, I have few ideas for batteries package.

The main observation is that same package is unlickely to depend on both
base-compat and base-compat-lite (my preference for package split :)

I'd prefer that both package provide same modules, e.g.
Prelude.Compat, thus upgrading from "small" to "big" package is trivial.
Also we'll have only single Control.Monad.Compat and not two names.

We also want bigger package to re-export identifiers from the smaller one,
so

import "base-compat"      Control.Monad.Compat
import "base-compat-lite" Control.Monad.Compat

won't cause name clashes.

[RyanGlScott]

(For the record, I'm not strongly leaning one way or the other at this point.)

One thing to consider is that we're going to have to change something here, and given the significance of this milestone—to the point where base-compat users will need to consider whether they want to depend on a library with actual dependencies or not—it would be wise to put out some sort of public service announcement advertising this choice, regardless of which choice we make.

One thing that we don't really have a strong feeling for at the moment is a consensus for what base-compat users want. That's part of the reason why we've been so conservative with dependencies, as we want to ensure no one can be offended by our dependency footprint! But maybe most base-compat users actually don't care that much in practice, and would in fact be willing to incur small dependencies like semigroups if it means they can have a Prelude that extends further back.

Perhaps what we need is an informal poll of some sort to gauge what folks desire here. I'm thinking of perhaps a blog post which has:

1. An explanation of what the issue is
2. A recap of the possible ways forward presented here
3. A call to readers asking which way they prefer

I don't really have the time to do this at the moment, alas. But getting feedback from a sizable amount of people (for some definition of "sizable") would be tremendously helpful here.

[RyanGlScott]

Just as an update—I made significant progress on this last night, in the 8.4 branch. At this point, the core infrastructure is in place—I've just got to resolve some annoying test failures.

I went with the base-compat + base-compat-batteries approach. @phadej made me realize that going with the approach in https://github.com/haskell-compat/base-compat/issues/36#issuecomment-372139029—namely, exporting the exact same modules in base-compat and base-compat-batteries—makes it dead simple to upgrade from base-compat to base-compat-batteries (or vice versa). As such, the choice about which library is the "core" one isn't really that significant, since switching between the two is almost entirely trivial.

In that light, I think we should err on the side of least surprise and not add a bunch of dependencies to base-compat all of a sudden. I'll go ahead and make a major version bump for base-compat so that folks are more likely to look at the changelog and discover the existence of base-compat-batteries, but at that point, the choice of whether to "upgrade" (i.e., incur more dependencies) is up to them.

[RyanGlScott]

See #43.

[ekmett]

One small downside to exporting the exact same module name is then you're forced into a small dilemma. Do you put exposed: False in the cabal config for one of them? If neither has that then ghci users suffer if they want to import one of them and get a clash, forcing use of package imports. If one of them, then you'd still need to do so (or export a redundant copy out of another namespace) if you want to use the other. This is a bit less of a concern here than it usually is for this sort of thing, because these are basically just instances and the like, but seems at least worth considering.

[phadej]

@ekmett note that e.g. cabal repl hides all non-direct dependencies.

Also, TBH, I very rarely import Prelude.Compat in GHCi, as I just use latest GHC for interactive stuff.

[ekmett]

That works great as long as you are in a cabal project. When you are just playing around with standalone haskell files, you still have the issue.

…

On Mon, Mar 26, 2018 at 3:50 AM, Oleg Grenrus ***@***.***> wrote: @ekmett <https://github.com/ekmett> note that e.g. cabal repl hides all non-direct dependencies. Also, TBH, I very rarely import Prelude.Compat in GHCi, as I just use latest GHC for interactive stuff. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASmEYn3NIRMJb49fpaXRP14pp2lVxj8ks5tiJ25gaJpZM4GkKol> .

[RyanGlScott]

Admittedly, choosing identical module names does make certain things much more inconvenient.

As @phadej notes, GHCi is not a use case I'm terribly concerned about, since (1) most folks don't reach for base-compat when trying things in GHCi, and (2) there exists -hide-package base-compat/-hide-package base-compat-batteries if you do.

Making it harder to use base-compat in standalone Haskell scripts is a more compelling argument. Unlike in Cabal-based projects, having base-compat and base-compat-batteries coexist with the same module names is much more likely to break any scripts you already have that use base-compat, since they're probably not importing .Compat modules with PackageImports. Even Stackage-based scripts are susceptible to this issue, since putting a stack invocation at the start of your script only ensures that you've installed the dependencies—in the end, if you wind up having a stack-curated database with both base-compat and base-compat-batteries installed, it'll end up breaking due to the ambiguity.

Perhaps this is a sign that standalone Haskell scripts should always be using PackageImports as part of proper hygiene. Or perhaps it's a sign we need to be shipping .cabal files with scripts. Regardless, this choice would break some common workflows.

That being said, there are downsides to having different module names in base-compat and base-compat-batteries. With identical module names, the two are almost completely interchangeable, which makes it extremely convenient for the Cabal–based workflow (as well as for those "upgrading" from base-compat). Plus, having .Compat.Batteries at the end of every module can lead to some comically long module names, like Control.Monad.ST.Lazy.Unsafe.Compat.Batteries. (There are other suffixes we could choose that are marginally shorter, like .Aux or .Extras, but the core issue is still there.)

tl;dr I'm somewhat reluctant to give up the sheer convenience that the current naming convention gives you. It can bite when you work out of a globally installed package database, but when working with Cabal, it's mighty helpful, and I think it would make the migration story from base-compat much cleaner.

[RyanGlScott]

After talking with @ekmett about this, we've struck a deal. To support the workflows of both library authors and folks who tinker with GHCi/write standalone scripts, a reasonable compromise is to provide versions of each module with a globally unique namespace. For instance, we currently have Prelude.Compat in both base-compat and base-compat-batteries, so we would:

• Reexport Prelude.Compat from Prelude.Compat.Repl in base-compat
• Reexport Prelude.Compat from Prelude.Compat.Repl.Batteries in base-compat-batteries

I'll update #43 accordingly when I get a chance.

[phadej]

👍 to re-exports