Handling of RTS ways with cabal-install

[hsyl20]

The RTS can be built using several ABI compatible ways. These are called RTS ways in the compiler and apparently flavours in Cabal. GHC can switch between them at program link time (so independently of any notion of a cabal package, so they shouldn't be represented as different installed packages to cabal afaict).

What happens with Hadrian:

• It builds the RTS more than once, with different flags
• It uses suffixes to identify the ways (e.g. "_thr" for the THREADED way) in the filenames of the rts libraries
• These libraries are put alongside the normal ("vanilla") one in the rts lib dir

When we build with cabal-install, cabal-install has to do this... Luckily Cabal has some notion of ways called "flavours" which are even documented in the manual:

Of course...

If we try to enable one (e.g. extra-library-flavours: _thr) and run the script in #3 we get:

Installing library in /home/hsyl20/.cabal/store/ghc-9.13-inplace/incoming/new-60663/home/hsyl20/.cabal/store/ghc-9.13-inplace/rts-1.0.3-6722ce6fbfa2d1ba89ce151d7d3f1c2c172ab3e37bf54cd952c07b60b3ca51bc/lib
/home/hsyl20/projects/ghc/stable/_build/stage1/cabal/tmp/src-60663/rts-1.0.3/dist/build/libHSrts-1.0.3-6722ce6fbfa2d1ba89ce151d7d3f1c2c172ab3e37bf54cd952c07b60b3ca51bc_thr.a: copyFile: does not exist (No such file or directory)

So apparently cabal doesn't build the different flavours but expect the resulting libraries to be present.

We probably need to design something in cabal to handle the flavours in a better way.

Currently the RTS contains:

      -- Here we declare several flavours to be available when passing the
      -- suitable (combination of) flag(s) when configuring the RTS from hadrian,
      -- using Cabal.
      if flag(threaded)
        extra-library-flavours: _thr
        if flag(dynamic)
          extra-dynamic-library-flavours: _thr

      if flag(profiling)
        extra-library-flavours: _p
        if flag(threaded)
          extra-library-flavours: _thr_p
        if flag(debug)
          extra-library-flavours: _debug_p
          if flag(threaded)
            extra-library-flavours: _thr_debug_p
        if flag(dynamic)
          extra-dynamic-library-flavours: _p
          if flag(threaded)
            extra-dynamic-library-flavours: _thr_p
          if flag(debug)
            extra-dynamic-library-flavours: _debug_p
            if flag(threaded)
              extra-dynamic-library-flavours: _thr_debug_p

      if flag(debug)
        extra-library-flavours: _debug
        if flag(dynamic)
          extra-dynamic-library-flavours: _debug
        if flag(threaded)
          extra-library-flavours: _thr_debug
          if flag(dynamic)
            extra-dynamic-library-flavours: _thr_debug

So I think we just need to add:

1. Flavour specific options

if flavour(_debug) || (_thr_debug) || ...
  ghc-options: -DDEBUG
if flavour(_thr) || (_thr_debug) || ...
  ghc-options: -DTHREADED

It would make flavours just like flags but invisible at the package level.

2. cabal-install feature to build libraries once per extra flavour

3. Optional: allow disabling the default flavour ("vanilla")? E.g. on ArchLinux they don't want static linking.

4. Optional: store the list of enabled flavours in the installed package config so that GHC can detect what is enabled (it can also detect which library files exist, so it's not mandatory).

[hsyl20]

@andreabedini @angerman Any opinion on this?

[hsyl20]

Note that "profiling" shouldn't be a cabal flavour, because it isn't ABI compatible with the vanilla flavour. Enabling profiling should be like switching the bignum backend: all the dependents have to be rebuilt when this kind of setting changes.

[angerman]

We do have flavours and ways, where ways try to distinguish abi incompatibility, while flavours don't right? This seems hoplessly confusing, also flavours seem to mostly designate "ways-to-build-ghc" (default, quick, quickest, ...).
While cabal calls "flavors" what is effectively (vanilla, threaded, debug, and profiling). As @hsyl20 mentioned profiling is a bit of a special case as it breaks abi. And then we also mix in linking with this 🙄

We have one dimension of abi incompatibilities (vanilla, profiled, bignum(?)); I'd also argue we should free llvm from the abi compatiblity requirements with the vanilla build. It would allow us to actually build the LLVM backend properly. It does have some benefits during boostrapping NCGs though I'll admit that.
We have two dimension of abi compatible, but different rts (debug, threaded).
And we have one dimension of linking (static, dynamic).

And then we have a few variations of this where we set different optimisation flags. (default, quick, quickest, perf, release, assert/debug).

And then we technically have all of this for the matrix of (host, target).

If we look at the RTS from the debug, threaded matrix we have, four configurations. And these are a bit special, because no one really depneds on the rts package, GHC will just inject it and then change the suffix based on -debug and -threaded being passed to GHC. It's as such much closer to a link time flag.

This also provides one benefit, we could have different rts packages, and change the logic in GHC to do the suffix swap, with just swapping out the rts package. As there is no dependency tree that depends on the rts. Otherwise this looks like a usecase for (in principle) backpack?

Does iserv vs no-iserv also consitute some dimension?

[angerman]

After spending quite some time reading GHCs source code, I think it might be viable to compile the RTS in various configurations:

• rts
• rts +threaded
• rts +debug
• rts +threaded +debug

as separate cabal package, but setting different unit-ids (rts, rts-threaded, rts-debug, rts-threaded-debug), and then have GHC load the respective RTS to write in instead of doing the way-suffix logic. The RTS is fairly special in this regard, as it's an implicit dependency. There are no packages that depend on the rts in their cabal file. It might also mean, we don't even need to ship all RTSs alongside the compiler and an end-user could just download and compile the rts on-demand for e.g. threaded, or debug builds if they so need. (Or single-threaded, if we default to shipping the threaded rts).

From a cursery look across the codebase, I believe this would end up in a net negative contribution to GHC, as a lot of custom handling of the RTS can potentially be removed.

[hsyl20]

After spending quite some time reading GHCs source code, I think it might be viable to compile the RTS in various configurations as separate cabal package, but setting different unit-ids (rts, rts-threaded, rts-debug, rts-threaded-debug),

How do you see it working more concretely?

1. if we add cabal flags to the rts to build the different variants, we end up with several rts-132456 units. Good.
2. if cabal makes the selection between the ABI compatible rts', it will rebuild the dependents unnecessarily (it doesn't know they are ABI compatible): not good, so GHC should make the decision, or we should be ok with rebuilding everything.
3. how should GHC select between the different rts units? cabal-install will only expose one of the rts units (it hides everything else afaik).

There are no packages that depend on the rts in their cabal file.

At least some boot packages depend on it (e.g. ghc-internal). We might still implicitly substitute another rts package in GHC though.

[angerman]

At least some boot packages depend on it (e.g. ghc-internal). We might still implicitly substitute another rts package in GHC though.

    build-depends:
        rts == 1.0.*

thank you for pointing that out! Indeed I did miss that.

ghc-internal does seem indeed to be the only one though.

$ rg " rts =="
libraries/ghc-internal/ghc-internal.cabal.in
122:        rts == 1.0.*

Let's discuss this later today :-)