exactDeps has bad interaction with source-repository-package and shellFor

[michaelpj]

I've observed what I think is the following interaction:

Project P has

1. A shell with exactDeps set
2. A package A which depends on B
3. A source-repository-package stanza patching C which is a dependency of B

In this scenario, cabal will always want to rebuild B. There is no way to convince it to take a prebuilt B from a package-db short of removing the source-repository-package stanza. But setting exactDeps says "we don't want cabal to build anything", and indeed this will "work": cabal will decide it wants to build B, but fail because it has no repositories ("unknown package B").

(In fact, we're clever enough to not even bother putting our prebuilt B into the shell, because we know cabal will ignore it. This can seem a little surprising if you're not expecting B to be missing!)

I think this is inescapable: cabal will absolutely insist on building B, and exactDeps won't let it. If we just wanted exactDeps to say "don't let cabal hit the network", then we could point it at a local repository instead, perhaps.

My preferred solution: don't use exactDeps.

[amesgen]

Another option FTR if one wants to use exactDeps + source-repository-package: Using iohk-nix's cabal-wrapper which provides the following workaround:

• You have to put the source-repository-packages at the bottom of your cabal.project, preceded by a special marker comment.
• Add the above cabal-wrapper to your shell. When you now call cabal, it creates .nix-shell-cabal.project, containing the original cabal.project without the source-repository-package stanzas.
• This wrapped cabal always uses this special project file, via --project-file=/path/to/.nix-shell-cabal.project.

Also see here and here for how this looks in practice.

In my experience, this works fairly well, but YMMV.

[michaelpj]

This ticket is partly a result of getting rid of the cabal wrapper :)

You missed a step: you also have to hack up your shellFor to cancel out the aforementioned haskell.nix cleverness. So it's overall a lot of hacks.

[amesgen]

Right, thanks for catching that, I guess I performatively proved the point that the situation here could be improved 😄

Concretely, for a shell for all components, one has to add

packages = ps:
  lib.attrValues (pkgs.haskell-nix.haskellLib.selectProjectPackages ps);

to the args to shellFor, which then makes the source-repository-packages appear in the shell.

[L-as]

If you add the stanzas in cabalProjectLocal when you call cabalProject', you avoid the problem entirely.

[L-as]

We can make a wrapper for cabal that ignores the stanzas before passing it on to the real cabal. You could use some overlayfs magic.

[L-as]

But just fixing it upstream seems easier.

[michaelpj]

Given that the goal of haskell.nix is to follow Cabal as much as possible, these workarounds are pretty undesirable. Often people want their cabal.project to work with bare Cabal and no Nix as well.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[qwbarch]

Still relevant

[ParetoOptimalDev]

For anyone wanting to implement this take a look at this related issue with commentary from Cabal developers:

haskell/cabal#8434 (comment)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ju1m]

If that may help anyone, here's my understanding and what I came up with, rediscovering @amesgen workarounds:

project.shellFor {
  # Force cabal-install (by setting CABAL_CONFIG=)
  # to use the packages selected by project.plan-nix.
  exactDeps = true;

  # Provision (in `ghc-pkg list`) the **dependencies** of the packages selected here,
  # which are not also selected here.
  # By default haskell.nix selects here all _local_ packages
  # (packages from both the packages and the source-repository-package stanzas)
  # which therefore excludes source-repository-packages from being provisioned.
  # Hence select only the _project_ packages (packages from the packages stanza).
  packages = ps: lib.attrValues (pkgs.haskell-nix.haskellLib.selectProjectPackages ps);

  nativeBuildInputs = [
    # Wraps cabal-install to remove source-repository-package stanzas,
    # otherwise it would download an use them,
    # likely because it does not know that those packages provisioned
    # in `ghc-pkg list` must be used in priority.
    (let cabal-install = project.tool "cabal" {
      version = "latest";
      #inherit index-state;
      modules = [
        {
          # Preserve Bash completion for cabal.
          packages.cabal-install.components.exes.cabal.postInstall = ''
            mkdir -p $out/share/bash-completion
            mv bash-completion $out/share/bash-completion/completions
          '';
        }
      ];
    };
    in
    # symlinkJoin cabal-install to preserve share/bash-completion/
    pkgs.symlinkJoin {
      name = "cabal-install";
      paths = [ cabal-install ];
      nativeBuildInputs = [ pkgs.makeWrapper ];
      # Wrap cabal-install to make it use the local cabal.project.flake.
      postBuild = ''
        rm "$out"/bin/cabal
        makeWrapper \
          ${cabal-install}/bin/cabal \
          "$out"/bin/cabal \
          --add-flags --project-file=cabal.project.flake
      '';
    })
  ];

  # cabal.project.flake must be in the same directory than the original cabal.project
  # for the relative paths in the packages stanza to point to the correct subdirs.
  shellHook = ''
    ln -sf ${pkgs.runCommandLocal "cabal.project" {
          nativeBuildInputs = with pkgs; [ gawk ];
        }
        # Strips source-repository-package stanzas from cabal.project.
        ''
          awk '/^[^ ]/{f=/^source-repository-package/} !f' >$out ${./cabal.project}
        ''
      } cabal.project.flake
  '';
};