Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IFD-less evaluation of local packages #144

Open
srid opened this issue Apr 13, 2023 · 4 comments · May be fixed by #231
Open

IFD-less evaluation of local packages #144

srid opened this issue Apr 13, 2023 · 4 comments · May be fixed by #231
Assignees
@srid
Labels
enhancement New feature or request package-set Should haskell-flake allow creating package sets, decoupled from projects? question Further information is requested

Comments

@srid
Copy link
Owner

srid commented Apr 13, 2023 •
edited
Loading

IFD can really slow down the evaluation of the flake, such as to make the launching of nix develop noticeably slow -- this is currently the case for https://github.com/nammayatri/nammayatri which has gazillion local packages with lots of dependencies.

The idea here is to do something similar to the hpack -> cabal workflow, but for cabal -> nix. ie., use cabal2nix to generate a default.nix in the local package directory (example), for which we provide a nix run'able flake app that the developer will run anytime changing the .cabal file. There will also be a flake check (analogous to #97) to make sure that generated default.nix is in sync with the .cabal file (useful in CI). (pre-commit-hooks-nix can do this).

We could actually rename the default.nix to something unique to haskell-flake: haskell-package.nix.

Then, the following configuration:

{
  haskellProjects.default = {
    packages = {
      foo.source = ./foo;
    };
}

... would look for haskell-packages.nix and use that; if not, it looks for the .cabal file falling back to using IFD (callCabal2nix).

cc @roberth @locallycompact @NorfairKing for feedback/suggestions/ideas.
@srid srid added enhancement New feature or request question Further information is requested labels Apr 13, 2023
@srid
Copy link
Owner Author

srid commented Apr 13, 2023 •
edited
Loading

Alternatively, is it possible to make callCabal2nix's drv re-evaluate only when the cabal file changes (less often), but not when the sources do (more often)?

@locallycompact
Copy link

The hpack -> cabal : cabal -> nix analog is how horizon-gen-nix works currently, although it only works by targetting remote urls, tarballs and hackage. Horizon did have FromLocal constructor at one point that could be used for overlays using local repositories.

https://gitlab.horizon-haskell.net/dhall/horizon-spec/-/blob/e7f19fc16cb5b47773bcca49fb1cab0d5c0011b9/horizon-spec/package.dhall#L23

But I got rid of it because I didn't have the bandwidth to think about how the pathing should work (pkgs/myPkg.nix should refer to ../something/myPkg or so) and also the workflow was rubbish without a way to check for staleness of the data. In the case of hpack staleness is obvious because the cabal file can be regenerated purely and cheaply, but in the case of cabal2nix over a gazillion projects - checking for staleness is just as expensive as the IFD it replaces. A lockfile would also confuse the matter here, where as a lockfile for urls, tarballs and hackage works fine because any change to those would dirty the lockfile and hence trigger a new cabal2nix regen for that item, where as with a FromLocal, the entry wouldn't change, so it would still require a call to cabal2nix everytime. So I think IFD is the only real option for local packages unfortunately.

Example of a horizon overlay style here: https://gitlab.horizon-haskell.net/package-sets/horizon-plutus-apps

@NorfairKing
Copy link

@srid I use https://github.com/cachix/pre-commit-hooks.nix to keep the default.nix up to date so that there is no IFD.

This was referenced Apr 14, 2023
Closed
Closed
@srid srid added the package-set Should haskell-flake allow creating package sets, decoupled from projects? label May 12, 2023
@srid srid mentioned this issue Oct 25, 2023
Draft
25 tasks
@srid srid pinned this issue Nov 1, 2023
@srid srid mentioned this issue Nov 11, 2023
Closed
@srid srid self-assigned this Dec 13, 2023
@srid
Copy link
Owner Author

srid commented Jan 9, 2024

Zulip discussion: https://nixos.zulipchat.com/#narrow/stream/413949-haskell-flake/topic/Removing.20IFD

@srid srid linked a pull request Apr 25, 2024 that will close this issue
Draft
@srid srid mentioned this issue Sep 6, 2024
Closed
@rsrohitsingh682 rsrohitsingh682 mentioned this issue Dec 8, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@srid srid

Labels
enhancement New feature or request package-set Should haskell-flake allow creating package sets, decoupled from projects? question Further information is requested
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Cache cabal2nix expr to avoid IFD srid/haskell-flake
3 participants
@srid @locallycompact @NorfairKing