[WIP] chore(packaging): use nix flakes for building graphql-engine reproducibly #9160

thenonameguy · 2022-10-31T00:24:02Z

@SamirTalwar I saw your work on adding the scaffolding for the development environment and decided to add Nix for building aarch64-darwin binaries out of graphql-engine. This project is a WIP branch that does everything without much regard to existing practices/best conventions to do a POC for Nix + Hasura => M1 native gql-engine.

Description

Changelog

Component : build

Type: enhancement

Product: community-edition

Short Changelog

Allow building reproducibly graphql-engine via Nix

Long Changelog

Related Issues

Solution and Design

Steps to test and verify

Limitations, known bugs & workarounds

Server checklist

Catalog upgrade

Does this PR change Hasura Catalog version?

No
Yes
- Updated docs with SQL for downgrading the catalog

Metadata

Does this PR add a new Metadata feature?

No
Yes
- Does run_sql auto manages the new metadata through schema diffing?
  - Yes
  - Not required
- Does run_sql auto manages the definitions of metadata on renaming?
  - Yes
  - Not required
- Does export_metadata/replace_metadata supports the new metadata added?
  - Yes
  - Not required

GraphQL

No new GraphQL schema is generated
New GraphQL schema is being generated:
- New types and typenames are correlated

Breaking changes

hasura-bot · 2022-10-31T00:24:06Z

Beep boop! 🤖

Hey @thenonameguy, thanks for your PR!

One of my human friends will review this PR and get back to you as soon as possible.

Stay awesome! 😎

CLAassistant · 2022-10-31T00:24:08Z

All committers have signed the CLA.

SamirTalwar

Looks like an awesome start. I've left a few comments. Can I help with anything?

SamirTalwar · 2022-10-31T10:26:59Z

nix/overlays/packages.nix

+              resource-pool = hself.hasura-resource-pool;
+            })));
+
+          # FIXME: for remote repos we have to calculate their default.nix until they get upstreamed/maintained


These seem legit. I suggest moving the actual fetches into the inputs in flake.nix though, so they're pinned with flake.lock. (You can still specify the exact commit ref.)

Yup, that's the plan. I need to do this anyway to not use IFDs.

SamirTalwar · 2022-10-31T10:35:39Z

server/CURRENT_VERSION

@@ -0,0 +1 @@
+2.14.0-beta.2


Please don't check this in. For some reason, this is missing from the .gitignore file; I am remedying this.

For local development, you can write "12345" to this file.

Flake builds can only use files that are tracked in version control (as the build depends on this not-deterministic file)

I thought the version was used to fetch the CDN FE assets for the Console, is this not correct?

Apologies, I thought I'd replied to your comments.

I suggest adding a preConfigure step that writes the file with the contents of the package version then.

SamirTalwar · 2022-10-31T10:37:21Z

server/default.nix

@@ -0,0 +1,112 @@
+{ mkDerivation, aeson, aeson-casing, aeson-ordered, aeson-pretty


Any reason not to use callCabal2nix for these?

If it's just for the version, you can override that later. I think this will work:

(callCabal2nix "graphql-engine" ./server {}).overrideAttrs(oldAttrs: { version = builtins.readFile ./server/CURRENT_VERSION; })

callCabal2nix is an anti-pattern in the flake/nixpkgs world, as it creates evaluations at runtime.
The computation of these files, while being more convenient, are done at eval time.
https://github.com/input-output-hk/haskell.nix uses this approach extensively, I tried using that on a different branch in the past days and the IFD-less version of this branch is way more snappier to eval/build with.
IMO for now it's worth to do this outside of Nix evalution, maybe automate it with a Makefile/CI script.

Further reading:
https://github.com/input-output-hk/haskell.nix/blob/master/docs/tutorials/materialization.md

Very helpful, thank you!

We'd need to add some CI to make sure this stays in sync with however it's generated, as well as include instructions on how to redo it.

We'd need to add some CI to make sure this stays in sync with however it's generated, as well as include instructions on how to redo it.

You can include a pre-commit hook that runs cabal2nix.

Note that devs don't have to install the hooks themselves, but only have to run pre-commit run after staging the changes. Else, nix flake check should be expected to fail.

cf. srid/haskell-flake#144 (comment)

SamirTalwar · 2022-10-31T10:38:00Z

server/graphql-engine.cabal

@@ -158,7 +158,8 @@ flag ghci-load-test-with-lib
 common common-all
  ghc-options:
    -foptimal-applicative-do
-    -Wall -Wcompat -Wincomplete-record-updates -Wincomplete-uni-patterns -Wredundant-constraints
+    -Wall -Wcompat -Wincomplete-record-updates -Wincomplete-uni-patterns
+    -- -Wredundant-constraints


Why did you have to remove this? Did you get extra warnings?

Yes, got a lot of compilation errors for redundant-constrains and typed holes, which I found strange. Temporary fix to get a build.

SamirTalwar · 2022-10-31T10:38:09Z

server/graphql-engine.cabal

@@ -293,6 +294,7 @@ common lib-depends
                     , some
                     , split
                     , template-haskell
+                     , ghc-heap-view


I assume this change is just for testing.

your CPP module was essentially upstreamed into the latest ghc-heap-view version. The surrounding code failed to compile until these changes were made. Of course the runtime dependency for this should be made optional, but Nix did not seem to find it using the conditional profiling include.

@jberryman: Do you think we can upgrade ghc-heap-view to take advantage of this?

thenonameguy · 2022-10-31T13:17:13Z

Looks like an awesome start. I've left a few comments. Can I help with anything?

Thanks! Currently:

x86_64-linux: builds
aarch64-darwin: compiles but linking segfaults, same as Haskell language server fails to build on aarch64-darwin NixOS/nixpkgs#194367 (comment)
x86_64-darwin: is compiling as we speak with Rossetta 2, should work.

For my personal purposes getting the aarch64-darwin building would be an end-goal for now.
Then we could work backwards, removing the need for IFDs, removing compilation errors from the Nix build, that don't happen in the shell cabal build scenarios, and get this mainlined if the Hasura team agrees.

SamirTalwar · 2022-11-02T23:07:59Z

We saw those same linking issues trying to build Ormolu with Nix, which was fixed by disabling the Template Haskell.

My guess is that it's similar here; the Template Haskell is causing serious problems somewhere. I'm not sure what to do about it, unfortunately. It definitely feels like an upstream bug, but I don't even know where. Is it with the version of ld shipped with nixpkgs on macOS/aarch64, or somewhere else?

What do you propose we do about it?

domenkozar · 2023-01-19T11:56:24Z

This would be so nice to have, I'd like to support Hasura in devenv.sh: cachix/devenv#308

SamirTalwar · 2023-01-19T17:03:24Z

@domenkozar, I take it your main motivation is Linux support, not macOS?

If we were to reorient this PR to just care about Linux, it might be easier to get it in.

domenkozar · 2023-01-19T17:41:52Z

Linux and macOS, I'm happy to help here if I can understand what are the blockers.

SamirTalwar · 2023-01-19T20:09:41Z

The main macOS blocker is that ld chokes on HGE. The error is the same as NixOS/nixpkgs#149692.

We would also need to clean this PR up a bit, but it's mostly small things; @thenonameguy has done a wonderful job.

thenonameguy · 2023-01-19T20:32:02Z

Cheers, FYI at my company have been using this branch + Cachix binary cache for our local Hasura development ever since without hiccups. I propose you take over the necessary bits and pieces over to a new branch as I do not have the time for foreseeable future to improve this further.

Super stoked to see this initiative going forward!

SamirTalwar · 2023-01-19T21:13:27Z

Thanks @thenonameguy, glad it's working well at your end. We'll do our best to incorporate it.

john-rodewald · 2023-06-13T15:38:10Z

I picked up where @thenonameguy left off, transformed my fork into a monstrosity of patches that I don't really want anyone to see, and eventually managed to build on aarch64-darwin with Nix. That means I can ditch Docker now. I'm not sure when or if I'll have time to clean it up, but it runs and may be helpful to someone looking into this PR.

https://github.com/john-rodewald/graphql-engine/tree/master-nix

the-sun-will-rise-tomorrow · 2024-11-18T22:05:50Z

server/lib/error-message/default.nix

+    aeson base graphql-parser hashable hspec text unordered-containers
+    vector
+  ];
+  homepage = "https://github.com/hasura/github-engine#readme";


s/github-engine/graphql-engine/ ?

the-sun-will-rise-tomorrow · 2024-11-18T22:07:54Z

@john-rodewald FYI, nix run github:john-rodewald/graphql-engine/master-nix now fails with error: Package ‘openapi3-3.2.3’ in /nix/store/qmyflblrsa4qikr5q54cfk0hxqx2a5p6-source/pkgs/development/haskell-modules/hackage-packages.nix:213450 is marked as broken, refusing to evaluate..

the-sun-will-rise-tomorrow · 2024-11-18T22:09:45Z

The main macOS blocker is that ld chokes on HGE. The error is the same as NixOS/nixpkgs#149692.

It looks like that problem may have been fixed since then.

thenonameguy added 6 commits October 30, 2022 18:25

Generate cabal2nix > default.nix + overlays for subpackages

572d6bf

Almost compiling graphql-engine

ddd579f

resource-pool misery

3ddebab

building pg-client-hs

f02b569

Add hasura-odbc

3160e02

1 step away from Nix compiling gql-engine

53075cf

thenonameguy added 2 commits October 31, 2022 02:01

Fix ghc-heap-view missing include

357f11f

working nix run .

9ea39ad

SamirTalwar reviewed Oct 31, 2022

View reviewed changes

thenonameguy mentioned this pull request Oct 31, 2022

Hasura-graphql-engine NixOS/nixpkgs#181630

Open

thenonameguy mentioned this pull request Jan 19, 2023

Add Hasura support cachix/devenv#308

Open

the-sun-will-rise-tomorrow reviewed Nov 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] chore(packaging): use nix flakes for building graphql-engine reproducibly #9160

[WIP] chore(packaging): use nix flakes for building graphql-engine reproducibly #9160

thenonameguy commented Oct 31, 2022 •

edited

Loading

hasura-bot commented Oct 31, 2022

CLAassistant commented Oct 31, 2022 •

edited

Loading

SamirTalwar left a comment

SamirTalwar Oct 31, 2022

thenonameguy Oct 31, 2022

SamirTalwar Oct 31, 2022

thenonameguy Oct 31, 2022

SamirTalwar Jan 19, 2023

SamirTalwar Oct 31, 2022

thenonameguy Oct 31, 2022

thenonameguy Oct 31, 2022

SamirTalwar Jan 19, 2023

srid Oct 25, 2023 •

edited

Loading

SamirTalwar Oct 31, 2022

thenonameguy Oct 31, 2022

SamirTalwar Oct 31, 2022

thenonameguy Oct 31, 2022

SamirTalwar Jan 19, 2023

thenonameguy commented Oct 31, 2022 •

edited

Loading

SamirTalwar commented Nov 2, 2022

domenkozar commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

domenkozar commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

thenonameguy commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

john-rodewald commented Jun 13, 2023 •

edited

Loading

the-sun-will-rise-tomorrow Nov 18, 2024

the-sun-will-rise-tomorrow commented Nov 18, 2024

the-sun-will-rise-tomorrow commented Nov 18, 2024

		@@ -0,0 +1,112 @@
		{ mkDerivation, aeson, aeson-casing, aeson-ordered, aeson-pretty

		@@ -0,0 +1 @@
		2.14.0-beta.2

[WIP] chore(packaging): use nix flakes for building graphql-engine reproducibly #9160

Are you sure you want to change the base?

[WIP] chore(packaging): use nix flakes for building graphql-engine reproducibly #9160

Conversation

thenonameguy commented Oct 31, 2022 • edited Loading

Description

Changelog

Short Changelog

Long Changelog

Related Issues

Solution and Design

Steps to test and verify

Limitations, known bugs & workarounds

Server checklist

Catalog upgrade

Metadata

GraphQL

Breaking changes

hasura-bot commented Oct 31, 2022

CLAassistant commented Oct 31, 2022 • edited Loading

SamirTalwar left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

srid Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thenonameguy commented Oct 31, 2022 • edited Loading

SamirTalwar commented Nov 2, 2022

domenkozar commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

domenkozar commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

thenonameguy commented Jan 19, 2023

SamirTalwar commented Jan 19, 2023

john-rodewald commented Jun 13, 2023 • edited Loading

Choose a reason for hiding this comment

the-sun-will-rise-tomorrow commented Nov 18, 2024

the-sun-will-rise-tomorrow commented Nov 18, 2024

thenonameguy commented Oct 31, 2022 •

edited

Loading

CLAassistant commented Oct 31, 2022 •

edited

Loading

srid Oct 25, 2023 •

edited

Loading

thenonameguy commented Oct 31, 2022 •

edited

Loading

john-rodewald commented Jun 13, 2023 •

edited

Loading