Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git: fix darwin to linux cross-compilation #348010

Open
wants to merge 2 commits into
base: staging
Choose a base branch
from

Conversation

szlend
Copy link
Member

@szlend szlend commented Oct 11, 2024

Git uses uname for platform detection in some places which breaks cross-compilation from darwin to linux.

We can use the deterministic-host-uname package to replace uname with a mock mimicking the host's platform.

$ nix build -f . pkgsCross.aarch64-multiplatform.gitMinimal -L --print-out-paths
/nix/store/k25gxwp5xh06d70dxaw6ha33ajig5ssa-git-minimal-aarch64-unknown-linux-gnu-2.46.0

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@szlend szlend added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Oct 11, 2024
@emilazy
Copy link
Member

emilazy commented Oct 12, 2024

Just to check: We really want it for the host platform here, not the build platform? That’s kinda screwed up.

@szlend
Copy link
Member Author

szlend commented Oct 12, 2024

Just to check: We really want it for the host platform here, not the build platform? That’s kinda screwed up.

@emilazy When compiling natively, buildPlatform == hostPlatform, so there shouldn't be any issues.

However when cross-compiling, you can see that the configuration scripts rely on uname a lot to make architectural build decisions:

In these cases we want the configuration script to think that we're building for the host platform. From what I could gather there's no way to override this other than swapping in a mock uname.

That’s kinda screwed up.

Don't get me wrong, this is definitely a nasty hack and ideally upstream would make it possible to explicitly set the host platform. Though on the other hand it's kind of neat that you can just throw in a mock host-based uname and not have to worry about patching configuration scripts.

@emilazy
Copy link
Member

emilazy commented Oct 12, 2024

Yeah, okay. It might be better for us to patch those uname_* variables (or just set them in makeFlags? I forget if those take precedence over variables in the Makefile), and ideally upstream would fix this, but this minimal fix seems reasonable. This causes a lot of rebuilds, so please retarget to the staging branch. A comment explaining why it’s necessary would be great too.

@szlend
Copy link
Member Author

szlend commented Oct 12, 2024

Overriding these variables using makeFlags does work to my surprise.

Something like this:

makeFlags = [
  "prefix=\${out}"
  "uname_S=${deterministic-host-uname.uSystem}"
  "uname_M=${deterministic-host-uname.processor}"
  "uname_O=${deterministic-host-uname.operatingSystem}"
  "uname_R=${deterministic-host-uname.modDirVersion}"
]

Though I feel a bit uncomfortable relying on internal details of the deterministic-uname derivation. I'm not sure if this is actually much better. Maybe deterministic-uname could expose a "public API" through passthru? Happy to go with this approach if preferred though.

I could also compute these values myself, at the cost of adding a lot of complexity to an already large derivation.

@szlend szlend changed the base branch from master to staging October 12, 2024 16:37
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@emilazy
Copy link
Member

emilazy commented Oct 12, 2024

I’m fine with the current solution. If you could report it upstream to Git, that would be great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: cross-compilation Building packages on a different platform than they will be used on 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants