Skip to content

Rebuilds of build.rs crate after 1.69 when building Nix #12603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dpc opened this issue Aug 30, 2023 · 1 comment
Closed

Rebuilds of build.rs crate after 1.69 when building Nix #12603

dpc opened this issue Aug 30, 2023 · 1 comment
Labels
C-bug Category: bug S-triage Status: This issue is waiting on initial triage.

Comments

@dpc
Copy link
Contributor

dpc commented Aug 30, 2023

Problem

Hi. I'm reporting as a Bug Report, but actually might not be cargo's "fault", though I suspect something is off somewhere because the issue is somewhat random (affects only certain projects). I'm looking for help trying to figure out why cargo's behavior changed in version 1.69 which is causing it to rebuild things that in 1.68 would not need rebuilding.

In Nix world when we build Rust code, we use various tools to trick cargo into playing nice with a sandbox. This is what tools like crane are for.

Among other things, what crane does is something that tools like cargo-chef are doing - storing and restoring ./target directory in cache, to allow "checkpointing" of a build. Eg. you build all the dependencies only as a separate step, save that state, then make the normal build starting from that saved state. This way CIs etc. can usually reuse parts of the build that typically do not change.

Recently I've noticed (and now other people are reporting) that after updating the toolchain, all the crates that use build.rs scripts rebuild every time, after every restoration, ruining advantages of caching.

Surprisingly when trying to reproduce on a minimized project, it did not reproduce.

I used CARGO_LOG=cargo::core::compiler::fingerprint=trace to try to figure the difference.

Affected:

[2023-08-23T18:58:43Z DEBUG cargo::core::compiler::fingerprint] old local fingerprints deps "/nix/store/8xlxixbffyyra5w4xv8fabyfcwynfdh0-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/proc-macro2-1.0.66" precalculated="1.0.66"

Not affected:

quick-start-build> [2023-08-23T19:18:54Z DEBUG cargo::core::compiler::fingerprint] new local fingerprints deps "/nix/store/wzs1f02h3xw6i0ql0a12r6kbla1savr2-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/proc-macro2-1.0.66"

which reading the code suggests that in the affected build cargo somehow lost the information proc-macro2 does output rerun-if-changed=build.rs every time and is the 'new-style' custom build.

At this point however it's very hard for me to figure out why would that be the case.

Notably Nix&crane reset the timestamps on artifacts in ./target to Unix epoch (to make things reproducible and not depend on current time). However this was not causing problems in cargo 1.68, and is being done all the same in both affected and unaffected projects using 1.69 and later.

Anyway - if anyone can throw me some pointers how would I go about debugging it, or has any ideas, I would appreciate it. If nothing else - maybe this issue can ever help anyone trying to correlate some other reports. etc.

The corresponding crane issue is here.

Edit:
I've been doing all sorts of experiments trying to debug the issue and it seems that the problem gone in a release build.

**Edit 2::
It seems it's all about debug symbols. debug = false fixes the issue in any build profile.

Steps

No response

Possible Solution(s)

No response

Notes

No response

Version

No response

@dpc dpc added C-bug Category: bug S-triage Status: This issue is waiting on initial triage. labels Aug 30, 2023
@dpc
Copy link
Contributor Author

dpc commented Sep 16, 2023

It was actually #12457

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: bug S-triage Status: This issue is waiting on initial triage.
Projects
None yet
Development

No branches or pull requests

1 participant