Skip to content

rust-lld on NixOS can't link the program if the target dir lives on a path with spaces in it #138513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
TheAwesome98-Real opened this issue Mar 14, 2025 · 21 comments
Open

rust-lld on NixOS can't link the program if the target dir lives on a path with spaces in it #138513

TheAwesome98-Real opened this issue Mar 14, 2025 · 21 comments
Assignees
@Noratrieb
Labels
A-linkage Area: linking into static, shared libraries and binaries C-external-bug Category: issue that is caused by bugs in software beyond our control O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@TheAwesome98-Real
Copy link

Code

any code works, even that which doesn't link to c libraries. to reproduce it, you can try this:

mkdir -p "/tmp/test dir/testing"
cd "/tmp/test dir/testing"
cargo init
cargo build

i expected it to compile, but it said this:

   Compiling testing v0.1.0 (/tmp/test dir/testing)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "-m64" "/tmp/rustcucYSDn/symbols.o" "<7 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,libcfg_if-*,liblibc-*,liballoc-*,librustc_std_workspace_core-*,libcore-*,libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcucYSDn/raw-dylibs" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,-znostart-stop-gc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/tmp/test dir/testing/target/debug/deps/testing-a26199dce8a80c8e" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.1f4zjopx4co8wg8v0m5gp177f.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.6rob1l3zpd0lgrz2ghmpkidqj.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.784v8tgyr5k5uima1n1d3z0mk.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.883o2n7eztjk0o7arvymjht7i.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.94740eyfkqulmvhrsi1fqyy0q.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.apbnli2khxbnh4tqkgi8ctsp6.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-a26199dce8a80c8e.budf0e3pu8uovxzzurmmql825.rcgu.o: No such file or directory
          collect2: error: ld returned 1 exit status


error: could not compile `testing` (bin "testing") due to 1 previous error

Version it worked on

according to the .comment section of one of my applications, it was rustc version 1.85.0-nightly (d4025ee45 2024-12-12), but this is not likely the latest version that successfully compiles it, as i hadn't updated rust in a while, as indicated by the last year build date in the version.

Version with regression

rustc --version --verbose:

rustc 1.87.0-nightly (cbfdf0b01 2025-03-13)
binary: rustc
commit-hash: cbfdf0b014cb04982a9cbeec1578001001167f6e
commit-date: 2025-03-13
host: x86_64-unknown-linux-gnu
release: 1.87.0-nightly
LLVM version: 20.1.0

the regression occurred between 2 nightly versions. i don't know what to do in that case
@TheAwesome98-Real TheAwesome98-Real added C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels Mar 14, 2025
@rustbot rustbot added I-prioritize Issue: Indicates that prioritization has been requested for this issue. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Mar 14, 2025
@Noratrieb
Copy link
Member

Noratrieb commented Mar 14, 2025 •
edited
Loading

Thank you for the report! It would be useful to bisect the regression to the exact commit using cargo-bisect-rustc to make it easier to figure out what happened and ping the relevant people.
@rustbot label E-needs-bisection

@rustbot rustbot added the E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc label Mar 14, 2025
@Noratrieb Noratrieb added A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. and removed E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Mar 14, 2025
@lqd
Copy link
Member

lqd commented Mar 15, 2025

This issue doesn't reproduce for me locally, does it need more than a hello world?

I don't think we can prioritize yet, let's do that once we have a bisection if it's a regression, and more information on what's needed to reproduce.

@jieyouxu jieyouxu added the S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. label Mar 15, 2025
@TheAwesome98-Real
Copy link
Author

does it need more than a hello world?

nope, i assumed it was something to do with ffi (i remember this being a problem with a -sys crate before) but it happens on hello world too.

i'll try to bisect rustc to see exactly which commit this starts happening on

@TheAwesome98-Real
Copy link
Author 

$ cargo bisect-rustc --start=2024-12-12 --end=2025-03-13
checking the start range to find a passing nightly
installing nightly-2024-12-12
rust-std-nightly-x86_64-unknown-linux-gnu: 29.32 MB / 29.32 MB [===========================================================================] 100.00 % 1.21 MB/s testing...
RESULT: nightly-2024-12-12, ===> Compile error
uninstalling nightly-2024-12-12

ERROR: the start of the range (nightly-2024-12-12) must not reproduce the regression

false alarm, not a regression. would appreciate some help figuring out why rust suddenly isn't able to compile in directories with spaces though.

@Noratrieb Noratrieb removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. regression-untriaged Untriaged performance or correctness regression. labels Mar 15, 2025
@Noratrieb
Copy link
Member

I don't see anything wrong with the printed linker command args, nothing has been split incorrectly at that stage. It's hard to tell what's causing the issue without a way to reproduce it.

@TheAwesome98-Real
Copy link
Author

nothing has been split incorrectly at that stage

that was what i found the most strange—all paths are in double quotes, so there should be no problem

about reproducing: i'm using nixos, from which i got my copy of rustup, which i used to get cargo, rustc, and the other rust programs. i could copy my configuration file, cargo directory, and project shell file to a virtual machine. if the issue reproduces, it means the issue probably lies somewhere within nixos, and i can make a minimum reproducible configuration file that reproduces the problem so it can be fixed. if it doesn't, it means my setup is somehow broken on my specific install in a strange way. i don't know what i would do in that case

@Noratrieb
Copy link
Member

I see, how did you get your Rust on NixOS? There may be a patched script inserted that can't handle the spaces... I tried on my NixOS (via rustup from nixpkgs) and it worked (though I'm not fully sure if I disabled mold, which I use by default, correctly).

@Noratrieb Noratrieb changed the title regression: rust-lld can no longer link the program if the target dir lives on a path with spaces in it rust-lld on NixOS can't link the program if the target dir lives on a path with spaces in it Mar 15, 2025
@Noratrieb Noratrieb added the O-NixOS Operating system: NixOS, https://nixos.org/ label Mar 15, 2025
@jyn514
Copy link
Member

jyn514 commented Mar 16, 2025

that was what i found the most strange—all paths are in double quotes, so there should be no problem

the CLI args were shortened. notice how none of them contain .rcgu.o. if you compile with --verbose (as a rustc flag, not a cargo flag), it will print the full path.

@petrochenkov
Copy link
Contributor

that was what i found the most strange—all paths are in double quotes, so there should be no problem

the CLI args were shortened. notice how none of them contain .rcgu.o. if you compile with --verbose (as a rustc flag, not a cargo flag), it will print the full path.

This shortening was probably a bad idea.
You typically need to read these diagnostics when something wrong is happening, and in that case you don't want the shortened version.
I encountered it when debugging #137498 and I wasn't actually able to get the un-shortened version for rustc running inside bootstrap (IIRC, I had to modify src\bootstrap\src\bin\rustc.rs or compiler to get it).

@TheAwesome98-Real
Copy link
Author

how did you get your Rust on NixOS

just by adding rustup to my configuration file, and using it to install/update rust normally.

as a rustc flag, not a cargo flag

how do you provide flags to rustc when running cargo build?

@jyn514
Copy link
Member

jyn514 commented Mar 16, 2025 •
edited
Loading

I encountered it when debugging #137498 and I wasn't actually able to get the un-shortened version for rustc running inside bootstrap (IIRC, I had to modify src\bootstrap\src\bin\rustc.rs or compiler to get it).

you can pass flags to rustc with MAGIC_EXTRA_RUSTFLAGS (we should probably change the name). it would be nice if config.toml also let you pass --verbose specifically, without invalidating the cache.

how do you provide flags to rustc when running cargo build?

RUSTFLAGS=--verbose, or use cargo rustc --bin -- --verbose for better caching.

You typically need to read these diagnostics when something wrong is happening, and in that case you don't want the shortened version.

hmm. do you remember which part you needed, was it also the .rcgu.o object files? i'm fine with showing a short version of those (like we do for the sysroot rlibs), i just don't want to fully glob expand them.

@TheAwesome98-Real
Copy link
Author

haha i looked it up at the same time as you sent this comment

anyway, here's the output:

longish command-line output 
$ RUSTFLAGS=--verbose cargo build
   Compiling testing v0.1.0 (/tmp/test dir/testing)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/Users/lily/.cargo/bin:/Users/lily/.local/bin:/run/wrappers/bin:/Users/lily/.nix-profile/bin:/nix/profile/bin:/Users/lily/.local/state/nix/profile/bin:/etc/profiles/per-user/lily/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcnRBXWa/symbols.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.1bfyoq6rlzd3j6rlzixd6uskq.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.8g1cx5khh1sbf5bkfi8dem3ng.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.9ovjrihh4cz1ccqq6va1kyplk.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.btx0i6707le7wjb6vblydmhri.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.c5pedz95lu4ynzmy14tvxncrq.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.clas9xqc2eedytiy6rg88kkr9.rcgu.o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb.41nm33s6nku8i3qh7g76xp28m.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-168b2af3cdbd9a8b.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-50ee00a6fa5c5e39.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-e23f881d85955200.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-a49c48c72917b7fd.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-3ca1e42d18edf654.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-a781adec4bf6a44b.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-2968ce6079bb1f71.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-5e63c5d4a51bde95.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-054c4e8a42992d63.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-7b1b7572bf2bef56.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-b1741068faf841b8.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler2-f0b81b684679327b.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-b6c8d2f9686f2666.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-c93a46af375896b2.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-7a21846f18ba2952.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-96fab14884229a11.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-186a023b8864be69.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-74a8d2c13eeedc93.rlib" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-b294c1429f78dd04.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/tmp/rustcnRBXWa/raw-dylibs" "-B/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,-znostart-stop-gc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/tmp/test dir/testing/target/debug/deps/testing-d710013f68c27dbb" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.1bfyoq6rlzd3j6rlzixd6uskq.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.8g1cx5khh1sbf5bkfi8dem3ng.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.9ovjrihh4cz1ccqq6va1kyplk.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.btx0i6707le7wjb6vblydmhri.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.c5pedz95lu4ynzmy14tvxncrq.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.clas9xqc2eedytiy6rg88kkr9.rcgu.o: No such file or directory
          rust-lld: error: cannot open /tmp/test: No such file or directory
          rust-lld: error: cannot open dir/testing/target/debug/deps/testing-d710013f68c27dbb.41nm33s6nku8i3qh7g76xp28m.rcgu.o: No such file or directory
          collect2: error: ld returned 1 exit status


error: could not compile `testing` (bin "testing") due to 1 previous error

it just seems the same as the last output but with all of the object files listed explicitly

@petrochenkov
Copy link
Contributor

hmm. do you remember which part you needed

I didn't know exactly what I needed, I wanted to get the full versions of command lines (including escaping and similar things) and compare them between the good and the bad versions of the compiler.
(For that specific issue even the full command line didn't help, because the issue was with response files.)

@jyn514
Copy link
Member

jyn514 commented Mar 16, 2025

anyway, here's the output:

yeah that all looks correct, the .o files are quoted correctly.

what linker are you using? i see -fuse-ld=lld, but the error says rust-lld. you can see it using -C link-arg=-Wl,-v -W linker-messages (again, these are rustc flags, not cargo flags). for me that points to /home/jyn/.local/lib/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld, which is an ELF executable - is it an executable for you or a shell script?

@TheAwesome98-Real
Copy link
Author

yep, it's a shell script:

/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld: Bourne-Again shell script, ASCII text executable

the script itself is:

#!/usr/bin/env bash
set -eu -o pipefail +o posix
shopt -s nullglob
export PROG="/Users/lily/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld-unwrapped/ld.lld"
"/nix/store/dyn2kdxcnhcjz13nqpdrpcgd3qj7996b-rustup-1.27.1/nix-support/ld-wrapper.sh" $@

@jyn514
Copy link
Member

jyn514 commented Mar 16, 2025

yeah there's your problem, $@ needs to be quoted. probably this is a bug in the nix package for rustup, @Noratrieb would know more than i do

@jyn514
Copy link
Member

jyn514 commented Mar 16, 2025 •
edited
Loading

(i'm really curious to find out who knows enough bash to know that you should use $@ over $* but not that you need to quote it to see the benefit ... anyway this is bash's fault, you never want word expansion on $@.)

@Noratrieb
Copy link
Member

The bug is here: https://github.com/NixOS/nixpkgs/blob/272717e36b1398e357fd8cc03df8128730abd170/pkgs/development/tools/rust/rustup/0001-dynamically-patchelf-binaries.patch#L42

(I've personally never seen $*, only $@, so it seems completely understandable that someone would use $@ but be unaware that you need to quote it)

@Noratrieb Noratrieb added C-external-bug Category: issue that is caused by bugs in software beyond our control and removed C-bug Category: This is a bug. S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. labels Mar 16, 2025
@Noratrieb Noratrieb self-assigned this Mar 16, 2025
@Noratrieb
Copy link
Member

I'll fix it upstream

@Noratrieb Noratrieb mentioned this issue Mar 16, 2025
Open
13 tasks
@TheAwesome98-Real
Copy link
Author

probably this is a bug in the nix package for rustup

i wonder why this would suddenly be a problem. i don't remember upgrading (or i think i did, it just failed). either way i'm glad we figured out what the problem was ^^

@Noratrieb
Copy link
Member

You upgraded nightly probably, which made rustup execute the patching, and created the bad wrapper. Maybe you had a very old nightly before that didn't default to rust-lld yet and also didn't have the patching?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Noratrieb Noratrieb

Labels
A-linkage Area: linking into static, shared libraries and binaries C-external-bug Category: issue that is caused by bugs in software beyond our control O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@lqd @petrochenkov @jyn514 @jieyouxu @rustbot @Noratrieb @TheAwesome98-Real