rustc cannot find crate if different target is used

[jpalus]

I have a self build of rust 1.73.0 on ARM/Linux configured to include two targets:

• armv7-unknown-linux-gnueabihf (Tier 2 with host tools)
• thumbv7neon-unknown-linux-gnueabihf (Tier 2 without host tools)

When building recent Mesa 23.2.1 for the latter target I've hit a build issue due to missing crate. The reduced reproducer is:

my_proc_macros.rs:

extern crate proc_macro;

lib.rs:

use my_proc_macros;

my_proc_macros.rs is compiled to shared object:

rustc --edition=2021  --crate-type proc-macro  --target thumbv7neon-unknown-linux-gnueabihf --crate-name my_proc_macros  -o libmy_proc_macros.so -C prefer-dynamic my_proc_macros.rs

while the lib.rs is compiled to static lib with use of above my_proc_macros:

rustc --edition=2021  --crate-type staticlib   --target thumbv7neon-unknown-linux-gnueabihf --crate-name app  -o lib.a --extern my_proc_macros=libmy_proc_macros.so lib.rs

The latter errors out with:

error[E0463]: can't find crate for `my_proc_macros`
 --> lib.rs:1:5
  |
1 | use my_proc_macros;
  |     ^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.

However if I substitute target thumbv7neon-unknown-linux-gnueabihf with armv7-unknown-linux-gnueabihf all builds fine.

Meta

rustc --version --verbose:

rustc 1.73.0 (cc66ad468 2023-10-03) (built from a source tarball)
binary: rustc
commit-hash: cc66ad468955717ab92600c770da8c1601a4ff33
commit-date: 2023-10-03
host: armv7-unknown-linux-gnueabihf
release: 1.73.0
LLVM version: 17.0.2

[jpalus]

Same behavior observed with x86_64-unknown-linux-gnu/x86_64-unknown-linux-gnux32 combination (Tier 1 with host tools, Tier 2 without host tools).

[ehuss]

The proc-macro must be compiled with the same target as the host rustc. From what I can tell, your host is armv7-unknown-linux-gnueabihf, so the proc-macro must also be that target (or no --target flag is easiest).

[jpalus]

@ehuss if I understand it correctly proc-macro is host library then? That is quite confusing I'd say. How would projects handle it correctly? If there's no option to pass "host triplet" I'd say it would be better to ignore --target for such case.

[ehuss]

Yes, the proc-macro is loaded into the compiler as a shared library, so it must match the same architecture. I recommend not passing --target for proc-macros.

[jpalus]

It's all done through meson, though I guess it applies to any other build tool out there, which is passed a single set of rust flags. So I guess build tool should be smart enough to detect crate type and filter out or not --target. That's quite a lot of logic.

[eli-schwartz]

Yes, the proc-macro is loaded into the compiler as a shared library, so it must match the same architecture. I recommend not passing --target for proc-macros.

More broadly in a cross compilation workflow a proc macro should be built for the native (build) platform rather than the cross (host) one, which means that its rustc compiler must be defined in native mode.

This should be fixed by meson in mesonbuild/meson#11714 which forced proc-macro to always be compiled as native, but it does require defining the native/cross platforms in the compilation environment which may be a contributing factor here.