rust.proc_macro should possibly filter out --target

[jpalus]

Describe the bug
Raising this to meson team attention as a consequence of rust-lang/rust#116562. Perhaps you'd like to express your view there.

proc-macro crate is a host tools' "utility" that should be compiled with triplet matching host. Currently however rust_args, that might include --target different than host triplet, is included when building all rust artifacts including proc-macro crate type. This results in build failure due to proc-macro crate being missing (not matching expected triplet).

Possibly proc-macro should filter out --target, although there is a possibility that someone actually would like to compile it for use on a different host. Not really sure how it should be handled correctly or whether meson is really the best place to handle it.

To Reproduce
Use --target that does not match host tools, build shared object with proc-macro and try to use it in some other crate.

Expected behavior
Build should work fine.

[xclaesse]

Could this be a regression in rustc? When I implemented rust.proc_macro() in Meson I did test what happens when --target is passed and rustc seemed to correctly ignore that argument. Maybe I did not pay enough attention. I agree Meson should filter out --target for proc-macro.

OOC, why do you have --target in your rust_args for the native platform? Sounds like you should instead cross compile?

[jpalus]

I'm not that much into rust ecosystem and never tried cross-compilation but the problematic platforms I'm interested in (thumbv7neon-unknown-linux-gnueabihf / x86_64-unknown-linux-gnux32) are Tier 2 without Host Tools platforms so it's not like rustc can be built for them and --target must always be used. At least that's my understanding.

[xclaesse]

Sounds like you should be cross compiling: https://mesonbuild.com/Cross-compilation.html. You can then have rust = ['rustc', '--target', 'thumbv7neon-unknown-linux-gnueabihf'] in the cross file.

[jpalus]

Thanks. Indeed that looks like a possibility though I'm a little hesitant. These are native ARMv7 (with NEON) and x32 platforms and bending entire distribution build infrastructure so it can cope with one weird compiler for a single crate type seems like an overkill (at the moment). Let's see where this ticket goes first.

[xclaesse]

If you don't cross compile, it means that Meson could use system libraries and pkg-config dependencies, which also could cause issues. If you build on a native armv7neon platform, then you don't need to pass --target at all, do you?

[jpalus]

I want to use system libraries and pkg-config dependencies and it is native armv7neon platform. However there is no such thing as native rustc for thumbv7neon-unknown-linux-gnueabihf -- thumbv7neon-unknown-linux-gnueabihf ships only std lib. For rustc I use armv7-unknown-linux-gnueabihf which needs --target in order to compile for thumbv7neon-unknown-linux-gnueabihf.

[eli-schwartz]

I want to use system libraries and pkg-config dependencies and it is native armv7neon platform.

So the build machine is armv7neon

However there is no such thing as native rustc for thumbv7neon-unknown-linux-gnueabihf -- thumbv7neon-unknown-linux-gnueabihf ships only std lib.

Not important since you're not running such a system.

For rustc I use armv7-unknown-linux-gnueabihf which needs --target in order to compile for thumbv7neon-unknown-linux-gnueabihf.

... and your --target for the host machine is thumbv7neon

...

The underlying question here is, are you configuring meson to perform a native build or a cross build?

If you're configuring as a native build then your entire toolchain should be native compilers running on and for the native abi and you don't need --target.

If you're configuring as a cross build then you should be building proc macros with the native rustc (no --target) and the installable artifacts with the cross rustc (has --target).

[jpalus]

I'd like to configure it for native build but there is no "native toolchain" for "thumbv7neon-unknown-linux-gnueabihf".

If you want to support rust tiers without host tools only through cross compilation then so be it and feel free to reject the ticket.

[xclaesse]

My understanding is this case is analogous to build for amd64 on amd64 machine (i.e. native build), but with a i686 compiler binary?

[dcbaker]

I'm confused, you have a system with "normal" (ie, not thumb) armv7 libraries, but you're compiling and linking those into thumb binaries?

[jpalus]

I'm not sure we're on the same page. Just to summarize link posted before:

Rust is available for multiple platforms with varying degree of support. It is expressed with tiers -- Tier 1 is the best support, Tier 3 is the lowest support. Tiers are further divided into those "with host tools" and those "without host tools". Host tools are ie rustc or cargo. Tiers with host tools have rustc, tiers without host tools don't have rustc. In order to compile source for tier wihtout host tools you need both rustc from tier with host tools as well as core/std lib from tier without host tools. armv7-unknown-linux-gnueabihf is an example of Tier 1 platform which includes host tools, while thumbv7neon-unknown-linux-gnueabihf is an example of Tier 2 platform without host tools. In order to compile programs for thumbv7neon-unknown-linux-gnueabihf you need host tools from Tier 1, ie from armv7-unknown-linux-gnueabihf.

My understanding is this case is analogous to build for amd64 on amd64 machine (i.e. native build), but with a i686 compiler binary?

I understand how "normal" compilers work and this analogy works in Rust as long as you compare tiers with host tools, but breaks as soon as you compare tiers without host tools. Say you can have x86_64 binary of g++ that produces x86_64 binaries by default and x86_64 (x32 ABI) binary of g++ that produces x86_64 (x32 ABI) binaries by default. But you can't have x86_64 (x32 ABI) binary of rustc that produces x86_64 (x32 ABI) binaries by default. You need host tools (rustc) from x86_64-unknown-linux-gnu, std library from x86_64-unknown-linux-gnux32 and use --target during compilation. Translating it to gcc it's like you need x86_64 binary of g++ and x86_64 (x32 ABI) library libstdc++ using g++ --target=x32 for compilation. That's because x86_64-unknown-linux-gnux32 platform is Tier 2 without host tools.

I'm confused, you have a system with "normal" (ie, not thumb) armv7 libraries, but you're compiling and linking those into thumb binaries?

I have system that has only armv7/thumb/neon libraries but in order to have such binaries from rust based projects I need host tools from armv7-unknown-linux-gnueabihf. Yes it is linked against armv7/thumb/neon libraries.

[eli-schwartz]

I'm not sure we're on the same page. Just to summarize link posted before:

Rust is available for multiple platforms with varying degree of support. It is expressed with tiers -- Tier 1 is the best support, Tier 3 is the lowest support. Tiers are further divided into those "with host tools" and those "without host tools". Host tools are ie rustc or cargo. Tiers with host tools have rustc, tiers without host tools don't have rustc. In order to compile source for tier wihtout host tools you need both rustc from tier with host tools as well as core/std lib from tier without host tools.

Yes, that's not actually different from C/C++, you have a libc and a stdlibc++ or platform variants, which is for the host platform, and a GCC / clang that is a build tool which emits machine code for the host and links to the host libraries.

"Tiers" without natively built GCC have no choice but to build a cross compiler running on the build machine (but in practice that tends to not be very applicable, and instead you're only forced to do so if your host machine is weak and you don't want to directly compile on it).

My understanding is this case is analogous to build for amd64 on amd64 machine (i.e. native build), but with a i686 compiler binary?

I understand how "normal" compilers work and this analogy works in Rust as long as you compare tiers with host tools, but breaks as soon as you compare tiers without host tools. Say you can have x86_64 binary of g++ that produces x86_64 binaries by default and x86_64 (x32 ABI) binary of g++ that produces x86_64 (x32 ABI) binaries by default. But you can't have x86_64 (x32 ABI) binary of rustc that produces x86_64 (x32 ABI) binaries by default.

You absolutely, positively, unquestionably can.

$ cat ./x86_64-unknown-linux-gnux32-rustc
#!/bin/sh

rustc --target x86_64-unknown-linux-gnux32 "$@"

This is an executable for rustc that produces x86_64-unknown-linux-gnux32 binaries by default.

Whether rust-lang chooses to support this at the ELF binary level or only implicitly against their will at the shell script level is irrelevant.

It's irrelevant on additional levels too, because as far as meson is concerned, if you're cross compiling to x86_64-unknown-linux-gnux32 then your cross compiler executable isn't "rustc" with rust arguments of "--target x86_64-unknown-linux-gnux32".

You need host tools (rustc) from x86_64-unknown-linux-gnu, std library from x86_64-unknown-linux-gnux32 and use --target during compilation. Translating it to gcc it's like you need x86_64 binary of g++ and x86_64 (x32 ABI) library libstdc++ using g++ --target=x32 for compilation. That's because x86_64-unknown-linux-gnux32 platform is Tier 2 without host tools.

Maybe you should compare to clang instead of GCC. GCC doesn't support --target but tends to involve rebuilding the entire binary for each target you want to target. The shell script visualization works for clang the same way though.

I'm confused, you have a system with "normal" (ie, not thumb) armv7 libraries, but you're compiling and linking those into thumb binaries?

I have system that has only armv7/thumb/neon libraries but in order to have such binaries from rust based projects I need host tools from armv7-unknown-linux-gnueabihf. Yes it is linked against armv7/thumb/neon libraries.

Right, so thumbv7neon is your host machine and you want to cross compile for it and pick up its libraries in your cross compile toolchain.

And armv7 is your build machine.

[jpalus]

Thanks for your contribution. As I understand request was rejected.