Building i686-unknown-linux-gnu target on OS X fails

[alexchandel]

I'm trying to build the rust libraries for multiple targets on OS X for cross-compilation purposes. Building with,

./configure --target=x86_64-apple-darwin,i686-unknown-linux-gnu,arm-unknown-linux-gnueabi
make

gives me:

rustc: x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat
rustc: x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libregex_macros
compile: i686-unknown-linux-gnu/rt/arch/i386/morestack.o
/Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S:93:18: error: unable to emit symbol attribute
 .private_extern ___morestack
                 ^
make: *** [i686-unknown-linux-gnu/rt/arch/i386/morestack.o] Error 1

[alexchandel]

However, manually compiling src/rt/arch/*/morestack.S for all three of those targets succeeds.

[brson]

Can you paste the error from make VERBOSE=1?

On x86 we use the in-tree llvm-mc to compile assembly, and if that program is being invoked correctly, it should not be possible for an error here.

[alexchandel]

Here's the make VERBOSE=1 error:

DYLD_LIBRARY_PATH=/Users/alex/GitHub/rust/x86_64-apple-darwin/stage2/lib:$DYLD_LIBRARY_PATH   x86_64-apple-darwin/stage2/bin/rustc --cfg stage2  -O --cfg rtopt --cfg debug  --target=x86_64-apple-darwin  -D warnings -L "x86_64-apple-darwin/rt" -L "/Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/lib" -L ""  --out-dir x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib -C extra-filename=-4e7c5e5c /Users/alex/GitHub/rust/src/libhexfloat/lib.rs
info: now are following matches for libhexfloat-*.dylib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat-4e7c5e5c.dylib
info: now are following matches for libhexfloat-*.rlib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libhexfloat-4e7c5e5c.rlib
MATCHES=""; if [ -n "$MATCHES" ] ; then echo "warning: removing previous" \'libregex_macros-*.dylib\' "libraries:" $MATCHES; rm $MATCHES ; fi
MATCHES=""; if [ -n "$MATCHES" ] ; then echo "warning: removing previous" \'libregex_macros-*.rlib\' "libraries:" $MATCHES; rm $MATCHES ; fi
DYLD_LIBRARY_PATH=/Users/alex/GitHub/rust/x86_64-apple-darwin/stage2/lib:$DYLD_LIBRARY_PATH   x86_64-apple-darwin/stage2/bin/rustc --cfg stage2  -O --cfg rtopt --cfg debug  --target=x86_64-apple-darwin  -D warnings -L "x86_64-apple-darwin/rt" -L "/Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/lib" -L ""  --out-dir x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib -C extra-filename=-4e7c5e5c /Users/alex/GitHub/rust/src/libregex_macros/lib.rs
info: now are following matches for libregex_macros-*.dylib libraries:
x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib/libregex_macros-4e7c5e5c.dylib
info: now are following matches for libregex_macros-*.rlib libraries:
clang -E -E -MMD -MP -MT i686-unknown-linux-gnu/rt/arch/i386/morestack.o -MF i686-unknown-linux-gnu/rt/arch/i386/morestack.d /Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S | /Users/alex/GitHub/rust/x86_64-apple-darwin/llvm/Release+Asserts/bin/llvm-mc -assemble -filetype=obj -triple=i686-unknown-linux-gnu -o=i686-unknown-linux-gnu/rt/arch/i386/morestack.o
/Users/alex/GitHub/rust/src/rt/arch/i386/morestack.S:93:18: error: unable to emit symbol attribute
 .private_extern ___morestack
                 ^
make: *** [i686-unknown-linux-gnu/rt/arch/i386/morestack.o] Error 1

[elexing]

@brson I have encountered the same problem many times, do you have a plan to solve it ?

[alexchandel]

Longer output. This is built with:

./configure --target=x86_64-apple-darwin,i686-unknown-linux-gnu,arm-unknown-linux-gnueabi
make

I believe this uses the clang version supplied with Xcode, but I have the LLVM head installed to /usr/local/opt/llvm. Would using that one help, and if so how do I change which clang the rust build uses?

[cmsd2]

I don't think there's anything wrong with the way llvm-mc is called.
But the initial call to clang on the left of the pipe doesn't pass any target options.
Clang seems to produce the private extern modifier only when targetting osx.

[cmsd2]

as a quick workaround i tried adding --target=i686-unknown-linux-gnu to the clang command above,
then resuming the build with make.
next it gets stuck on:

clang '-Wl,--as-needed' '-m32' '-L' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib' '-o' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o' '-Wl,-O1' '-nodefaultlibs' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib' '-L' 'i686-unknown-linux-gnu/rt' '-L' '.' '-L' '.' '-L' '/Users/chris/Development/cmsd2/rust/.rust' '-L' '/Users/chris/Development/cmsd2/rust' '-Wl,--whole-archive' '-Wl,-Bstatic' '-lrust_builtin' '-lbacktrace' '-lrustrt_native' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-ldl' '-lpthread' '-lrt' '-lgcc_s' '-lpthread' '-lc' '-lm' '-shared' '-lcompiler-rt’
ld: unknown option: --as-needed

i guessed again that there should be a --target option since it's trying to build a .so, not a .dylib.
also added is the -v option to show the linker command:

clang --target=i686-unknown-linux-gnu '-Wl,--as-needed' '-m32' '-L' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib' '-o' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' 'x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o' '-Wl,-O1' '-nodefaultlibs' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib' '/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib' '-L' 'i686-unknown-linux-gnu/rt' '-L' '.' '-L' '.' '-L' '/Users/chris/Development/cmsd2/rust/.rust' '-L' '/Users/chris/Development/cmsd2/rust' '-Wl,--whole-archive' '-Wl,-Bstatic' '-lrust_builtin' '-lbacktrace' '-lrustrt_native' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-ldl' '-lpthread' '-lrt' '-lgcc_s' '-lpthread' '-lc' '-lm' '-shared' '-lcompiler-rt' -v
Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
Target: i386-unknown-linux-gnu
Thread model: posix
Selected GCC installation: 
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" --eh-frame-hdr -m elf_i386 -shared -o x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so crti.o crtbeginS.o -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -Li686-unknown-linux-gnu/rt -L. -L. -L/Users/chris/Development/cmsd2/rust/.rust -L/Users/chris/Development/cmsd2/rust -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib -L/usr/lib --as-needed x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o --whole-archive -lmorestack --no-whole-archive x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o -O1 /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib --whole-archive -Bstatic -lrust_builtin -lbacktrace -lrustrt_native --no-whole-archive -Bdynamic -ldl -lpthread -lrt -lgcc_s -lpthread -lc -lm -lcompiler-rt crtendS.o crtn.o
ld: warning: directory not found for option '-L/Users/chris/Development/cmsd2/rust/.rust'
ld: unknown option: --eh-frame-hdr

I don't know how to fix that --eh-frame-hdr option. I tried messing with the ld command line to leave out the option to see what would happen.
I also have a gcc cross compiler installed, so i try the linker from that.
ld then whinges about the crt objects. I don't know but I suppose that's not surprising, so i remove the references to the first two that it complains about (crti.o and crtbeginS.o), again to see what happens:

/usr/local/cross/ia32/bin/i686-pc-elf-ld --eh-frame-hdr -m elf_i386 -shared -o x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4e7c5e5c.so -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -Li686-unknown-linux-gnu/rt -L. -L. -L/Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib -L/Users/chris/Development/cmsd2/rust/.rust/lib/i686-unknown-linux-gnu -L/Users/chris/Development/cmsd2/rust/lib/i686-unknown-linux-gnu -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib -L/usr/lib --as-needed x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.o --whole-archive -lmorestack --no-whole-archive x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/std-4e7c5e5c.metadata.o -O1 /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcollections-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/librand-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libunicode-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-4e7c5e5c.rlib /Users/chris/Development/cmsd2/rust/x86_64-apple-darwin/stage2/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-4e7c5e5c.rlib --whole-archive -Bstatic -lrust_builtin -lbacktrace -lrustrt_native --no-whole-archive -Bdynamic -ldl -lpthread -lrt -lgcc_s -lpthread -lc -lm -lcompiler-rt crtendS.o crtn.o
i686-unknown-linux-gnu/rt/librust_builtin.a: member i686-unknown-linux-gnu/rt/librust_builtin.a(rust_builtin.o) in archive is not an object

weird. it's trying to produce a static library for the gnu platform but rust_builtin.o is a mach (osx) object file:

file i686-unknown-linux-gnu/rt/rust_builtin.o
i686-unknown-linux-gnu/rt/rust_builtin.o: Mach-O object i386

What on earth is supposed to happen?

[brson]

Oh, man. I didn't realize this was trying to cross from darwin to linux. I'm positive nobody has ever done that before, so certainly there will be issues.

[alexchandel]

@cmsd2 You get that error when clang invokes darwin's ld with arguments for linux's ld.

We need to invoke a ELF-compatible linker. Clang is infamous for completely ignoring its linker path argument, so I usually invoke clang with -c and link separately. This probably means configuring Rust with a separate linker path for i686-unknown-linux-gnu.

Alternatively, LLD can cross-link without any problems. However due to a bug in how it handles static libraries, LLD is not presently capable of statically linking Rust programs (the issue on LLVM), although it can dynamically link them. cc #9367 for linker issues

[cmsd2]

Right ok.
In the short term I might have a go at replicating what the android port does with its platform-specific mk/cfg/ file.
What I was really hoping for ultimately though was actually a bit different.
I've written a kernel in C, which compiles using standard gcc tools. Developing on osx though, the target isn't sufficiently similar to the host, so that's why i started looking at rust as a cross compiler.
A bigger problem though is that I don't yet have a libc hosted target environment. Gcc is fine with this because when building it as a cross compiler for e.g. i686-pc-elf, it knows that -elf is without a libc.
Rust build doesn't have that option available, and tries building a complete runtime when all I currently need is #[no_std] support and the ability to compiler .rs to elf object files.