Allow Apple SDK to be missing on non-host macOS

[madsmtm]

Allow the SDK to be missing when cross-compiling to Apple platforms from non-host macOS, and instead emit a warning, since it is often available in the cross-compilation tooling the user has installed.

Note that in the common cross-compilation case (to macOS while linking with cc-like compiler), users won't actually hit this, since xcrun isn't invoked in that case. But I intend to change that in #131477, and then the error message here will start to matter a lot more.

The message now looks something like:

$ rustc +stage1 example.rs --target aarch64-apple-ios
warning: invoking `"xcrun" "--sdk" "iphoneos" "--show-sdk-path"` to find iPhoneOS.sdk failed: No such file or directory (os error 2)
  |
  = note: the SDK is needed by the linker to know where to find symbols in system libraries and for embedding the SDK version in the final object file
  = help: pass the path to the SDK using the SDKROOT environment variable
  = help: the SDK can be downloaded and extracted from https://developer.apple.com/download/all/?q=xcode (requires an Apple ID).
          
          The full Xcode bundle should contain the SDK in Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk.
  = warning: you will also need to use a linker capable of linking Mach-O files, consider using the bundled `lld` with `-Clinker=rust-lld`
  = warning: cross-compiling on iOS, tvOS, visionOS or watchOS (in particular the linking step) is ill supported on non-macOS hosts

Suggestions on how to improve it welcome!

Might also make sense to emit some of these help messages if the linker invocation itself fails? Unsure, but I'll work on that in a different PR.

(I originally considered not even attempting to invoke xcrun, but decided to continue doing that, since an xcrun-compatible interface can be present on other platforms, it's just unusual (e.g. Facebook developed xcbuild for a while, similar probably exist)).

Follow-up to #139010.
Part of #129432.
Related to rust-lang/rustup#1483 in that we now document the steps required for cross-compilation in the error message.

@rustbot label O-apple
r? wesleywiser since you just reviewed #139010
CC @BlackHoleFox @thomcc

[rustbot]

Some changes occurred in compiler/rustc_codegen_ssa

cc @WaffleLapkin

[madsmtm]

Converted to draft, because I want to think a little bit if we should make it a warning on host macOS too (to allow linking with e.g. zig cc on macOS while not having Xcode installed).

[wesleywiser]

As written, this looks fine to me so please ping me when you move it out of draft state (whichever way you decide) 🙂

[Mark-Simulacrum]

We might want to be a bit careful around recommending usage of the Apple SDK if our host triple isn't an apple triple - IIRC, the license terms don't permit usage of some parts of the SDK on non-apple hardware?

[madsmtm]

Ah, yeah, the Xcode SLA does say in section "2.2 A Permitted Uses and Restrictions":

Install a reasonable number of copies of the Apple Software [including the Apple SDKs] on Apple-branded computers that are owned or controlled by You to be used internally by You or Your Authorized Developers

And in section "2.7 Restrictions; No Other Permitted Uses":

The grants set forth in this Agreement do not permit You to, and You agree not to, install, use or run the Apple Software [including the Apple SDKs] or Apple Services on any non-Apple-branded computer or device, or to enable others to do so.

Btw, zig cc is clearly already violating that, since they bundle libSystem.tbd and headers from the macOS SDK, and thereby "enable others to use the Apple SDK on non-Apple-branded computers"...

I can change the wording to state that what the user is trying to do (cross link) is (probably, IANAL) not allowed according to Apple's terms? Though I still feel it important to provide these instructions in some shape or form.

---

Minutiae:

• codegen_ssa_xcrun_cross_about: technical description ("this is what an SDK is").
• codegen_ssa_xcrun_cross_env_var: also technical description ("this is how you pass SDKs to rustc").
• codegen_ssa_xcrun_cross_download_sdk: describes how the SDKs can be legally obtained.
  Note that I explicitly also chose to not mention that there are many ways of obtaining the SDKs without an Apple developer account.
  This way, I was hoping to defer the decision to the user, as they're obligated to read the Xcode SLA before using Xcode.
  But I could state explicitly here that doing this on non-Apple platforms is likely a violation of the SLA.
• codegen_ssa_xcrun_cross_linker_not_explicitly_set: technical recommendation ("here's an example of a linker you can easily and legally use on your platform").
• codegen_ssa_xcrun_cross_ill_supported_target: not sure, I'm really documenting the state of affairs of the linkers themselves (zig cc or lld), rustc's support is as good as for macOS. Should maybe just remove this.

[Mark-Simulacrum]

I think consulting with Foundation legal could be a good idea; I generally agree that as long as we're just pointing to Apple's pages on downloads etc. that's OK. Maybe adding some note around reading the terms would help.

I think the larger goal of this PR is not as clear to me though. It seems like, today, if I want to produce e.g. a dylib with Rust's standard library/core but no C code, I don't need the Apple SDK to do so -- lld is more or less happy to produce a Mach-O shared object (or non-shared object) from a pure Rust crate. If I want to compile C code for an Apple target, it's also possible that Apple SDKs are not required (e.g., because I'm not calling anything that requires Apple SDK headers).

But if we start requiring xcrun that seems to imply we think they are or should be required? Maybe this aspect of things relates more to the goals in #131477...?

[madsmtm]

It seems like, today, if I want to produce e.g. a dylib with Rust's standard library/core but no C code, I don't need the Apple SDK to do so -- lld is more or less happy to produce a Mach-O shared object (or non-shared object) from a pure Rust crate.

Hmm, perhaps I haven't been clear enough here. The Apple SDKs provides two things: headers and linker stubs.

lld is not able to produce neither a dylib nor binary without the linker stubs being available.

zig cc can in some cases because it bundles the linker stubs for system libraries (e.g. libSystem.tdb, the linker stub for libSystem.dylib, as well as libc.tdb etc.), but not system frameworks (CoreFoundation.framework/CoreFoundation.tdb etc.).

[Mark-Simulacrum]

lld is not able to produce neither a dylib nor binary without the linker stubs being available.

This seems untrue to me -- e.g., I produced this:

$ file libt.dylib
libt.dylib: Mach-O 64-bit arm64 dynamically linked shared library, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|NO_REEXPORTED_DYLIBS|HAS_TLV_DESCRIPTORS>

Without linking any shared libraries on a x86_64-unknown-linux-gnu host with:

mkdir -p /tmp/sdk
SDKROOT=/tmp/sdk rustc +nightly t.rs --target=aarch64-apple-darwin --crate-type=dylib -Zunstable-options '-Clink-arg=-undefined dynamic_lookup' -Clinker=$HOME/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld64.lld -Clinker-flavor=darwin-lld --verbose -Csave-temps

That fails with:

  = note: rust-lld: error: unknown argument '-flavor'
          rust-lld: error: unknown argument '-undefined dynamic_lookup'

But, if I edit that command to:

• Remove -flavor
• Fix -undefined ... to be two arguments, not one
• Remove -lSystem, -lc, -lm

That produces a libt.dylib that looks functional to me. Copying that to my macOS laptop, I can then link it with C and run it:

$ cat test.c
void rust_shared_foo();

int main() {
    rust_shared_foo();
    return 0;
}
$ clang test.c -lt -L. -o test
$ ./test
hello world from rust

Unless lld has bundled something (or maybe Rust has), I didn't use zig or similar here and AFAIK there's no linker stubs involved in producing this result.