Upgrade to XCode 16

[jwnimmer-tri]

Is your feature request related to a problem? Please describe.

In #21529 and #22029, we discovered that the standard library in XCode 14 and 15 is missing some support for spaceship (<=>).

Describe the solution you'd like

Upgrade our macOS Sonoma CI image(s) to XCode 16.

When creating our macOS Sequoia CI image(s), use XCode 16.

Of course then also we need to update our docs to reflect current CI, e.g.:

• https://drake.mit.edu/installation.html
• https://drake.mit.edu/from_source.html

Describe alternatives you've considered

Pollute our code with backcompat work-arounds.

Additional context

This is not an emergency. It can wait until we are already reworking the CI images.

In particular the until we do the #21910 Sequoia images and resize the images for #21977.

[rpoyner-tri]

FWIW, Xcode 15.3 seems to be at odds with updated bazel tool chain rules as well:

  /Users/ricopoyner/checkout/drake/WORKSPACE:12:19: in <toplevel>
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:74:23: in apple_cc_configure
Repository rule _apple_cc_autoconf defined at:
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:55:37: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_apple_cc':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl", line 49, column 49, in _apple_cc_autoconf_impl
                success, error = configure_osx_toolchain(repository_ctx)
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 206, column 21, in configure_osx_toolchain
                _compile_cc_file(repository_ctx, libtool_check_unique_src_path, "libtool_check_unique")
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 121, column 13, in _compile_cc_file
                fail(out_name + " failed to generate. Please file an issue at " +
Error in fail: libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: /Users/ricopoyner/checkout/drake/multibody/parsing/BUILD.bazel:272:17: //multibody/parsing:_detail_usd_parser_headers_cc_impl depends on @@local_config_apple_cc//:cc-compiler-darwin_arm64 in repository @@local_config_apple_cc which failed to fetch. no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//multibody/parsing:_detail_usd_parser_headers_cc_impl' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.627s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested
FAILED: 
    Fetching repository @@mujoco_menagerie_internal; starting
    Fetching repository @@buildifier; starting
    Fetching repository @@xmlrunner_py; starting
    Fetching repository @@crate__lapack-sys-0.14.0; starting
    Fetching repository @@vtk_internal; starting
    Fetching repository @@dm_control_internal; starting
    Fetching /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/crate__lapack-sys-0.14.0; Extracting download.tar.gz
    Fetching repository @@uwebsockets_internal; starting
ricopoyner@macsim-m1-1 drake % bazel clean --expunge --async
INFO: Starting clean.
INFO: Output base moved to /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985_tmp_99902_e60368b1-798a-43a1-bb39-a5fd5f949c0e for deletion
ricopoyner@macsim-m1-1 drake % bazel test //multibody/parsing/...
Starting local Bazel server and connecting to it...
INFO: Repository local_config_apple_cc instantiated at:
  /Users/ricopoyner/checkout/drake/WORKSPACE:12:19: in <toplevel>
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:74:23: in apple_cc_configure
Repository rule _apple_cc_autoconf defined at:
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:55:37: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_apple_cc':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl", line 49, column 49, in _apple_cc_autoconf_impl
                success, error = configure_osx_toolchain(repository_ctx)
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 206, column 21, in configure_osx_toolchain
                _compile_cc_file(repository_ctx, libtool_check_unique_src_path, "libtool_check_unique")
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 121, column 13, in _compile_cc_file
                fail(out_name + " failed to generate. Please file an issue at " +
Error in fail: libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: /Users/ricopoyner/checkout/drake/multibody/parsing/BUILD.bazel:104:17: //multibody/parsing:collision_filter_groups depends on @@local_config_apple_cc//:cc-compiler-darwin_arm64 in repository @@local_config_apple_cc which failed to fetch. no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: Analysis of target '//multibody/parsing:collision_filter_groups' failed; build aborted: Analysis failed
INFO: Elapsed time: 65.308s, Critical Path: 0.84s
INFO: 33 processes: 1 internal, 32 darwin-sandbox.
ERROR: Build did NOT complete successfully

See on macos 14.7, xcode 15.3.

[rpoyner-tri]

I've got a local fleet of macs with macos 14.7, xcode 16. None of them can build drake without link-time failures. If I downgrade to 15.4 (currently), all is well.

I think we need to figure out how to have Drake live with xcode 16. It doesn't necessarily block this CI work. I'll file a separate issue to track the gory Drake details.

[jwnimmer-tri]

New(?) approach:

• Keep Sonoma (i.e., our existing platform) on XCode 15.
• For the new Sequoia images (i.e., the new platform) install XCode 16. That means that CI will surely fail due to Drake circa 1.34 vs: Clang 16 or greater ==> link failures. #22061, but that's good. We can test PR fixes in CI.

[williamjallen]

The new Sequoia images discussed in #21910 (comment) contain Xcode 16.

[rpoyner-tri]

I believe that when #22176 is closed, the way will be clear to upgrade to Xcode 16 on either or both supported OS versions.

[rpoyner-tri]

Not so fast. Still outstanding is a fix for dynamic_cast in processes with multiple shared libraries. Recent sequoia CI builds will show the failures. See #22204

[jwnimmer-tri]

@BetsyMcPhail @williamjallen we have #22227 nearly ready, with the fix for XCode 16. Concurrently with merging the PR, we will also need to switch the Sonoma images to XCode 16, since the flag we need to use moving forward is only available in XCode 16. Please prep the new images, when possible.

[jwnimmer-tri]

From f2f, ETA would be the week of December 9th(-13th).

[williamjallen]

The Sequoia (un)provisioned images contain Xcode 16. We have built a new Sonoma stack which has Xcode 16, which we will turn on tomorrow in conjunction with the merging of #22227.