macOS Sequoia (15) support / CI

[BetsyMcPhail]

Prior issue adding Sonoma (14) support: #20339

Tasks:

• Determine when our VM hosting in CI can run Sequoia
• Remove Ventura Support #21978
• Add initial set of experimental sequoia jobs to d-j-j https://github.com/RobotLocomotion/drake-jenkins-jobs/pull/156
• Update Jenkins cloud configuration from Ventura to Sequoia
• Remove Ventura images from orka cluster
• Create unprovisioned Sequoia images
• Updated drake-ci: setup script, cmake scripts, etc. Updates necessary to run jobs on Sequoia drake-ci#297
• Fix any issues with unprovisioned jobs setup (so that provisioned jobs can be created)
• Create provisioned Sequoia images
• Fix remaining issues so that all experimental Sequoia jobs pass. PR to disable failing test [examples] Disable test case on macOS Sequoia #22320, tracking issue something is blocking LCM traffic on macOS Sequoia CI #22322
• Create Production Continuous and Nightly jobs
• Update Docs: Add Sonoma to our support matrix. Likewise update the release playbook.

[williamjallen]

As per MacStadium's response to my recent support ticket, MacOS Sequoia will not be supported in Orka Cluster until "mid October".

[williamjallen]

MacStadium added this update to our ticket today:

Mid-October is to be expected for Sequoia support as it is coming with Orka 3.2-GA. We are in a release and testing cycle it is closing soon. I will keep this ticket open for more updates as they become available.

[williamjallen]

Orka 3.2 is now available, and I have scheduled the upgrade for 10/21/2024 @ 9:00 AM EST. It is unclear to me what this process requires, so I have reached out to MacStadium to determine if downtime will be necessary. Edit: MacStadium responded and said that that there will be downtime of up to 3 hours on the 21st.

[williamjallen]

We built new Sequoia images today (11/01/2024) but cannot deploy new VMs from them. Attempting to deploy our Sequoia VM produces the error: Error: Unable to deploy the VM as the underlying VM process has crashed.

I opened this ticket with MacStadium to address the issue.

[williamjallen]

The issue was resolved by MacStadium. MacOS Sequoia unprovisioned jobs are now available for testing. The old Ventura images have also been removed.

@BetsyMcPhail, all of the tasks in the issue description have been resolved. Do you want to keep this open until the provisioned images are available, or should we close this now and open new issues for individual tasks as they come up?

[BetsyMcPhail]

Thanks @williamjallen. Please keep it open, I will add more tasks for getting jobs to pass, adding continuous and nightly jobs, etc.

[jwnimmer-tri]

From the Drake Developers point of view, it's okay now to bake a Sonoma Provisioned images. All of the remaining fixes we anticipate will be to Drake's code, not Drake's setup scripts.

[williamjallen]

Sequoia provisioned images are now available for use. Closing this ticket since no further work is needed.

[BetsyMcPhail]

My checklist had a few items missing! Reopening so that I can finish those.

[BetsyMcPhail]

Based on the prior Ventura/Sonoma configurations (documented here), the Sequoia jobs that need to be created are:

Continuous
mac-arm-sequoia-clang-bazel-continuous-release

Nightly
mac-arm-sequoia-clang-bazel-nightly-everything-release
mac-arm-sequoia-clang-cmake-nightly-everything-release
mac-arm-sequoia-clang-cmake-nightly-release
mac-arm-sequoia-unprovisioned-clang-bazel-nightly-release

@jwnimmer-tri can you please confirm if this seems reasonable to you. Also, I was going to go ahead and put them into production immediately, with a warning to buildcops.

[jwnimmer-tri]

Yes, exactly. Agreed on both counts.