Skip to content

ci: parallelize xcframework slice builds #5218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 73 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
5e09c96
ci: remove workflow that pushes commits to branches
armcknight May 23, 2025
de8e3e1
also remove script and update contributing
armcknight May 23, 2025
31f52c4
just use the main init and brewfile
armcknight May 23, 2025
541ce9a
ref: extract xcframework-based jobs from build.yml into new build-xcf…
armcknight May 9, 2025
eeac6b7
fix new build trigger
armcknight May 9, 2025
3d52789
wip splitting up
armcknight May 9, 2025
b31dc3b
more wip
armcknight May 9, 2025
2abdfa1
more wip
armcknight May 12, 2025
8d6b5bc
parameterize everything in a matrix
armcknight May 17, 2025
221c5ef
DONOTMERGE remove other ci jobs
armcknight May 17, 2025
f6670ce
try assembly as well
armcknight May 17, 2025
f2f2a94
fix yaml spec
armcknight May 17, 2025
8ca70ef
try to fix suffix
armcknight May 17, 2025
f7d81dd
fix matrix access
armcknight May 17, 2025
8182223
capture logfile
armcknight May 17, 2025
4713730
fix logs
armcknight May 17, 2025
8f5ff02
step ordering with checkout
armcknight May 17, 2025
94d8d9f
put xcbeautify in build ci brewfile
armcknight May 17, 2025
adb0215
oh yeah xcbeautify is preinstalled
armcknight May 17, 2025
f794b76
DONOTMERGE let all jobs run
armcknight May 17, 2025
92b0202
fix args
armcknight May 17, 2025
7347998
remove dylib machotype for uikitless variant
armcknight May 17, 2025
1e22c5e
try the correct macho type
armcknight May 17, 2025
62eff3d
remove target dependency of sentryswiftui on sentry
armcknight May 17, 2025
6b36de1
implement assembly steps
armcknight May 17, 2025
dc1f0aa
differentiate separate xcframeworks
armcknight May 17, 2025
c86924b
add some missing trigger files
armcknight May 17, 2025
bcd8e9f
Revert "remove target dependency of sentryswiftui on sentry"
armcknight May 17, 2025
d193556
fix create xcframework name, TEMP comment out other jobs
armcknight May 17, 2025
a277b97
uncomment
armcknight May 17, 2025
63c3b39
Revert "DONOTMERGE let all jobs run"
armcknight May 17, 2025
f3050f6
Revert "DONOTMERGE remove other ci jobs"
armcknight May 17, 2025
f898d9a
DONOTMERGE remove all but build and release jobs
armcknight May 17, 2025
eb2e5db
put back release job and use the two new parallel steps
armcknight May 17, 2025
1e7c5bc
extract assembly to script
armcknight May 17, 2025
b741791
put in the release version
armcknight May 17, 2025
8a61b47
fix perms
armcknight May 17, 2025
7d4f81b
fix awks, remove unnecessary workflow job steps
armcknight May 17, 2025
e285af8
properly interpolate path into awk script
armcknight May 17, 2025
96f0096
dont need to append path prefix
armcknight May 17, 2025
4fe1801
eval as one long string
armcknight May 17, 2025
889ea3a
interleave framework/debugsymbol args
armcknight May 17, 2025
e67bfd4
separately check for existence of dsym and framework; static framewor…
armcknight May 17, 2025
c69cfd4
do the sentry dep build removal before even archiving
armcknight May 17, 2025
27357dd
use pattern for all the validation downloads
armcknight May 17, 2025
746500e
update the version in release.yml
armcknight May 17, 2025
335339e
merge multiple artfact downloads for validation steps
armcknight May 17, 2025
2d85054
only download sentry-dynamic and sentryswiftui for the valdation step
armcknight May 18, 2025
44b08de
need the static sentry framework not the dynamic one
armcknight May 20, 2025
cca1c02
try fixing paths?
armcknight May 20, 2025
54f9320
correctly remove sentry.framework from sentryswiftui build
armcknight May 20, 2025
e31ad72
fix comparison
armcknight May 20, 2025
3ea10bf
try single quotes
armcknight May 20, 2025
2c79300
ls some directories
armcknight May 20, 2025
dc97c90
fix find invocations, use -t on xargs to print every invocation
armcknight May 20, 2025
760bb97
find instead of ls
armcknight May 20, 2025
3b20a6a
find zipfiles
armcknight May 20, 2025
dff5aaf
move up to Carthage/
armcknight May 21, 2025
3e312a5
fix xargs mv
armcknight May 21, 2025
9762535
clean up
armcknight May 21, 2025
8fd80de
add back in the other workflows
armcknight May 21, 2025
4609768
remove reference scripts
armcknight May 21, 2025
0bcfb02
clean up makefile and more workflow file triggers
armcknight May 21, 2025
ea25a75
ci: refactor xcframework build workflows (#5277)
armcknight May 23, 2025
517598f
resolve merge leftovers
armcknight May 23, 2025
adf3872
put back some stuff after rebase
armcknight May 23, 2025
8de450f
format
armcknight May 23, 2025
0dbd80c
Merge branch 'main' into armcknight/ci/parallelize-xcframework-build
armcknight May 23, 2025
306772a
more format
armcknight May 23, 2025
5ec9eef
more format
armcknight May 23, 2025
4c80072
need to archive the maccatalyst slices too
armcknight May 23, 2025
11fb006
only build a catalyst slice for macosx sdk
armcknight May 23, 2025
ab6cd13
look more generally for frameworks and dsyms
armcknight May 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/assemble-xcframework-variant.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: "Assemble Sentry Cocoa XCFramework variant"

on:
workflow_call:
inputs:
name:
description: |-
The Sentry project target to build an XCFramework slice for.
Possible values: Sentry, SentrySwiftUI.
required: true
type: string

suffix:
description: |-
The suffix to add to the build product name.
E.g. "-Dynamic" or "-WithoutUIKitOrAppKit".
required: false
type: string

variant-id:
description: |-
The ID of the variant to build an XCFramework slice for. Used to collect appropriate slices for final deliverable assembly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: An example for a possible value would be nice.

required: true
type: string

jobs:
assemble-xcframework-variant:
name: Assemble ${{inputs.name}}${{inputs.suffix}} XCFramework Variant

runs-on: macos-14
steps:
- uses: actions/checkout@v4

- name: Download ${{inputs.variant-id}} Slices
uses: actions/download-artifact@v4
with:
pattern: xcframework-${{inputs.variant-id}}-slice-*
path: xcframework-slices

- name: Assemble XCFramework
run: ./scripts/assemble-xcframework.sh "${{github.workspace}}/xcframework-slices" "${{inputs.name}}${{inputs.suffix}}"
shell: bash

- name: Archive XCFramework
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: This uploads and doesn't archive the XCFramework, I believe.

Suggested change
- name: Archive XCFramework
- name: Upload XCFramework

uses: actions/upload-artifact@v4
with:
name: xcframework-${{github.sha}}-${{inputs.variant-id}}
if-no-files-found: error
path: ${{inputs.name}}${{inputs.suffix}}.xcframework.zip
36 changes: 25 additions & 11 deletions .github/workflows/benchmarking.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,14 @@ on:
- ".sauce/benchmarking-config.yml"
- "fastlane/**"
- "scripts/ci-select-xcode.sh"
- "scripts/build-xcframework.sh"
- "Samples/iOS-Swift/iOS-Swift.yml"
- "Samples/iOS-Swift/iOS-Swift.xcconfig"
- "Samples/iOS-Swift/iOS-SwiftClilp.xcconfig"
- "Samples/iOS-Swift/iOS-Benchmarking.xcconfig"
- "scripts/build-xcframework-slice.sh"
- "scripts/assemble-xcframework.sh"
- ".github/workflows/build-xcframework-variant-slices.yml"
- ".github/workflows/assemble-xcframework-variant.yml"

# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
Expand Down Expand Up @@ -169,9 +172,26 @@ jobs:
--tags benchmark \
--verbose

build-xcframework-variant-slices:
uses: ./.github/workflows/build-xcframework-variant-slices.yml
with:
name: Sentry
suffix: "-Dynamic"
variant-id: sentry-dynamic
sdk-list: '["iphoneos", "iphonesimulator"]'

assemble-xcframework-variant:
needs: build-xcframework-variant-slices
uses: ./.github/workflows/assemble-xcframework-variant.yml
with:
name: Sentry
suffix: "-Dynamic"
variant-id: sentry-dynamic

app-metrics:
name: Collect app metrics
runs-on: macos-15
needs: assemble-xcframework-variant
steps:
- name: Git checkout
uses: actions/checkout@v4
Expand All @@ -195,17 +215,11 @@ jobs:
MATCH_GIT_PRIVATE_KEY: ${{ secrets.MATCH_GIT_PRIVATE_KEY }}
MATCH_PASSWORD: ${{ secrets.MATCH_PASSWORD }}
MATCH_USERNAME: ${{ secrets.MATCH_USERNAME }}
- name: Build Framework
run: ./scripts/build-xcframework.sh iOSOnly

- name: Archive build log if failed
uses: actions/upload-artifact@v4
if: ${{ failure() || cancelled() }}
- uses: actions/download-artifact@v4
with:
name: raw-build-output-build-xcframework
path: |
build-xcframework.log

pattern: xcframework-${{github.sha}}-sentry-dynamic
path: Carthage/
- run: find Carthage -name "Sentry-Dynamic.xcframework.zip" -print0 | xargs -t0I @ unzip @ -d Carthage
- name: Build test app with sentry
run: bundle exec fastlane build_perf_test_app_sentry
env:
Expand Down
100 changes: 100 additions & 0 deletions .github/workflows/build-xcframework-variant-slices.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
name: "Build Sentry Cocoa XCFramework variant slices"

on:
workflow_call:
inputs:
sdk-list:
description: |-
The list of Apple platform SDKs for which to build slices and assemble into an XCFramework. This must be a JSON array of strings, itself in a string since GitHub Actions doesn't support arrays as inputs.
Possible values: iphoneos, iphonesimulator, macosx, appletvos, appletvsimulator, watchos, watchsimulator, xros, xrsimulator.
required: false
default: '["iphoneos", "iphonesimulator", "macosx", "appletvos", "appletvsimulator", "watchos", "watchsimulator", "xros", "xrsimulator"]'
type: string

name:
description: |-
The Sentry project target to build an XCFramework slice for.
Possible values: Sentry, SentrySwiftUI.
required: true
type: string

suffix:
description: |-
The suffix to add to the build product name.
E.g. "-Dynamic" or "-WithoutUIKitOrAppKit".
required: false
type: string

macho-type:
description: |-
The Mach-O type of the build product.
Possible values: mh_dylib, staticlib.
required: false
type: string
default: "mh_dylib"

configuration-suffix:
description: |-
The suffix to add to the build product name to build an alternate configuration of the target.
E.g. "WithoutUIKit".
required: false
type: string

variant-id:
description: |-
The ID of the variant to build an XCFramework slice for. Used to collect appropriate slices for final deliverable assembly.
required: true
type: string

jobs:
build-xcframework-variant-slices:
name: Build ${{inputs.name}}${{inputs.suffix}} XCFramework Variant Slice for ${{matrix.sdk}}

# We must compile this on an arm64 runner, cause it's required for visionOS. macos-14 uses arm64.
# To see the available runners see https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories.
runs-on: macos-14

strategy:
matrix:
sdk: ${{ fromJson(inputs.sdk-list) }}

steps:
- uses: actions/checkout@v4

# We have to compile on Xcode 15.2 because compiling on Xcode 15.4 fails with
# Data+SentryTracing.swift:21:62: error: 'ReadingOptions' aliases 'Foundation.ReadingOptions'
# and cannot be used here because C++ types from imported module 'Foundation' do not support
# library evolution; this is an error in the Swift 6 language mode
# We also can't use Xcode 16.x because validating the XCFramework then fails with Xcode 15.x.
- run: ./scripts/ci-select-xcode.sh 15.2
shell: bash

- name: Build ${{inputs.name}}${{inputs.suffix}} XCFramework slice for ${{matrix.sdk}}
if: startsWith(github.ref, 'refs/heads/release/') == false
run: ./scripts/build-xcframework-slice.sh ${{matrix.sdk}} ${{inputs.name}} "${{inputs.suffix}}" ${{inputs.macho-type}} ${{inputs.configuration-suffix}}
shell: bash

- name: Remove Sentry.framework from SentrySwiftUI build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: It would be great to have a comment here why we need to do this.

if: inputs.name == 'SentrySwiftUI'
run: |
find "${{github.workspace}}/Carthage/archive" -name "Sentry.framework" -print0 | xargs -t0 rm -rf
find "${{github.workspace}}/Carthage/archive" -name "Sentry.framework.dSYM" -print0 | xargs -t0 rm -rf
shell: bash

- name: Archiving xcarchive
uses: actions/upload-artifact@v4
with:
name: xcframework-${{inputs.variant-id}}-slice-${{matrix.sdk}}
if-no-files-found: error
path: |
${{github.workspace}}/Carthage/archive/${{inputs.name}}${{inputs.suffix}}/${{matrix.sdk}}.xcarchive
${{github.workspace}}/Carthage/DerivedData/Build/Products/${{inputs.name}}${{inputs.suffix}}.framework
${{github.workspace}}/Carthage/DerivedData/Build/Products/${{inputs.name}}${{inputs.suffix}}.framework.dSYM

- name: Archive build log if failed
uses: actions/upload-artifact@v4
if: ${{ failure() || cancelled() }}
with:
name: raw-build-output-build-xcframework-${{inputs.variant-id}}-${{matrix.sdk}}
path: |
*.log
154 changes: 154 additions & 0 deletions .github/workflows/build-xcframework.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
name: "Build Sentry XCFramework"
on:
push:
branches:
- main
- release/**

pull_request:
paths:
- "Sources/**"
- ".github/workflows/build-xcframework.yml"
- "scripts/ci-select-xcode.sh"
- Sentry.xcworkspace/**
- Sentry.xcodeproj/**
- "Package.swift"
- "scripts/build-xcframework-slice.sh"
- "scripts/assemble-xcframework.sh"
- ".github/workflows/build-xcframework-variant-slices.yml"
- ".github/workflows/assemble-xcframework-variant.yml"
- Samples/macOS-SPM-CommandLine/**
- Samples/SPM-Dynamic/**

# https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

jobs:
build-xcframework-variant-slices:
uses: ./.github/workflows/build-xcframework-variant-slices.yml
with:
name: ${{matrix.variant.name}}
suffix: ${{matrix.variant.suffix}}
macho-type: ${{matrix.variant.macho-type}}
configuration-suffix: ${{matrix.variant.configuration-suffix}}
variant-id: ${{matrix.variant.id}}
strategy:
matrix:
variant:
- name: Sentry
suffix: "-Dynamic"
id: sentry-dynamic
- name: Sentry
macho-type: staticlib
id: sentry-static
- name: SentrySwiftUI
id: sentryswiftui
- name: Sentry
suffix: "-WithoutUIKitOrAppKit"
macho-type: mh_dylib
configuration-suffix: WithoutUIKit
id: sentry-withoutuikit-dynamic

assemble-xcframework-variant:
needs: build-xcframework-variant-slices
uses: ./.github/workflows/assemble-xcframework-variant.yml
with:
name: ${{matrix.variant.name}}
suffix: ${{matrix.variant.suffix}}
variant-id: ${{matrix.variant.id}}
strategy:
matrix:
variant:
- name: Sentry
suffix: "-Dynamic"
id: sentry-dynamic
- name: Sentry
id: sentry-static
- name: SentrySwiftUI
id: sentryswiftui
- name: Sentry
suffix: "-WithoutUIKitOrAppKit"
id: sentry-withoutuikit-dynamic

validate-xcframework:
name: Validate XCFramework
runs-on: macos-13
needs: assemble-xcframework-variant
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: xcframework-${{github.sha}}-sentry-static
path: Carthage/
- uses: actions/download-artifact@v4
with:
pattern: xcframework-${{github.sha}}-sentryswiftui
path: Carthage/
- run: find Carthage -name "*.zip" -print0 | xargs -t0I @ mv @ Carthage
- run: ./scripts/ci-select-xcode.sh 15.2
- run: make build-xcframework-sample
shell: sh

# Use github.event.pull_request.head.sha instead of github.sha when available as
# the github.sha is be the pre merge commit id for PRs.
# See https://github.community/t/github-sha-isnt-the-value-expected/17903/17906.
validate-spm:
name: Validate Swift Package Manager
runs-on: macos-13
needs: assemble-xcframework-variant
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: xcframework-${{github.sha}}-*
merge-multiple: true
- name: Change path of the framework
run: |
sed -i '' 's/url.*//g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Static/path: "Sentry.xcframework.zip"/g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Dynamic/path: "Sentry-Dynamic.xcframework.zip"/g' Package.swift
shell: bash
- run: swift build
working-directory: Samples/macOS-SPM-CommandLine
shell: sh

validate-spm-dynamic:
name: Validate Swift Package Manager Dynamic
runs-on: macos-13
needs: assemble-xcframework-variant
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: xcframework-${{github.sha}}-*
merge-multiple: true
- name: Change path of the framework
run: |
sed -i '' 's/url.*//g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Static/path: "Sentry.xcframework.zip"/g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Dynamic/path: "Sentry-Dynamic.xcframework.zip"/g' Package.swift
shell: bash
- run: swift build
working-directory: Samples/SPM-Dynamic
shell: sh

swift-build:
name: Build with Swift
runs-on: macos-13
needs: assemble-xcframework-variant
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: xcframework-${{github.sha}}-*
merge-multiple: true
- name: Change path of the framework
run: |
sed -i '' 's/url.*//g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Static/path: "Sentry.xcframework.zip"/g' Package.swift
sed -i '' 's/checksum: ".*" \/\/Sentry-Dynamic/path: "Sentry-Dynamic.xcframework.zip"/g' Package.swift
shell: bash
- run: swift build
shell: sh
Loading
Loading