-
Notifications
You must be signed in to change notification settings - Fork 16
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update doc to Swift 6 and NDK 27b and remove the now unused destinati…
…on config files
- Loading branch information
1 parent
db148b2
commit aa00f19
Showing
4 changed files
with
93 additions
and
171 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,95 +1,78 @@ | ||
| # Swift cross-compilation SDK for Android | ||
| # Swift cross-compilation SDK bundle for Android | ||
|
|
||
| The patch used to build this SDK is open source and listed below. I | ||
| The patches used to build this SDK bundle are open source and listed below. I | ||
| maintain [a daily CI on github Actions](https://github.com/finagolfin/swift-android-sdk/actions?query=event%3Aschedule) | ||
| that [cross-compiles the SDK from the release and development source branches of | ||
| that [cross-compiles the SDK bundle from the release and development source branches of | ||
| the Swift toolchain for AArch64, armv7, and x86_64, builds several Swift | ||
| packages against those SDKs, and then runs their tests in the Android x86_64 | ||
| emulator](https://github.com/finagolfin/swift-android-sdk/blob/main/.github/workflows/sdks.yml). | ||
|
|
||
| Now that Swift 5.10 supports [the new SDK bundle | ||
| format](https://github.com/apple/swift-evolution/blob/main/proposals/0387-cross-compilation-destinations.md), | ||
| I plan to distribute an Android SDK bundle in the coming months. | ||
| ## Cross-compiling and testing Swift packages with the Android SDK buhdle | ||
|
|
||
| ## Cross-compiling and testing Swift packages with the Android SDK | ||
|
|
||
| To build with the Swift 5.10 SDK, first download [the latest Android LTS NDK | ||
| 27](https://developer.android.com/ndk/downloads) and [Swift 5.10.1 | ||
| compiler](https://swift.org/download/#releases) (make sure to install the Swift | ||
| compiler's dependencies linked there). Unpack these archives and the SDK. | ||
|
|
||
| Change the symbolic link at `swift-5.10-android-24-sdk/usr/lib/swift/clang` | ||
| to point to the clang headers that come with your swift compiler, eg | ||
| To build with the Swift 6 SDK bundle, first download [the official open-source | ||
| Swift 6.0.1 toolchain for linux or macOS](https://swift.org/download/#releases) | ||
| (make sure to install the Swift dependencies linked there). Install the OSS | ||
| toolchain on macOS as detailed in [the instructions for using the static linux | ||
| Musl SDK bundle at swift.org](https://www.swift.org/documentation/articles/static-linux-getting-started.html). | ||
| On linux, simply download the toolchain, unpack it, and add it to your `PATH`. | ||
|
|
||
| Next, install the Android SDK bundle by having the Swift toolchain directly | ||
| download it: | ||
| ``` | ||
| ln -sf /home/yourname/swift-5.10.1-RELEASE-ubuntu22.04/usr/lib/clang/15.0.0 | ||
| swift-5.10-android-24-sdk/usr/lib/swift/clang | ||
| swift sdk install https://github.com/finagolfin/swift-android-sdk/releases/download/6.0.1/swift-6.0.1-RELEASE-android-24-0.1.artifactbundle.tar.gz --checksum 28d019e91902681e04bf62b9535888441aa7c0cc96902940964013f29020d100 | ||
| ``` | ||
| You can check if it was properly installed by running `swift sdk list`. | ||
|
|
||
| Next, modify the cross-compilation JSON file `android-aarch64.json` in this repo | ||
| similarly: | ||
|
|
||
| 1. All paths to the NDK should change from `/home/finagolfin/android-ndk-r27` | ||
| to the path to your NDK, `/home/yourname/android-ndk-r27`. | ||
|
|
||
| 2. The path to the compiler should change from `/home/finagolfin/swift-5.10.1-RELEASE-ubuntu22.04` | ||
| to the path to your Swift compiler, `/home/yourname/swift-5.10.1-RELEASE-ubi9`. | ||
|
|
||
| 3. The paths to the Android SDK should change from `/home/finagolfin/swift-5.10-android-24-sdk` | ||
| to the path where you unpacked the Android SDK, `/home/yourname/swift-5.10-android-24-sdk`. | ||
|
|
||
| Now you're ready to cross-compile a Swift package with the cross-compilation | ||
| configuration JSON file, `android-aarch64.json`, and run its tests on Android. | ||
| Now you're ready to cross-compile a Swift package and run its tests on Android. | ||
| I'll demonstrate with the swift-argument-parser package: | ||
| ``` | ||
| git clone --depth 1 https://github.com/apple/swift-argument-parser.git | ||
| cd swift-argument-parser/ | ||
| /home/yourname/swift-5.10.1-RELEASE-ubuntu22.04/usr/bin/swift build --build-tests | ||
| --destination ~/swift-android-sdk/android-aarch64.json | ||
| -Xlinker -rpath -Xlinker \$ORIGIN/swift-5.10-android-24-sdk/usr/lib/aarch64-linux-android | ||
| swift build --build-tests --swift-sdk aarch64-unknown-linux-android24 | ||
| ``` | ||
| This will cross-compile the package for Android aarch64 and produce a test | ||
| runner executable with the `.xctest` extension, in this case at | ||
| This will cross-compile the package for Android aarch64 at API 24 and produce a | ||
| test runner executable with the `.xctest` extension, in this case at | ||
| `.build/aarch64-unknown-linux-android24/debug/swift-argument-parserPackageTests.xctest`. | ||
| It adds a rpath for where it expects the SDK libraries to be relative to the | ||
| test runner when run on Android. | ||
|
|
||
| Sometimes the test runner will depend on additional files or executables: this | ||
| one depends on the example executables `generate-manual`, `math`, `repeat`, and | ||
| `roll` in the same build directory. Other packages use `#file` to point at test | ||
| data in the repo: I've had success moving this data with the test runner, after | ||
| modifying the test source so it has the path to this test data in the Android | ||
| test environment. See the example of [swift-crypto on the | ||
| CI](https://github.com/finagolfin/swift-android-sdk/blob/5.10/.github/workflows/sdks.yml#L317). | ||
|
|
||
| You can copy these executables and the SDK to [an emulator or a USB | ||
| debugging-enabled device with adb](https://github.com/apple/swift/blob/release/5.10/docs/Android.md#3-deploying-the-build-products-to-the-device), | ||
| one depends on the example executables `color`, `generate-manual`, `math`, | ||
| `repeat`, and `roll` in the same build directory. Other packages use `#file` to | ||
| point at test data in the repo: I've had success moving this data with the test | ||
| runner, after modifying the test source so it has the path to this test data in | ||
| the Android test environment. See the example of [swift-crypto on the | ||
| CI](https://github.com/finagolfin/swift-android-sdk/blob/6.0.1/.github/workflows/sdks.yml#L492). | ||
|
|
||
| You can copy these executables and the Swift runtime libraries to [an emulator | ||
| or a USB debugging-enabled device with adb](https://github.com/swiftlang/swift/blob/release/6.0/docs/Android.md#3-deploying-the-build-products-to-the-device), | ||
| or put them on an Android device with [a terminal emulator app like Termux](https://termux.dev/en/). | ||
| I test aarch64 with Termux so I'll show how to run the test runner there, but | ||
| the process is similar with adb, [as can be seen on the CI](https://github.com/finagolfin/swift-android-sdk/blob/5.10/.github/workflows/sdks.yml#L355). | ||
| the process is similar with adb, [as can be seen on the CI](https://github.com/finagolfin/swift-android-sdk/blob/6.0.1/.github/workflows/sdks.yml#L440). | ||
|
|
||
| Copy the test executables to the same directory as the SDK: | ||
| Copy the test executables to the same directory as the Swift 6 runtime libraries, | ||
| removing a few Android stub libraries that aren't needed: | ||
| ``` | ||
| cp .build/aarch64-unknown-linux-android24/debug/{swift-argument-parserPackageTests.xctest,generate-manual,math,repeat,roll} .. | ||
| cp .build/aarch64-unknown-linux-android24/debug/{swift-argument-parserPackageTests.xctest,color,generate-manual,math,repeat,roll} .. | ||
| cp ~/.swiftpm/swift-sdks/swift-6.0.1-RELEASE-android-24-0.1.artifactbundle/swift-6.0.1-release-android-24-sdk/android-27b-sysroot/usr/lib/aarch64-linux-android/24/lib*.so .. | ||
| rm ../lib{c,dl,log,m,z}.so | ||
| ``` | ||
| You can copy the SDK and test executables to Termux using scp from OpenSSH, run | ||
| these commands in Termux on the Android device: | ||
| You can copy the test executables and Swift 6 runtime libraries to Termux using | ||
| scp from OpenSSH, run these commands in Termux on the Android device: | ||
| ``` | ||
| uname -m # check if you're running on the right architecture, should say `aarch64` | ||
| cd # move to the Termux app's home directory | ||
| pkg install openssh | ||
| scp [email protected]:{swift-5.10-android-24-sdk.tar.xz, | ||
| swift-argument-parserPackageTests.xctest,generate-manual,math,repeat,roll} . | ||
| tar xf swift-5.10-android-24-sdk.tar.xz | ||
| scp [email protected]:{lib*.so,swift-argument-parserPackageTests.xctest,color,generate-manual,math,repeat,roll} . | ||
| ./swift-argument-parserPackageTests.xctest | ||
| ``` | ||
| I've tried several Swift packages, including some mostly written in C or C++, | ||
| and all the cross-compiled tests passed. | ||
| and all the cross-compiled tests passed. Note that while this SDK bundle is | ||
| compiled against Android API 24, there was a regression in Swift 6 so that | ||
| Foundation can only be run on Android API 29 or later, #175. I will update the | ||
| SDK bundle when I find a fix for that new issue. | ||
|
|
||
| You can even run armv7 tests on an aarch64 device, though Termux may require | ||
| running `unset LD_PRELOAD` before invoking an armv7 test runner on aarch64. | ||
|
|
@@ -99,28 +82,25 @@ mode. | |
|
|
||
| ## Porting Swift packages to Android | ||
|
|
||
| The most commonly needed change is to simply import Glibc for Android too (while | ||
| Bionic is the name of the C library on Android, currently Swift uses the name | ||
| `Glibc` as a placeholder for most non-Darwin, non-Windows C libraries), so add | ||
| or change these two lines for Android: | ||
| The most commonly needed change is to import the new Android overlay, so add | ||
| these two lines for Android when calling Android's C APIs: | ||
| ``` | ||
| #if canImport(Glibc) | ||
| import Glibc | ||
| #if canImport(Android) | ||
| import Android | ||
| ``` | ||
| For example, that is all I had to do [to port swift-argument-parser to | ||
| Android](https://github.com/apple/swift-argument-parser/pull/14/files). | ||
|
|
||
| You may also need to add some Android-specific support using `#if os(Android)`, | ||
| You may also need to add some Android-specific support using `#if canImport(Android)`, | ||
| for example, since FILE is an opaque struct since Android 7, you will [have to | ||
| refer to any FILE pointers like this](https://github.com/apple/swift-tools-support-core/pull/243/files): | ||
| refer to any FILE pointers like this](https://github.com/swiftlang/swift-tools-support-core/pull/243/files): | ||
| ``` | ||
| #if os(Android) | ||
| #if canImport(Android) | ||
| typealias FILEPointer = OpaquePointer | ||
| ``` | ||
| Those changes are all I had to do [to port swift-argument-parser to | ||
| Android](https://github.com/apple/swift-argument-parser/pull/651/files). | ||
|
|
||
| ## Building an Android app with Swift | ||
|
|
||
| Some people have reported an issue with using the libraries from this SDK in | ||
| Some people have reported an issue with using previous libraries from this SDK in | ||
| their Android app, that the Android toolchain strips `libdispatch.so` and | ||
| complains that it has an `empty/missing DT_HASH/DT_GNU_HASH`. You can [work | ||
| around this issue by adding the following to your `build.gradle`](https://github.com/finagolfin/swift-android-sdk/issues/67#issuecomment-1227460068): | ||
|
|
@@ -132,39 +112,34 @@ packagingOptions { | |
| } | ||
| ``` | ||
|
|
||
| Note that the FoundationNetworking and FoundationXML libraries won't actually | ||
| run on Android with this SDK, as their dependencies libcurl and libxml2 have other | ||
| library dependencies that aren't included. If you want to use either of these | ||
| separate Foundation libraries, you will have to track down those other libcurl/xml2 | ||
| dependencies and include them yourself. | ||
|
|
||
| ## Building the Android SDKs from source | ||
| ## Building an Android SDK from source | ||
|
|
||
| Download the Swift 5.10.1 compiler and Android NDK 27 as above. Check out this | ||
| repo and run | ||
| `SWIFT_TAG=swift-5.10.1-RELEASE ANDROID_ARCH=aarch64 swift get-packages-and-swift-source.swift` | ||
| to get some prebuilt Android libraries and the Swift source to build the SDK. If | ||
| you pass in a different tag like `swift-DEVELOPMENT-SNAPSHOT-2024-03-30-a` | ||
| Download the Swift 6.0.1 compiler as above and Android NDK 27b (only building | ||
| the Android SDKs on linux works for now). Check out this repo and run | ||
| `SWIFT_TAG=swift-6.0.1-RELEASE ANDROID_ARCH=aarch64 swift get-packages-and-swift-source.swift` | ||
| to get some prebuilt Android libraries and the Swift source to build an AArch64 | ||
| SDK. If you pass in a different tag like `swift-DEVELOPMENT-SNAPSHOT-2024-10-08-a` | ||
| for the latest Swift trunk snapshot and pass in the path to the corresponding | ||
| prebuilt Swift toolchain to `build-script` below, you can build a Swift trunk | ||
| SDK too, as seen on the CI. | ||
|
|
||
| Next, apply a patch to the Swift source, `swift-android.patch` from this repo, | ||
| which adds a dependency for the Foundation core library in this Android SDK, and | ||
| four more patches that make modifications for the nullability annotations newly | ||
| added in NDK 26: | ||
| plus three more patches that make modifications for NDK 27 and [the Foundation | ||
| rewrite in Swift 6 that was merged this summer](https://www.swift.org/blog/foundation-preview-now-available/) | ||
| and substitute a string for NDK 27: | ||
| ``` | ||
| git apply swift-android.patch swift-android-both-ndks.patch swift-android-foundation-ndk26.patch swift-android-stdlib-ndk26.patch swift-android-stdlib-except-trunk.patch | ||
| git apply swift-android.patch swift-android-foundation.patch swift-android-foundation-release.patch swift-android-foundation-except-trunk.patch | ||
| perl -pi -e 's%r26%r27%' swift/stdlib/cmake/modules/AddSwiftStdlib.cmake | ||
| ``` | ||
|
|
||
| After making sure [needed build tools like python 3, CMake, and ninja](https://github.com/apple/swift/blob/release/5.10/docs/HowToGuides/GettingStarted.md#linux) | ||
| After making sure [needed build tools like python 3, CMake, and ninja](https://github.com/swiftlang/swift/blob/release/6.0/docs/HowToGuides/GettingStarted.md#linux) | ||
| are installed, run the following `build-script` command with your local paths | ||
| substituted instead: | ||
| ``` | ||
| ./swift/utils/build-script -RA --skip-build-cmark --build-llvm=0 --android | ||
| --android-ndk /home/finagolfin/android-ndk-r27/ --android-arch aarch64 --android-api-level 24 | ||
| --build-swift-tools=0 --native-swift-tools-path=/home/finagolfin/swift-5.10.1-RELEASE-ubuntu22.04/usr/bin/ | ||
| --native-clang-tools-path=/home/finagolfin/swift-5.10.1-RELEASE-ubuntu22.04/usr/bin/ | ||
| --android-ndk /home/finagolfin/android-ndk-r27b/ --android-arch aarch64 --android-api-level 24 | ||
| --build-swift-tools=0 --native-swift-tools-path=/home/finagolfin/swift-6.0.1-RELEASE-ubuntu22.04/usr/bin/ | ||
| --native-clang-tools-path=/home/finagolfin/swift-6.0.1-RELEASE-ubuntu22.04/usr/bin/ | ||
| --host-cc=/usr/bin/clang-13 --host-cxx=/usr/bin/clang++-13 | ||
| --cross-compile-hosts=android-aarch64 --cross-compile-deps-path=/home/finagolfin/swift-release-android-aarch64-24-sdk | ||
| --skip-local-build --xctest --swift-install-components='clang-resource-dir-symlink;license;stdlib;sdk-overlay' | ||
|
|
@@ -176,63 +151,61 @@ Make sure you have an up-to-date CMake and not something old like 3.16. The | |
| `--host-cc` and `--host-cxx` flags are not needed if you have a `clang` and | ||
| `clang++` in your `PATH` already, but I don't and they're unused for this build | ||
| anyway but required by `build-script`. Substitute armv7 or x86_64 for aarch64 | ||
| into these commands to build for those architectures instead. | ||
| into these commands to build SDKs for those architectures instead. | ||
|
|
||
| Finally, copy `libc++_shared.so` from the NDK and modify the cross-compiled | ||
| `libdispatch.so` and Swift corelibs to include `$ORIGIN` and other relative | ||
| directories in their rpaths: | ||
| Swift corelibs to include `$ORIGIN` and other relative directories in their rpaths: | ||
| ``` | ||
| cp /home/yourname/android-ndk-r27/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so swift-release-android-aarch64-24-sdk/usr/lib | ||
| cp /home/yourname/android-ndk-r27b/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so swift-release-android-aarch64-24-sdk/usr/lib | ||
| patchelf --set-rpath \$ORIGIN/../..:\$ORIGIN swift-release-android-aarch64-24-sdk/usr/lib/swift/android/lib*.so | ||
| ``` | ||
|
|
||
| Here is a description of what the above Swift script is doing: | ||
|
|
||
| This prebuilt SDK was compiled against Android API 24, because the Swift | ||
| stdlib and corelibs require some libraries like libicu, that are pulled from the | ||
| Foundation libraries require some libraries like libcurl, that are pulled from the | ||
| prebuilt library packages used by the Termux app, which are built against Android | ||
| API 24. Specifically, it downloads the libicu, libicu-static, libandroid-spawn, | ||
| libcurl, and libxml2 packages from the [Termux package | ||
| API 24. Specifically, it downloads the libandroid-spawn, libcurl, and libxml2 | ||
| packages and their handful of dependencies from the [Termux package | ||
| repository](https://packages.termux.dev/apt/termux-main/pool/main/). | ||
|
|
||
| Each one is unpacked with `ar x libicu_74.2_aarch64.deb; tar xf data.tar.xz` and | ||
| Each one is unpacked with `ar x libcurl_8.10.1-1_aarch64.deb; tar xf data.tar.xz` and | ||
| the resulting files moved to a newly-created Swift release SDK directory: | ||
| ``` | ||
| mkdir swift-release-android-aarch64-24-sdk | ||
| mv data/data/com.termux/files/usr swift-release-android-aarch64-24-sdk | ||
| ``` | ||
| It removes two config scripts in `usr/bin`, runs `patchelf` to remove the | ||
| Termux rpath from all Termux shared libraries, and modifies the ICU libraries | ||
| to get rid of the versioning and symlinks (three libicu libraries are removed | ||
| since they're unused by Swift): | ||
| Termux rpath from all Termux shared libraries, removes some unused libraries | ||
| and config files, and modifies the libraries to get rid of the versioning and | ||
| symlinks, which can't always be used on Android: | ||
| ``` | ||
| rm swift-release-android-aarch64-24-sdk/usr/bin/*-config | ||
| cd swift-release-android-aarch64-24-sdk/usr/lib | ||
| rm libicu{io,test,tu}* | ||
| patchelf --set-rpath \$ORIGIN libandroid-spawn.so libcurl.so libicu*so.74.2 libxml2.so | ||
| patchelf --set-rpath \$ORIGIN libandroid-spawn.so libcurl.so libxml2.so | ||
| # repeat the following for libicui18n.so and libicudata.so, as needed | ||
| rm libicuuc.so libicuuc.so.74 | ||
| readelf -d libicuuc.so.74.2 | ||
| mv libicuuc.so.74.2 libicuuc.so | ||
| patchelf --set-soname libicuuc.so libicuuc.so | ||
| patchelf --replace-needed libicudata.so.74 libicudata.so libicuuc.so | ||
| # repeat the following for all versioned Termux libraries, as needed | ||
| rm libxml2.so libxml2.so.2 | ||
| readelf -d libxml2.so.2.13.4 | ||
| mv libxml2.so.2.13.4 libxml2.so | ||
| patchelf --set-soname libxml2.so libxml2.so | ||
| patchelf --replace-needed libz.so.1 libz.so libxml2.so | ||
| ``` | ||
| The libcurl and libxml2 packages are [only needed for the FoundationNetworking | ||
| and FoundationXML libraries respectively](https://github.com/apple/swift-corelibs-foundation/blob/release/5.10/Docs/ReleaseNotes_Swift5.md), | ||
| and FoundationXML libraries respectively](https://github.com/swiftlang/swift-corelibs-foundation/blob/release/5.10/Docs/ReleaseNotes_Swift5.md), | ||
| so you don't have to deploy them on the Android device if you don't use those | ||
| extra Foundation libraries. I simply include all four libraries since there's | ||
| currently no way to disable building them in the CMake configuration. | ||
| extra Foundation libraries. | ||
|
|
||
| The libicu dependency can be [cross-compiled for Android from scratch using | ||
| these instructions](https://github.com/apple/swift/blob/release/5.5/docs/Android.md#1-downloading-or-building-the-swift-android-stdlib-dependencies) | ||
| instead, so this Swift SDK for Android could be built without using | ||
| any prebuilt Termux packages, if you're willing to put in the effort to | ||
| cross-compile them yourself, for example, against a different Android API. | ||
| This Swift SDK for Android could be built without using any prebuilt Termux | ||
| packages, by compiling against a more recent Android API that doesn't need the | ||
| `libandroid-spawn` backport, and by cross-compiling libcurl/libxml2 and their | ||
| dependencies yourself or not using FoundationNetworking and FoundationXML, by | ||
| disabling their build. | ||
|
|
||
| Finally, it gets [the 5.10.1 source](https://github.com/apple/swift/releases/tag/swift-5.10.1-RELEASE) | ||
| tarballs for seven Swift repos and renames them to `llvm-project/`, `swift/`, | ||
| Finally, it gets [the 6.0.1 source](https://github.com/swiftlang/swift/releases/tag/swift-6.0.1-RELEASE) | ||
| tarballs for ten Swift repos and renames them to `llvm-project/`, `swift/`, | ||
| `swift-syntax`, `swift-experimental-string-processing`, `swift-corelibs-libdispatch`, | ||
| `swift-corelibs-foundation`, and `swift-corelibs-xctest`, as required by the Swift | ||
| `swift-corelibs-foundation`, `swift-collections`, `swift-foundation`, | ||
| `swift-foundation-icu`, and `swift-corelibs-xctest`, as required by the Swift | ||
| `build-script`. | ||
Oops, something went wrong.