Skip to content

Commit

Permalink
Add support for generating Swift SDKs for Ubuntu 24.04 Noble (#188)
Browse files Browse the repository at this point in the history
I was working on adding Debian 12 support for #116, but realized that adding Ubuntu Noble is "low-hanging fruit", since it's very straightforward. The default is still Ubuntu 22.04 Jammy, but this adds the option of generating the Swift SDK for 24.04 Noble now.

```
swift run swift-sdk-generator make-linux-sdk --linux-distribution-version 24.04
```

I have also changed the packages download to get `Packages.xz` instead of `Packages.gz` since it is a smaller file download. I wonder if it is okay to use `xz` here since I noticed that the directories on the Debian mirrors only have `Packages.xz` files available, unlike the Ubuntu mirrors which all have `Packages.gz` AND `Packages.xz` files available.
  • Loading branch information
xtremekforever authored Feb 25, 2025
1 parent a9f8e2a commit 0aa5486
Show file tree
Hide file tree
Showing 7 changed files with 101 additions and 19 deletions.
1 change: 1 addition & 0 deletions Brewfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
brew 'xz'
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,16 @@ swift experimental-sdk list
The output will either state that no Swift SDKs are available, or produce a list of those you previously had
installed, in case you've used the `swift experimental-sdk install` command before.

### macOS Requirements

The generator depends on the `xz` utility for more efficient downloading of package lists for Ubuntu. This is optional, but can be installed via the included `Brewfile`:

```bash
brew bundle install
```

If `xz` is not found, the generator will fallback on `gzip`.

## Supported platforms and minimum versions

macOS as a host platform and Linux as both host and target platforms are supported by the generator.
Expand All @@ -36,7 +46,7 @@ The generator also allows cross-compiling between any Linux distributions offici
| -: | :- | :- |
| macOS (arm64) | ✅ macOS 13.0+ ||
| macOS (x86_64) | ✅ macOS 13.0+[^1] ||
| Ubuntu | ✅ 20.04+ | ✅ 20.04 / 22.04 |
| Ubuntu | ✅ 20.04+ | ✅ 20.04+ |
| RHEL | ✅ Fedora 39[^2], UBI 9 | ✅ UBI 9 |
| Amazon Linux 2 | ✅ Supported | ✅ Supported[^3] |
| Debian 12 | ✅ Supported[^2] | ✅ Supported[^2][^3] |
Expand Down
6 changes: 3 additions & 3 deletions Sources/GeneratorCLI/GeneratorCLI.swift
Original file line number Diff line number Diff line change
Expand Up @@ -199,9 +199,9 @@ extension GeneratorCLI {

@Option(
help: """
Version of the Linux distribution used as a target platform. Available options for Ubuntu: `20.04`, \
`22.04` (default when `--linux-distribution-name` is `ubuntu`). Available options for RHEL: `ubi9` (default when \
`--linux-distribution-name` is `rhel`).
Version of the Linux distribution used as a target platform.
Available options for Ubuntu: `20.04`, `22.04` (default when `--linux-distribution-name` is `ubuntu`), `24.04`.
Available options for RHEL: `ubi9` (default when `--linux-distribution-name` is `rhel`).
"""
)
var linuxDistributionVersion: String?
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,27 +82,39 @@ extension SwiftSDKGenerator {
) async throws {
logger.debug("Parsing Ubuntu packages list...")

// Find xz path
let xzPath = try await which("xz")
if xzPath == nil {
logger.warning("""
The `xz` utility was not found in `PATH`. \
Consider installing it for more efficient downloading of package lists.
""")
}

async let mainPackages = try await client.parseUbuntuPackagesList(
ubuntuRelease: versionsConfiguration.linuxDistribution.release,
repository: "main",
targetTriple: self.targetTriple,
isVerbose: self.isVerbose
isVerbose: self.isVerbose,
xzPath: xzPath
)

async let updatesPackages = try await client.parseUbuntuPackagesList(
ubuntuRelease: versionsConfiguration.linuxDistribution.release,
releaseSuffix: "-updates",
repository: "main",
targetTriple: self.targetTriple,
isVerbose: self.isVerbose
isVerbose: self.isVerbose,
xzPath: xzPath
)

async let universePackages = try await client.parseUbuntuPackagesList(
ubuntuRelease: versionsConfiguration.linuxDistribution.release,
releaseSuffix: "-updates",
repository: "universe",
targetTriple: self.targetTriple,
isVerbose: self.isVerbose
isVerbose: self.isVerbose,
xzPath: xzPath
)

let allPackages = try await mainPackages
Expand Down Expand Up @@ -175,21 +187,31 @@ extension SwiftSDKGenerator {
extension HTTPClientProtocol {
private func downloadUbuntuPackagesList(
from url: String,
unzipWith zipPath: String,
isVerbose: Bool
) async throws -> String? {
guard let packages = try await get(url: url).body?.unzip(isVerbose: isVerbose) else {
guard let packages = try await get(url: url).body?.unzip(zipPath: zipPath, isVerbose: isVerbose) else {
throw FileOperationError.downloadFailed(url)
}

return String(buffer: packages)
}

func packagesFileName(isXzAvailable: Bool) -> String {
if isXzAvailable {
return "Packages.xz"
}
// Use .gz if xz is not available
return "Packages.gz"
}

func parseUbuntuPackagesList(
ubuntuRelease: String,
releaseSuffix: String = "",
repository: String,
targetTriple: Triple,
isVerbose: Bool
isVerbose: Bool,
xzPath: String?
) async throws -> [String: URL] {
let mirrorURL: String
if targetTriple.arch == .x86_64 {
Expand All @@ -200,13 +222,13 @@ extension HTTPClientProtocol {

let packagesListURL = """
\(mirrorURL)/dists/\(ubuntuRelease)\(releaseSuffix)/\(repository)/binary-\(
targetTriple.arch!
.debianConventionName
)/Packages.gz
targetTriple.arch!.debianConventionName
)/\(packagesFileName(isXzAvailable: xzPath != nil))
"""

guard let packages = try await downloadUbuntuPackagesList(
from: packagesListURL,
unzipWith: xzPath ?? "/usr/bin/gzip", // fallback on gzip if xz not available
isVerbose: isVerbose
) else {
throw GeneratorError.ubuntuPackagesDecompressionFailure
Expand Down
22 changes: 17 additions & 5 deletions Sources/SwiftSDKGenerator/PlatformModels/LinuxDistribution.swift
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,6 @@
//
//===----------------------------------------------------------------------===//

private let ubuntuReleases = [
"22.04": "jammy",
]

public enum LinuxDistribution: Hashable, Sendable {
public enum Name: String {
case rhel
Expand All @@ -27,13 +23,16 @@ public enum LinuxDistribution: Hashable, Sendable {
public enum Ubuntu: String, Sendable {
case focal
case jammy
case noble

init(version: String) throws {
switch version {
case "20.04":
self = .focal
case "22.04":
self = .jammy
case "24.04":
self = .noble
default:
throw GeneratorError.unknownLinuxDistribution(name: LinuxDistribution.Name.ubuntu.rawValue, version: version)
}
Expand All @@ -43,6 +42,7 @@ public enum LinuxDistribution: Hashable, Sendable {
switch self {
case .focal: return "20.04"
case .jammy: return "22.04"
case .noble: return "24.04"
}
}

Expand All @@ -60,7 +60,6 @@ public enum LinuxDistribution: Hashable, Sendable {
"linux-libc-dev",
"zlib1g",
"zlib1g-dev",
"libc6",
]
case .jammy: return [
"libc6",
Expand All @@ -75,6 +74,19 @@ public enum LinuxDistribution: Hashable, Sendable {
"zlib1g",
"zlib1g-dev",
]
case .noble: return [
"libc6",
"libc6-dev",
"libgcc-s1",
"libgcc-13-dev",
"libicu74",
"libicu-dev",
"libstdc++-13-dev",
"libstdc++6",
"linux-libc-dev",
"zlib1g",
"zlib1g-dev",
]
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions Sources/SwiftSDKGenerator/SystemUtils/ByteBuffer+Utils.swift
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ import Foundation
import NIOCore

public extension ByteBuffer {
func unzip(isVerbose: Bool) async throws -> ByteBuffer? {
func unzip(zipPath: String, isVerbose: Bool) async throws -> ByteBuffer? {
let result = try await ProcessExecutor.runCollectingOutput(
executable: "/usr/bin/gzip", ["-cd"],
executable: zipPath, ["-cd"],
standardInput: [self].async,
collectStandardOutput: true,
collectStandardError: false,
Expand Down
37 changes: 37 additions & 0 deletions Sources/SwiftSDKGenerator/SystemUtils/which.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift open source project
//
// Copyright (c) 2022-2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import AsyncProcess
import Foundation

/// Look for an executable using the `which` utility.
///
/// - Parameter executableName: The name of the executable to search for.
/// - Throws: Any errors thrown by the ProcessExecutor.
/// - Returns: The path to the executable if found, otherwise nil.
func which(_ executableName: String) async throws -> String? {
let result = try await ProcessExecutor.runCollectingOutput(
executable: "/usr/bin/which", [executableName], collectStandardOutput: true, collectStandardError: false,
environment: ProcessInfo.processInfo.environment
)

guard result.exitReason == .exit(0) else {
return nil
}

if let output = result.standardOutput {
let path = String(buffer: output).trimmingCharacters(in: .whitespacesAndNewlines)
return path.isEmpty ? nil : path
}

return nil
}

0 comments on commit 0aa5486

Please sign in to comment.