Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish preview images, add checks for the multi-arch image variants #120

Merged
merged 8 commits into from
May 23, 2024
Merged

Publish preview images, add checks for the multi-arch image variants #120

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
691bc35
chore
gesellix Feb 29, 2024
67e8082
Publish preview images in the build workflow
gesellix Feb 29, 2024
e7e322d
Check the multi-arch image for completeness
gesellix Feb 29, 2024
7894779
Build images only when running on the default branch
gesellix Mar 8, 2024
a4203fe
Make the "build" job require only the "test-*" jobs
gesellix Mar 8, 2024
792b500
Make the "build" job require only the "test-*" jobs
gesellix Mar 18, 2024
cd569e9
RELEASE -> RELEASING :)
gesellix Mar 18, 2024
55b610f
be more lax checking arch/os versions
gesellix Mar 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 159 additions & 20 deletions .github/workflows/build-docker-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,11 @@ env:
IMAGE_REPOSITORY: testcontainers/ryuk

jobs:
test:
test-linux:
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
- run: go version
Expand All @@ -35,6 +32,31 @@ jobs:
- name: go-test
run: go test -v ./...

test-windows:
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
- run: go version

- name: go-mod verify
run: go mod verify

- name: go-mod tidy
run: go mod tidy

- name: go-build
env:
GOOS: windows
run: go build

# TODO enable tests on Windows
# Currently doesn't succeed, see https://github.com/testcontainers/testcontainers-go/issues/948
# - name: go-test
# run: go test -v ./...

build-image-linux:
runs-on: ubuntu-latest
steps:
Expand All @@ -44,6 +66,22 @@ jobs:
go-version-file: 'go.mod'
- run: go version

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.IMAGE_REPOSITORY }}
tags: |
type=sha,format=long
flavor: |
suffix=-linux

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
Expand All @@ -55,15 +93,15 @@ jobs:
- name: Buildx inspect
run: docker buildx inspect

- name: Build image
- name: Build and push image
uses: docker/build-push-action@v3
with:
context: .
file: linux/Dockerfile
platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x,linux/386,linux/arm/v6
push: false
# Use a 'temp' tag, that won't be pushed for non-release builds
tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'temp' }}-linux
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

build-image-windows:
strategy:
Expand All @@ -74,26 +112,127 @@ jobs:
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v3

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
go-version-file: 'go.mod'
- run: go version
images: ${{ env.IMAGE_REPOSITORY }}
tags: |
type=sha,format=long
flavor: |
suffix=-windows.${{ matrix.os-version }}

- name: Login to Docker Hub
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Docker info
run: docker info

- name: Build image
# Use a 'temp' tag, that won't be pushed for non-release builds
run: |
docker build -f windows/Dockerfile --build-arg BASE_IMAGE=mcr.microsoft.com/windows/nanoserver:${{ matrix.os-version }} -t ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'temp' }}-windows.amd64.${{ matrix.os-version }} .
docker build -f windows/Dockerfile --build-arg BASE_IMAGE=mcr.microsoft.com/windows/nanoserver:${{ matrix.os-version }} -t ${{ steps.meta.outputs.tags }} .

build:
runs-on: ubuntu-latest
- name: Push image
run: |
docker push ${{ steps.meta.outputs.tags }}

publish-multi-arch-image:
needs:
- test
- test-linux
- test-windows
- build-image-linux
- build-image-windows
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.IMAGE_REPOSITORY }}
tags: |
type=sha,format=long

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2

- name: Docker info
run: docker info
- name: Buildx inspect
run: docker buildx inspect

- name: Get os version for nanoserver:ltsc2019
run: >-
full_version=$(docker manifest inspect mcr.microsoft.com/windows/nanoserver:ltsc2019 | jq -r '.manifests[]|.platform|."os.version"'| sed 's@.*:@@') || true;
echo "OS_VERSION_ltsc2019=${full_version}" >> $GITHUB_ENV;

- name: Get os version for nanoserver:ltsc2022
run: >-
full_version=$(docker manifest inspect mcr.microsoft.com/windows/nanoserver:ltsc2022 | jq -r '.manifests[]|.platform|."os.version"'| sed 's@.*:@@') || true;
echo "OS_VERSION_ltsc2022=${full_version}" >> $GITHUB_ENV;

- name: Docker Manifest
run: >-
target_image=${{ steps.meta.outputs.tags }};
linux_manifest=$(docker manifest inspect ${target_image}-linux);
linux_digests=$(docker manifest inspect ${target_image}-linux | jq -r '.manifests[].digest');
manifest_list=${linux_digests//sha256:/${target_image%%:*}@sha256:};
manifest_list+=" ${target_image}-windows.ltsc2019";
manifest_list+=" ${target_image}-windows.ltsc2022";
docker manifest create ${target_image} ${manifest_list};
docker manifest annotate \
--os-version ${OS_VERSION_ltsc2019} \
--os windows \
--arch amd64 \
${target_image} "${target_image}-windows.ltsc2019";
docker manifest annotate \
--os-version ${OS_VERSION_ltsc2022} \
--os windows \
--arch amd64 \
${target_image} "${target_image}-windows.ltsc2022";
docker manifest push ${target_image};

build:
runs-on: ubuntu-latest
needs:
- publish-multi-arch-image
steps:
- name: join point for test, linux build, and windows build
- uses: actions/checkout@v4

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.IMAGE_REPOSITORY }}
tags: |
type=sha,format=long

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Print multi-arch manifest
run: |
docker manifest inspect --verbose ${{ steps.meta.outputs.tags }} > manifest_complete.json
cat manifest_complete.json | jq ".[].Descriptor.platform" | jq "(.architecture + \"_\" + .os + \"_\") + .\"os.version\"" | sort | uniq > architectures_actual.txt

- name: Check multi-arch manifest
run: |
echo "done"
cat architectures.txt | sort | uniq > architectures_expected.txt
diff architectures_expected.txt architectures_actual.txt
84 changes: 31 additions & 53 deletions .github/workflows/publish-docker-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,38 +11,16 @@ jobs:
release-linux:
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: 'go.mod'
- run: go version
- uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: go-mod verify
run: go mod verify

- name: go-mod tidy
run: go mod tidy

- name: go-build
env:
GOOS: linux
run: go build

- name: go-test
run: go test -v ./...

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
Expand All @@ -58,7 +36,6 @@ jobs:
context: .
file: linux/Dockerfile
platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x,linux/386,linux/arm/v6
# Only push if we are publishing a release
push: true
tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }}-linux

Expand All @@ -70,46 +47,24 @@ jobs:
- ltsc2022
runs-on: windows-2022
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version-file: 'go.mod'
- run: go version
- uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: go-mod verify
run: go mod verify

- name: go-mod tidy
run: go mod tidy

- name: go-build
env:
GOOS: windows
run: go build

#- name: go-test
# run: go test -v ./...

- name: Docker info
run: docker info

- name: Build image
run: |
docker build -f windows/Dockerfile --build-arg BASE_IMAGE=mcr.microsoft.com/windows/nanoserver:${{ matrix.os-version }} -t ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }}-windows.amd64.${{ matrix.os-version }} .
docker build -f windows/Dockerfile --build-arg BASE_IMAGE=mcr.microsoft.com/windows/nanoserver:${{ matrix.os-version }} -t ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }}-windows.${{ matrix.os-version }} .

- name: Push image
# Only push if we are publishing a release
run: |
docker push ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }}-windows.amd64.${{ matrix.os-version }}
docker push ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }}-windows.${{ matrix.os-version }}

release:
needs:
Expand Down Expand Up @@ -150,17 +105,40 @@ jobs:
linux_manifest=$(docker manifest inspect ${target_image}-linux);
linux_digests=$(docker manifest inspect ${target_image}-linux | jq -r '.manifests[].digest');
manifest_list=${linux_digests//sha256:/${target_image%%:*}@sha256:};
manifest_list+=" ${target_image}-windows.amd64.ltsc2019";
manifest_list+=" ${target_image}-windows.amd64.ltsc2022";
manifest_list+=" ${target_image}-windows.ltsc2019";
manifest_list+=" ${target_image}-windows.ltsc2022";
docker manifest create ${target_image} ${manifest_list};
docker manifest annotate \
--os-version ${OS_VERSION_ltsc2019} \
--os windows \
--arch amd64 \
${target_image} "${target_image}-windows.amd64.ltsc2019";
${target_image} "${target_image}-windows.ltsc2019";
docker manifest annotate \
--os-version ${OS_VERSION_ltsc2022} \
--os windows \
--arch amd64 \
${target_image} "${target_image}-windows.amd64.ltsc2022";
${target_image} "${target_image}-windows.ltsc2022";
docker manifest push ${target_image};

check-release:
runs-on: ubuntu-latest
needs:
- publish-multi-arch-image
steps:
- uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Print multi-arch manifest
run: |
docker manifest inspect --verbose ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name }} > manifest_complete.json
cat manifest_complete.json | jq ".[].Descriptor.platform" | jq "(.architecture + \"_\" + .os + \"_\") + .\"os.version\"" | sort | uniq > architectures_actual.txt

- name: Check multi-arch manifest
run: |
cat architectures.txt | sort | uniq > architectures_expected.txt
diff architectures_expected.txt architectures_actual.txt
8 changes: 8 additions & 0 deletions architectures.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"386_linux_"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should document this file at some place, like a RELEASING.md file or similar. Wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's a good idea. I could also rename the file to be more descriptive?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"supported-architectures.txt"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good and descriptive name 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the filename and added minimal documentation.

"amd64_linux_"
"amd64_windows_10.0.17763.5458"
"amd64_windows_10.0.20348.2322"
"arm_linux_"
"arm64_linux_"
"ppc64le_linux_"
"s390x_linux_"
Loading