Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release: allow pushing container images to multipes repositories #1091

Merged
merged 1 commit into from
Feb 19, 2025

Conversation

jbtrystram
Copy link
Contributor

In RHCOS we need to publish the oscontainer image to the release repo and also the CI repo. Changing the registry_repo object to be an array instead of a map to allow multiple entries.

Patch best reviwed without whitespace changes

@jbtrystram jbtrystram force-pushed the multi-container-upload branch 3 times, most recently from 6a88c81 to f46e326 Compare February 12, 2025 15:19
Copy link
Member

@jlebon jlebon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, but approach LGTM overall!

This changes the schema, so we'll need to make sure to have the changes for the RHCOS pipecfg as well lined up so they merge at the same time.

@@ -259,26 +259,28 @@ lock(resource: "release-${params.STREAM}", extra: locks) {
if (push_containers) {
stage("Push Containers") {
parallel push_containers.collectEntries{configname, val -> [configname, {
if (!registry_repos?."${configname}"?.'repo') {
if (!registry_repos?."${configname}"?.isEmpty()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does isEmpty() work if the $configname key is missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I understand correctly, if $configname is missing the ? will short circuit the expression

Copy link
Contributor Author

@jbtrystram jbtrystram Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did some testing and haven't had issues. However I see what you mean : it would not exit and fail afterwards I suppose.
WDYT of if (!registry_repos?."${configname}") { as empty collections evaluate to false.
I tested that and it works as expected (see below comment)

@jbtrystram jbtrystram added the hold Waiting for something label Feb 13, 2025
@jbtrystram jbtrystram force-pushed the multi-container-upload branch from f46e326 to ac12015 Compare February 13, 2025 10:52
@jbtrystram
Copy link
Contributor Author

Thanks @dustymabe @jlebon for the review comments. I addressed those.
I'll do some testing on the devel pipeline today

This changes the schema, so we'll need to make sure to have the changes for the RHCOS pipecfg as well lined up so they merge at the same time.

Yep, The PR over there is ready. I added the hold label just in case.

Copy link
Member

@dustymabe dustymabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@jlebon jlebon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbtrystram
Copy link
Contributor Author

This is now blocked on https://issues.redhat.com/browse/DPTP-4328.

Is it ? The RHCOS change is, yes, but we can test this change before adding the extra repository in the RHCOS config

@jlebon
Copy link
Member

jlebon commented Feb 14, 2025

This is now blocked on issues.redhat.com/browse/DPTP-4328.

Is it ? The RHCOS change is, yes, but we can test this change before adding the extra repository in the RHCOS config

Yes, that's fine assuming we remove that part of the pipecfg PR.

@jbtrystram jbtrystram force-pushed the multi-container-upload branch 7 times, most recently from b31a48d to fe95c57 Compare February 17, 2025 13:31
In RHCOS we need to publish the oscontainer image to the release
repo and also the CI repo. Changing the registry_repo object to be an
array instead of a map to allow multiple entries.
@jbtrystram jbtrystram force-pushed the multi-container-upload branch from fe95c57 to 8de05b1 Compare February 17, 2025 13:34
@jbtrystram
Copy link
Contributor Author

jbtrystram commented Feb 17, 2025

Allright, I did some testing on the staging FCOS pipeline to iron out the small details :)

With the following patch, this is the result :

diff --git a/jobs/release.Jenkinsfile b/jobs/release.Jenkinsfile
index 7a45c6e6..a120327c 100644
--- a/jobs/release.Jenkinsfile
+++ b/jobs/release.Jenkinsfile
@@ -270,16 +270,17 @@ lock(resource: "release-${params.STREAM}", extra: locks) {
                         for (repo in repos) {
                             def tag_args = repo.tags.collect{"--tag=$it"}
                             def v2s2_arg = repo.v2s2 ? "--v2s2" : ""
-                            shwrap("""
-                            export COSA_SUPERMIN_MEMORY=1024 # this really shouldn't require much RAM
-                            cp \${REGISTRY_SECRET} tmp/push-secret-${metajsonname}
-                            cosa supermin-run /usr/lib/coreos-assembler/cmd-push-container-manifest \
-                                --auth=tmp/push-secret-${metajsonname} \
-                                --repo=${repo.repo} ${tag_args.join(' ')} \
-                                --artifact=${artifact} --metajsonname=${metajsonname} \
-                                --build=${params.VERSION} ${v2s2_arg}
-                            rm tmp/push-secret-${metajsonname}
-                            """)
+                            println "Would push to ${repo.repo} with tags ${tag_args.join(' ')}"
                         }
                     }
                 }]}

Result:

[2025-02-17T13:35:43.478Z] Would push to quay.io/jbtrystram/fcos with tags --tag=rawhide --tag=test
[2025-02-17T13:35:43.500Z] Would push to quay.io/jbtrystram/someotherrepo with tags --tag=sometag

Using the following config :

  rawhide:
    type: mechanical
    env:
      COSA_TESTISO_DEBUG: true
      RUNVM_CACHE_SIZE: "30G"
    osbuild_experimental: true
    additional_registry_repos:
      oscontainer:
        - repo: quay.io/jbtrystram/fcos
          tags: ["rawhide", "test"]
        - repo: quay.io/jbtrystram/someotherrepo
          tags: ["sometag"]

@jbtrystram jbtrystram force-pushed the multi-container-upload branch from 8de05b1 to 2035d97 Compare February 17, 2025 13:38
@jbtrystram jbtrystram force-pushed the multi-container-upload branch 2 times, most recently from 9db798a to 2035d97 Compare February 19, 2025 17:43
Copy link
Member

@jlebon jlebon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Will merge when ready to merge the pipecfg one.

@jlebon jlebon merged commit b3e5ab1 into coreos:main Feb 19, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hold Waiting for something
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants