Rebuild axosyslog image with auto patch version increment #411

mitzkia · 2024-12-06T14:00:32Z

After this PR when we request a new stable image job the axosyslog patch version will increment automatically

requirements.txt

mitzkia · 2024-12-06T14:51:43Z

I would like to provide a test job run (from my fork) here (AxoSyslog stable image), but until now it fails every time on image build.

update:
it seems works :) :
https://github.com/mitzkia/axosyslog/actions/runs/12154279030/job/33894119353

.github/workflows/axosyslog-docker.yml

- this is for automatically update python libraries Signed-off-by: Andras Mitzki <[email protected]>

- after this change next docker tag will automatically calculated in a following way: - if actual git tag version is greater than previous docker tag example: current git tag: 4.10.0 last docker tag: 4.9.0-2 next docker tag: 4.10.0-1 - if actual git tag version is equals with previous docker tag example: current git tag: 4.9.0 last docker tag: 4.9.0-2 next docker tag: 4.9.0-3 - if actual git tag version is less than previous docker tag, than compare script will display a warning and will exit Signed-off-by: Andras Mitzki <[email protected]>

mitzkia · 2025-01-14T08:58:32Z

The PR is ready for the review.
I have the following proof runs:

When the actual git tag version is greater than the previous docker tag version

for example:
CI job link (in my fork): https://github.com/mitzkia/axosyslog/actions/runs/12752419559/job/35542271733
actual git tag: 4.99.0
previous docker tag: 4.9.0-2
next docker tag will be: 4.99.0-1

When the actual git tag version is equals with the previous docker tag version

for example:
CI job link (in my fork): https://github.com/mitzkia/axosyslog/actions/runs/12753284689/job/35545260970
actual git tag: 4.9.0
previous docker tag: 4.9.0-2
next docker tag will be: 4.9.0-3

When the actual git tag version is less than the previous docker tag version in this case the compare script
will display a warning and will exit
Check the generated docker image tags with:

skopeo inspect docker://ghcr.io/mitzkia/axosyslog:latest | jq '.RepoTags'
[
  "dev-main",
  "dev-main-dbg",
  "4.9.0-3",
  "4.9.0",
  "latest",
  "4.9.0-dbg",
  "4.9.0-3-dbg",
  "latest-dbg",
  "4.99.0-1",
  "4.99.0",
  "4.99.0-dbg",
  "4.99.0-1-dbg"
]

MrAnno

This must have been really difficult to test properly, thank you for going through everything.

MrAnno · 2025-01-16T14:56:45Z

.github/workflows/axosyslog-docker.yml

-          tags: type=match,pattern=axosyslog-([0-9].*),group=1
-          sep-tags: ','
+        run: |
+          previous_docker_tag=$(skopeo inspect docker://ghcr.io/axoflow/axosyslog:latest | jq '.RepoTags' | sed 's/ \|"\|,//g' | grep "^[0-9]*.[0-9]*.[0-9]*$\|^[0-9]*.[0-9]*.[0-9]*-[0-9]*$" | tail -n 1)


jq --raw-output '.RepoTags.[]' produces the desired output, so the sed can be removed.

MrAnno · 2025-01-16T14:58:16Z

.github/workflows/axosyslog-docker.yml

-          tags: type=match,pattern=axosyslog-([0-9].*),group=1
-          sep-tags: ','
+        run: |
+          previous_docker_tag=$(skopeo inspect docker://ghcr.io/axoflow/axosyslog:latest | jq '.RepoTags' | sed 's/ \|"\|,//g' | grep "^[0-9]*.[0-9]*.[0-9]*$\|^[0-9]*.[0-9]*.[0-9]*-[0-9]*$" | tail -n 1)


There is a problem with the skopeo query: it returns all tags, not the corresponding version for latest.
I think I have an idea how to query the exact version.

Very verbose, but it works. Let me know what you think:

- name: "Query previous container image version" uses: actions/github-script@v7 with: script: | const versionPattern = /^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+$/; const package_name = "axosyslog", org = "axoflow"; const allPackageVersions = await github.paginate( github.rest.packages.getAllPackageVersionsForPackageOwnedByOrg, { package_type: "container", package_name, org } ); const latest = allPackageVersions.find(p => p.metadata.container?.tags.includes("latest") ); const { data: { metadata: { container: { tags } } } } = await github.rest.packages.getPackageVersionForOrganization({ package_type: "container", package_name, org, package_version_id: latest.id }); previousDockerTag = tags.find(t => versionPattern.test(t)); core.exportVariable('previous_docker_tag', previousDockerTag);

After this step, you will get 4.9.0-2 in the $previous_docker_tag variable, and you can use in your script.

MrAnno · 2025-01-16T15:02:38Z

.github/workflows/axosyslog-docker.yml

+          previous_docker_tag=$(skopeo inspect docker://ghcr.io/axoflow/axosyslog:latest | jq '.RepoTags' | sed 's/ \|"\|,//g' | grep "^[0-9]*.[0-9]*.[0-9]*$\|^[0-9]*.[0-9]*.[0-9]*-[0-9]*$" | tail -n 1)
+          previous_docker_tag_short=$(echo $previous_docker_tag | cut -d"-" -f1)
+          previous_docker_tag_patch=$(echo $previous_docker_tag | cut -d"-" -f2)
+          current_git_tag=$(echo ${{ github.ref }} | cut -d"-" -f2)


We need to make sure everything below this line is skipped when github.ref is not a version tag.
We use this job to produce nightly and snapshot builds as well.

MrAnno · 2025-01-16T15:03:52Z

.github/workflows/axosyslog-docker.yml

+          current_git_tag=$(echo ${{ github.ref }} | cut -d"-" -f2)
+          function version { echo "$@" | awk -F. '{ printf("%d%03d%03d\n", $1,$2,$3); }'; }
+          if [ $(version $current_git_tag) -gt $(version $previous_docker_tag_short) ]; then
+              echo "Given git tag version is greater than previous docker tag version: $current_git_tag > $previous_docker_tag_short"


Now that we have debugged everything, we could replace this echo with something like this:

New AxoSyslog release detected (new version: , previous docker version: )

MrAnno · 2025-01-16T15:04:45Z

.github/workflows/axosyslog-docker.yml

+              next_docker_tag=$current_git_tag
+              echo "Next docker version tag: $next_docker_tag_with_patch"
+          elif [ $(version $current_git_tag) -eq $(version $previous_docker_tag_short) ]; then
+              echo "Given git tag version is equals with previous docker tag version: $current_git_tag == $previous_docker_tag_short"


AxoSyslog image rebuild detected (version: )

MrAnno · 2025-01-16T15:06:08Z

.github/workflows/axosyslog-docker.yml

+              echo "Next docker version tag: $next_docker_tag_with_patch"
+          else
+              echo "Given git tag version is less than previous docker tag version: $current_git_tag < $previous_docker_tag_short"
+              echo "This is not allowed. Please provide a valid git tag version."


Rebuilding an older AxoSyslog version is not allowed

MrAnno · 2025-01-16T15:29:29Z

.github/workflows/axosyslog-docker.yml

+        run: |
+          previous_docker_tag=$(skopeo inspect docker://ghcr.io/axoflow/axosyslog:latest | jq '.RepoTags' | sed 's/ \|"\|,//g' | grep "^[0-9]*.[0-9]*.[0-9]*$\|^[0-9]*.[0-9]*.[0-9]*-[0-9]*$" | tail -n 1)
+          previous_docker_tag_short=$(echo $previous_docker_tag | cut -d"-" -f1)
+          previous_docker_tag_patch=$(echo $previous_docker_tag | cut -d"-" -f2)


Nitpick: this is the rebuild number, not the patch part.

mitzkia marked this pull request as draft December 6, 2024 14:00

MrAnno reviewed Dec 6, 2024

View reviewed changes

requirements.txt Outdated Show resolved Hide resolved

mitzkia force-pushed the axosyslog_image_rebuild branch from 2762970 to c6c4a45 Compare December 6, 2024 14:32

mitzkia marked this pull request as ready for review December 6, 2024 14:57

bazsi reviewed Dec 7, 2024

View reviewed changes

.github/workflows/axosyslog-docker.yml Outdated Show resolved Hide resolved

mitzkia force-pushed the axosyslog_image_rebuild branch from c6c4a45 to ab1ede8 Compare December 9, 2024 10:27

MrAnno reviewed Dec 10, 2024

View reviewed changes

.github/workflows/axosyslog-docker.yml Outdated Show resolved Hide resolved

mitzkia force-pushed the axosyslog_image_rebuild branch from 9ec7b0c to ff36c55 Compare December 11, 2024 11:49

mitzkia marked this pull request as draft December 13, 2024 12:28

Andras Mitzki added 2 commits January 14, 2025 09:35

requirements: Remove patch version from 3rd party python dependencies

b4aca29

- this is for automatically update python libraries Signed-off-by: Andras Mitzki <[email protected]>

mitzkia force-pushed the axosyslog_image_rebuild branch from b78243d to 4391192 Compare January 14, 2025 08:49

mitzkia marked this pull request as ready for review January 14, 2025 08:58

MrAnno self-requested a review January 14, 2025 17:45

MrAnno reviewed Jan 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebuild axosyslog image with auto patch version increment #411

Rebuild axosyslog image with auto patch version increment #411

mitzkia commented Dec 6, 2024

mitzkia commented Dec 6, 2024 •

edited

Loading

mitzkia commented Jan 14, 2025

MrAnno left a comment

MrAnno Jan 16, 2025 •

edited

Loading

MrAnno Jan 16, 2025

MrAnno Jan 16, 2025 •

edited

Loading

MrAnno Jan 16, 2025

MrAnno Jan 16, 2025

MrAnno Jan 16, 2025

MrAnno Jan 16, 2025

MrAnno Jan 16, 2025

Rebuild axosyslog image with auto patch version increment #411

Are you sure you want to change the base?

Rebuild axosyslog image with auto patch version increment #411

Conversation

mitzkia commented Dec 6, 2024

mitzkia commented Dec 6, 2024 • edited Loading

mitzkia commented Jan 14, 2025

MrAnno left a comment

Choose a reason for hiding this comment

MrAnno Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

MrAnno Jan 16, 2025 • edited Loading

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

MrAnno Jan 16, 2025

Choose a reason for hiding this comment

mitzkia commented Dec 6, 2024 •

edited

Loading

MrAnno Jan 16, 2025 •

edited

Loading

MrAnno Jan 16, 2025 •

edited

Loading