use reusable workflows in create-release workflow. #454

friedrichwilken · 2024-01-31T13:44:37Z

Description

Changes proposed in this pull request:

use reusable workflows for the Create release workflow
add unit tests and linting to the Create release workflow
automatically bump the sec-scanners-config.yaml

Resons for the removal of some files:

scripts/check_release_tag.sh was used in create-release to verify the version the user provided via input. Since we now derive the version from the branch name, this is no longer needed ( it is done here instead).
scripts/check_sec-scanners-config.sh was used in the create-release workflow to verify if the release version is the version used in the sec-scanners-config.yaml as the eventing-manager image tag. Since we now render the sec-scanners-config.yaml via bump-sec-scanners-config, this is no longer needed.
scripts/create_changelog.sh will be moved to eventing-tools by this PR
scripts/create_draft_release.sh will be replaced by create-draft-release-reusable.yml
scripts/publish_release.sh will be replaced by publish-release-reusable.yml
scripts/render_and_upload_manifests.sh will be replace by render-and-uploader-manifests-reusable.yml
scripts/verify-status.sh was used to wait the prow build job to finish successfully. This will happen in trigger-prow-build-job-reusable.yml by using kyma-project/wait-for-commit-status-action.

Related issue(s)
#361

marcobebway · 2024-02-01T16:44:43Z

.github/workflows/create-release.yml

-      version: ${{ steps.gen-version.outputs.VERSION }}
+  gen-version:
+    name: Generate semantic version from branch and tags
+    uses: kyma-project/eventing-tools/.github/workflows/get-version-from-release-branch-reusable.yml@main


Will this always return the workflow yaml from main~head? If yes, can we use a commit SHA instead, just in case main~head gets broken for some reason.

Note: This also applies to other places in this PR using @main.

Well, I am happy that you ask about this because I was also between these two options.
Sticking this to a commit sha makes this less susceptible to bug, so yes, let's do that.

I changed it to a commit sha everywhere.

I need to update the create-draft-release workflow sha, after this PR got merged.

marcobebway · 2024-02-01T16:56:28Z

.github/workflows/create-release.yml

-        with:
-          fetch-depth: 0
+  bump-sec-scanners-config:
+    name: Bump the sec-scandners-config.yaml


Suggested change

name: Bump the sec-scandners-config.yaml

name: Bump the sec-scanners-config.yaml

marcobebway · 2024-02-01T17:01:08Z

.github/workflows/create-release.yml

+  bump-sec-scanners-config:
+    name: Bump the sec-scandners-config.yaml
+    needs: gen-version
+    uses: kyma-project/eventing-tools/.github/workflows/bump-sec-scanners-config-reusable.yml@main


(nit) ignore for now, after noticing how the reusable workflows are used, maybe it would have been better to use the following naming instead of the current one:

kyma-project/eventing-tools/.github/workflows/reusable/bump-sec-scanners-config.yaml

but this is only my personal preference, so feel free to ignore.

That might have been a helpful solution. You would still find workflows at the expected place and the extra dir would make clear what this is, especially when we are calling them from other workflow like in this case. Learnings for the future.

marcobebway · 2024-02-01T17:18:02Z

.github/workflows/create-release.yml

-          echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
+  run-unit-test:
+    name: Run Unit Tests
+    needs: [gen-version, bump-sec-scanners-config]


Why does run-unit-test need bump-sec-scanners-config?

It does not need it, it will just wait to run the unit tests after bump-sec-scanners-config (because of the PR we create in the that part). In general, all the jobs will run consecutively. Only lint and unit-test will run in parallel because they are independent of each other. That flow is controlled by the need instructions.

the only flow that is really needed is the gen-version because only if it get's marked as needs the output (VERSION) gets available in the other jobs.

marcobebway · 2024-02-01T17:28:12Z

.github/workflows/create-release.yml

+    uses: kyma-project/eventing-tools/.github/workflows/render-and-upload-manifests-reusable.yml@main
+    with:
+      VERSION: ${{ needs.gen-version.outputs.VERSION }}
+      CR_FILE: "somedir/cr_file.yaml"


Oh , I forgot to change the names from tests that I have done earlier. Thanks. I fixed it.

But while I fixed this, I noticed that I have done a mistake when I implemented the flow and only tested it with "somedir/cr_file.yaml".

We need a way to rename the default cr, since gh cannot do it or upload it with a different name.

Here is the fix: would you also take a look at this? I will update the commit sha later, once it was merged.
https://github.com/kyma-project/eventing-tools/pull/66/files

marcobebway · 2024-02-01T17:59:17Z

hack/scripts/create_changelog.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash


Does it make sense to move this script to the common tools repo so other repositories may use it?

I thought about this. My intention was to give more flexibility with the changelog (similar to the sec-scanners-config and render-manifest). But you are right, in this case it is not needed at all. I added this PR to do so.

hack/scripts/create_changelog.sh

marcobebway · 2024-02-01T18:08:56Z

hack/scripts/create_changelog.sh

+done
+
+# Create a new file (with a unique name based on the process ID of the current shell).
+NEW_CONTRIB=$$.new


(nit) $$.new -> $$.authors

marcobebway · 2024-02-01T18:14:22Z

hack/scripts/create_changelog.sh

+echo -e "\n**Full changelog**: https://github.com/$REPOSITORY/compare/${PREVIOUS_RELEASE}...${RELEASE_TAG}" >>${CHANGELOG_FILE}
+
+# Cleanup the NEW_CONTRIB file.
+rm ${NEW_CONTRIB} || echo "cleaned up"


(nit) IMO, no need to echo anything here, because removing the file is an internal housekeeping.

I removed the whole section. It is just not needed.

marcobebway · 2024-02-01T18:21:48Z

hack/scripts/render-sec-scanners-config.sh

+echo "fetching webhook image from ${WEBHOOK_FILE}"
+WEBHOOK_IMAGE=$(yq eval '.images[0].newName' <"$WEBHOOK_FILE")
+WEBHOOK_TAG=$(yq eval '.images[0].newTag' <"$WEBHOOK_FILE")
+echo -e "webhook image is ${WEBHOOK_IMAGE}:${WEBHOOK_TAG} \n"


(nit) Maybe set the var WEBHOOK_IMAGE to be ${WEBHOOK_IMAGE}:${WEBHOOK_TAG}.

I like the idea. Done.

marcobebway · 2024-02-01T18:24:19Z

hack/scripts/render-sec-scanners-config.sh

+  - ${PUBLISHER_IMAGE}
+  - ${WEBHOOK_IMAGE}:${WEBHOOK_TAG}


After applying this comment, these lines should read:

- ${PUBLISHER_IMAGE} - ${WEBHOOK_IMAGE}

hack/scripts/render-sec-scanners-config.sh

Co-authored-by: Marco Bebway <[email protected]>

friedrichwilken · 2024-02-02T16:29:27Z

I will set this to halt for now. We will first merge this to a test branch, test the release flow and then cherry-pick it to main, once it is ready.

now useschanges from kyma-project/eventing-tools#68

kyma-project/eventing-tools@7113a71

Co-authored-by: kyma-eventing-bot <kyma-eventing-bot"@users.noreply.github.com>

kyma-bot · 2024-02-08T13:29:28Z

@friedrichwilken: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-eventing-manager-build	`0b0d1e1`	link	true	`/test pull-eventing-manager-build`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

add a reusable release workflow

a34787a

friedrichwilken requested review from the1bit and a team as code owners January 31, 2024 13:44

kyma-bot added area/ci Issues or PRs related to CI related topics cla: yes Indicates the PR's author has signed the CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 31, 2024

friedrichwilken added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 31, 2024

friedrichwilken linked an issue Jan 31, 2024 that may be closed by this pull request

auto-bump images in Create release workflow #361

Closed

3 tasks

friedrichwilken added 4 commits February 1, 2024 14:21

add scripts

99654f5

remove old script

d0f972c

add new needs

deb19cb

remove obsolete scripts

655263b

kyma-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 1, 2024

fix spelling

7dd2fac

friedrichwilken changed the title ~~add a reusable release workflow~~ use reusable workflows in create-release workflow. Feb 1, 2024

kyma-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 1, 2024

marcobebway self-requested a review February 1, 2024 16:40

marcobebway reviewed Feb 1, 2024

View reviewed changes

hack/scripts/create_changelog.sh Outdated Show resolved Hide resolved

marcobebway reviewed Feb 1, 2024

View reviewed changes

hack/scripts/render-sec-scanners-config.sh Outdated Show resolved Hide resolved

friedrichwilken and others added 7 commits February 1, 2024 21:26

rename manifest files

7c15fad

Update hack/scripts/create_changelog.sh

29df00a

Co-authored-by: Marco Bebway <[email protected]>

fix typos in create-changelog

2698b0d

simplify variables

01c0742

Update hack/scripts/render-sec-scanners-config.sh

336e3c7

Co-authored-by: Marco Bebway <[email protected]>

new to authors

4491d35

remove removal of temp files

5eb07cb

friedrichwilken added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 2, 2024

grischperl modified the milestone: 1.2.0 Feb 4, 2024

friedrichwilken and others added 11 commits February 7, 2024 09:28

update bump-sec-scanners-config sha

f5c245a

now useschanges from kyma-project/eventing-tools#68

add persmissions to scripts

78e1ae7

update sha for trigger-prow-build-job

98b0ee2

kyma-project/eventing-tools@7113a71

bump trigger-prow-build-job

5738fa5

change gh token

3f88f1e

auto-bump sec-scanners-config: 2024-02-07-10-07-42 (kyma-project#464)

440aafa

Co-authored-by: kyma-eventing-bot <kyma-eventing-bot"@users.noreply.github.com>

change token

51d7832

inherit secrets to trigger-prow=build-job-reusable

9235769

bump create-draft-release

6c27b43

bump sha for create draft release

fe5e1bc

bump redener manifests

302370e

friedrichwilken removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 8, 2024

use old create_changelog.sh

3720d1f

friedrichwilken force-pushed the add-reusable-release-workflow branch from 8968d81 to 3720d1f Compare February 8, 2024 09:49

add step to update kusomization

0b0d1e1

friedrichwilken force-pushed the add-reusable-release-workflow branch from e94291a to 0b0d1e1 Compare February 8, 2024 13:28

friedrichwilken removed a link to an issue Feb 8, 2024

auto-bump images in Create release workflow #361

Closed

3 tasks

friedrichwilken closed this Feb 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use reusable workflows in create-release workflow. #454

use reusable workflows in create-release workflow. #454

friedrichwilken commented Jan 31, 2024 •

edited

Loading

marcobebway Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 1, 2024

friedrichwilken Feb 2, 2024

friedrichwilken Feb 2, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 1, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 1, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 1, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 1, 2024 •

edited

Loading

marcobebway Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 2, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 2, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 2, 2024

marcobebway Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 2, 2024

marcobebway Feb 1, 2024

friedrichwilken Feb 2, 2024

friedrichwilken commented Feb 2, 2024

kyma-bot commented Feb 8, 2024

	name: Bump the sec-scandners-config.yaml
	name: Bump the sec-scanners-config.yaml

use reusable workflows in create-release workflow. #454

use reusable workflows in create-release workflow. #454

Conversation

friedrichwilken commented Jan 31, 2024 • edited Loading

marcobebway Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

friedrichwilken Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

friedrichwilken Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

marcobebway Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcobebway Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

friedrichwilken commented Feb 2, 2024

kyma-bot commented Feb 8, 2024

friedrichwilken commented Jan 31, 2024 •

edited

Loading

marcobebway Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 1, 2024 •

edited

Loading

friedrichwilken Feb 1, 2024 •

edited

Loading

marcobebway Feb 1, 2024 •

edited

Loading

marcobebway Feb 1, 2024 •

edited

Loading