-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use reusable workflows in create-release workflow. #454
Changes from 6 commits
a34787a
99654f5
d0f972c
deb19cb
655263b
7dd2fac
cd49f77
a78a69b
7c15fad
29df00a
2698b0d
01c0742
336e3c7
4491d35
5eb07cb
f5c245a
78e1ae7
98b0ee2
5738fa5
3f88f1e
440aafa
51d7832
9235769
6c27b43
fe5e1bc
302370e
3720d1f
0b0d1e1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -1,96 +1,78 @@ | ||||||
name: "Create release" | ||||||
name: Create release | ||||||
|
||||||
on: | ||||||
workflow_dispatch: | ||||||
inputs: | ||||||
DRY_RUN: | ||||||
type: boolean | ||||||
default: false | ||||||
description: "Dry run: setting this to `true` will skip the final step of publishing the release, leaving it as a draft-release. This can be helpful for testing and debugging. Note that the tag and release are still getting generated and the next run of this result in incremented release version. To avoid this, do not forget to delete the resulting release and tag." | ||||||
|
||||||
jobs: | ||||||
verify-release: | ||||||
name: Verify release | ||||||
runs-on: ubuntu-latest | ||||||
outputs: | ||||||
version: ${{ steps.gen-version.outputs.VERSION }} | ||||||
gen-version: | ||||||
name: Generate semantic version from branch and tags | ||||||
uses: kyma-project/eventing-tools/.github/workflows/get-version-from-release-branch-reusable.yml@main | ||||||
# outputs: VERSION | ||||||
|
||||||
steps: | ||||||
- name: Checkout code | ||||||
uses: actions/checkout@v4 | ||||||
with: | ||||||
fetch-depth: 0 | ||||||
bump-sec-scanners-config: | ||||||
name: Bump the sec-scandners-config.yaml | ||||||
needs: gen-version | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed! |
||||||
uses: kyma-project/eventing-tools/.github/workflows/bump-sec-scanners-config-reusable.yml@main | ||||||
with: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (nit) ignore for now, after noticing how the reusable workflows are used, maybe it would have been better to use the following naming instead of the current one:
but this is only my personal preference, so feel free to ignore. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That might have been a helpful solution. You would still find workflows at the expected place and the extra dir would make clear what this is, especially when we are calling them from other workflow like in this case. Learnings for the future. |
||||||
VERSION: ${{ needs.gen-version.outputs.VERSION }} | ||||||
secrets: | ||||||
BOT_PAT: ${{ secrets.BOT_PAT }} | ||||||
|
||||||
- name: Generate version number | ||||||
id: gen-version | ||||||
run: | | ||||||
# get script | ||||||
GET_VERSION=$(mktemp /tmp/get-version-from-branch.XXXXX) | ||||||
curl -L https://raw.githubusercontent.com/kyma-project/eventing-tools/main/hack/scripts/get-version-from-branch.sh -o "${GET_VERSION}" | ||||||
chmod +x "${GET_VERSION}" | ||||||
# get version via script | ||||||
VERSION=$("${GET_VERSION}") | ||||||
# push version to output environment file | ||||||
echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT | ||||||
run-unit-test: | ||||||
name: Run Unit Tests | ||||||
needs: [gen-version, bump-sec-scanners-config] | ||||||
uses: kyma-project/eventing-tools/.github/workflows/unit-test-reusable.yml@main | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why does There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It does not need it, it will just wait to run the unit tests after There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the only flow that is really needed is the |
||||||
|
||||||
- name: Check image Tag | ||||||
env: | ||||||
VERSION: ${{ steps.gen-version.outputs.VERSION }} | ||||||
run: ./scripts/check_sec-scanners-config.sh $VERSION | ||||||
run-golint-cli: | ||||||
name: Run Golint-cli | ||||||
needs: [gen-version, bump-sec-scanners-config] | ||||||
uses: kyma-project/eventing-tools/.github/workflows/lint-reusable.yml@main | ||||||
|
||||||
create-draft: | ||||||
name: Create draft release | ||||||
needs: verify-release | ||||||
runs-on: ubuntu-latest | ||||||
env: | ||||||
VERSION: ${{ needs.verify-release.outputs.VERSION }} | ||||||
outputs: | ||||||
release_id: ${{ steps.create-draft.outputs.release_id }} | ||||||
build-image: | ||||||
name: Build image | ||||||
needs: [gen-version, run-unit-test, run-golint-cli] | ||||||
uses: kyma-project/eventing-tools/.github/workflows/trigger-prow-build-job-reusable.yml@main | ||||||
with: | ||||||
VERSION: ${{ needs.gen-version.outputs.VERSION }} | ||||||
TIMEOUT: 120000 # 20 minutes | ||||||
INTERVAL: 6000 # 1 minute | ||||||
CONTEXT: "release-eventing-manager-build" | ||||||
secrets: | ||||||
BOT_PAT: ${{ secrets.BOT_PAT }} | ||||||
GH_TOKEN: ${{ secrets.GH_TOKEN }} | ||||||
|
||||||
steps: | ||||||
- name: Checkout code | ||||||
uses: actions/checkout@v4 | ||||||
with: | ||||||
fetch-depth: 0 | ||||||
create-draft-release: | ||||||
name: Create a draft release | ||||||
needs: [gen-version, run-unit-test, run-golint-cli, build-image] | ||||||
uses: kyma-project/eventing-tools/.github/workflows/create-draft-release-reusable.yml@main | ||||||
with: | ||||||
VERSION: ${{ needs.gen-version.outputs.VERSION }} | ||||||
secrets: | ||||||
BOT_PAT: ${{ secrets.BOT_PAT }} | ||||||
GH_TOKEN: ${{ secrets.GH_TOKEN }} | ||||||
|
||||||
- name: Create changelog | ||||||
env: | ||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
run: ./scripts/create_changelog.sh $VERSION | ||||||
|
||||||
- name: Create draft release | ||||||
id: create-draft | ||||||
env: | ||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
run: | | ||||||
RELEASE_ID=$(./scripts/create_draft_release.sh $VERSION) | ||||||
echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT | ||||||
|
||||||
- name: Create lightweight tag | ||||||
run: | | ||||||
git tag $VERSION | ||||||
git push origin $VERSION | ||||||
|
||||||
- name: Verify job status | ||||||
run: ./scripts/verify-status.sh ${{ github.ref_name }} 600 10 30 | ||||||
|
||||||
- name: Create and upload eventing-manager.yaml and eventing-default-cr.yaml | ||||||
env: | ||||||
PULL_BASE_REF: ${{ needs.verify-release.outputs.VERSION }} | ||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
IMG: "europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${VERSION}" | ||||||
MODULE_REGISTRY: "europe-docker.pkg.dev/kyma-project/prod/unsigned" | ||||||
KUSTOMIZE_VERSION: "v4.5.6" | ||||||
run: | | ||||||
./scripts/render_and_upload_manifests.sh | ||||||
render-and-upload-manifest: | ||||||
name: Render and upload the manifests | ||||||
needs: [gen-version, create-draft-release, build-image] | ||||||
uses: kyma-project/eventing-tools/.github/workflows/render-and-upload-manifests-reusable.yml@main | ||||||
with: | ||||||
VERSION: ${{ needs.gen-version.outputs.VERSION }} | ||||||
CR_FILE: "somedir/cr_file.yaml" | ||||||
CRD_FILE: "crd_file.yaml" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh , I forgot to change the names from tests that I have done earlier. Thanks. I fixed it. But while I fixed this, I noticed that I have done a mistake when I implemented the flow and only tested it with "somedir/cr_file.yaml". We need a way to rename the default cr, since gh cannot do it or upload it with a different name. Here is the fix: would you also take a look at this? I will update the commit sha later, once it was merged. |
||||||
secrets: | ||||||
BOT_PAT: ${{ secrets.BOT_PAT }} | ||||||
|
||||||
publish-release: | ||||||
name: Publish release | ||||||
needs: [verify-release, create-draft] | ||||||
runs-on: ubuntu-latest | ||||||
|
||||||
steps: | ||||||
- name: Checkout code | ||||||
uses: actions/checkout@v4 | ||||||
with: | ||||||
fetch-depth: 0 | ||||||
|
||||||
- name: Publish release | ||||||
env: | ||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||||||
run: ./scripts/publish_release.sh ${{ needs.create-draft.outputs.release_id }} | ||||||
if: ${{ !inputs.DRY_RUN }} | ||||||
needs: [gen-version, render-and-upload-manifest, build-image] | ||||||
name: Publish the release | ||||||
uses: kyma-project/eventing-tools/.github/workflows/publish-release-reusable.yml@main | ||||||
with: | ||||||
VERSION: ${{ needs.gen-version.outputs.VERSION }} | ||||||
secrets: | ||||||
BOT_PAT: ${{ secrets.BOT_PAT }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
#!/usr/bin/env bash | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does it make sense to move this script to the common tools repo so other repositories may use it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought about this. My intention was to give more flexibility with the changelog (similar to the |
||
|
||
# Error handling. | ||
set -o nounset # treat unset variables as an error and exit immediately. | ||
set -o errexit # exit immediately when a command fails. | ||
set -E # needs to be set if we want the ERR trap | ||
set -o pipefail # prevents errors in a pipeline from being masked | ||
|
||
RELEASE_TAG=$1 | ||
|
||
REPOSITORY=${REPOSITORY:-kyma-project/eventing-manager} | ||
GITHUB_URL=https://api.github.com/repos/${REPOSITORY} | ||
GITHUB_AUTH_HEADER="Authorization: token ${GH_TOKEN}" | ||
CHANGELOG_FILE="CHANGELOG.md" | ||
|
||
# The git describe --tag --abbrev=0 command is used to find the most recent tag that is reachable from a commit. | ||
# The --tag option tells git describe to consider any tag found in the refs/tags namespace, enabling matching a lightweight (non-annotated) tag. | ||
PREVIOUS_RELEASE=$(git describe --tags --abbrev=0) | ||
|
||
# Generate the changelog in the CHANGELOG.md. | ||
echo "## What has changed" >>${CHANGELOG_FILE} | ||
|
||
# Iterate over all commits since the previous release. | ||
git log "${PREVIOUS_RELEASE}"..HEAD --pretty=tformat:"%h" --reverse | while read -r commit; do | ||
# If the author of the commit is not kyma-bot, show append the commit message to the changelog. | ||
friedrichwilken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
COMMIT_AUTHOR=$(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/commits/${commit}" | jq -r '.author.login') | ||
if [ "${COMMIT_AUTHOR}" != "kyma-bot" ]; then | ||
git show -s "${commit}" --format="* %s by @${COMMIT_AUTHOR}" >>${CHANGELOG_FILE} | ||
fi | ||
done | ||
|
||
# Create a new file (with a unique name based on the process ID of the current shell). | ||
NEW_CONTRIB=$$.new | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (nit) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
|
||
# Find unique authors that contribute since the last release, but not before it, and to the NEW_CONTRIB file. | ||
join -v2 \ | ||
<(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/compare/$(git rev-list --max-parents=0 HEAD)...${PREVIOUS_RELEASE}" | jq -r '.commits[].author.login' | sort -u) \ | ||
<(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/compare/${PREVIOUS_RELEASE}...HEAD" | jq -r '.commits[].author.login' | sort -u) >${NEW_CONTRIB} | ||
|
||
# Add new contributors to the 'new contributors' section of the changelog. | ||
if [ -s ${NEW_CONTRIB} ]; then | ||
echo -e "\n## New contributors" >>${CHANGELOG_FILE} | ||
while read -r user; do | ||
REF_PR=$(grep "@${user}" ${CHANGELOG_FILE} | head -1 | grep -o " (#[0-9]\+)" || true) | ||
if [ -n "${REF_PR}" ]; then #reference found | ||
REF_PR=" in ${REF_PR}" | ||
fi | ||
echo "* @${user} made first contribution${REF_PR}" >>${CHANGELOG_FILE} | ||
done <${NEW_CONTRIB} | ||
fi | ||
|
||
# Append link to the full-changelog this changelog. | ||
echo -e "\n**Full changelog**: https://github.com/$REPOSITORY/compare/${PREVIOUS_RELEASE}...${RELEASE_TAG}" >>${CHANGELOG_FILE} | ||
|
||
# Cleanup the NEW_CONTRIB file. | ||
rm ${NEW_CONTRIB} || echo "cleaned up" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (nit) IMO, no need to echo anything here, because removing the file is an internal housekeeping. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I removed the whole section. It is just not needed. |
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -e | ||
|
||
# This scrpit generates the sec-scanners-config by fetching all relevant images. | ||
|
||
TAG=$1 | ||
OUTPUT_FILE=${2:-"sec-scanners-config.yaml"} | ||
WEBHOOK_FILE=${3-"config/webhook/kustomization.yaml"} | ||
PUBLISHER_FILE=${4-"config/manager/manager.yaml"} | ||
|
||
# Fetch Webhook Image. | ||
echo "fetching webhook image from ${WEBHOOK_FILE}" | ||
WEBHOOK_IMAGE=$(yq eval '.images[0].newName' <"$WEBHOOK_FILE") | ||
WEBHOOK_TAG=$(yq eval '.images[0].newTag' <"$WEBHOOK_FILE") | ||
echo -e "webhook image is ${WEBHOOK_IMAGE}:${WEBHOOK_TAG} \n" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (nit) Maybe set the var There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like the idea. Done. |
||
|
||
# Fetch Publisher Image. | ||
echo "fetching publisher image from ${PUBLISHER_FILE}" | ||
PUBLISHER_IMAGE=$(yq eval '.spec.template.spec.containers[0].env[] | select(.name == "PUBLISHER_IMAGE") | .value' <"${PUBLISHER_FILE}") | ||
echo -e "publisher image is ${PUBLISHER_IMAGE} \n" | ||
|
||
# Generating File. | ||
echo -e "generating to ${OUTPUT_FILE} \n" | ||
cat <<EOF | tee "${OUTPUT_FILE}" | ||
# Dont edit this file; it is autogenerated by github action 'Create release'. | ||
# The value for the publisher image are extracted from ${PUBLISHER_FILE}. | ||
# The value for the webhook image are extracted from ${WEBHOOK_FILE}.yaml. | ||
friedrichwilken marked this conversation as resolved.
Show resolved
Hide resolved
|
||
module-name: eventing | ||
rc-tag: ${TAG} | ||
protecode: | ||
- europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${TAG} | ||
- ${PUBLISHER_IMAGE} | ||
- ${WEBHOOK_IMAGE}:${WEBHOOK_TAG} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After applying this comment, these lines should read: - ${PUBLISHER_IMAGE}
- ${WEBHOOK_IMAGE} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated. |
||
whitesource: | ||
language: golang-mod | ||
subprojects: false | ||
exclude: | ||
- "**/test/**" | ||
- "**/*_test.go" | ||
- "/hack/**" | ||
EOF |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
#!/usr/bin/env bash | ||
|
||
# standard bash error handling | ||
set -o nounset # treat unset variables as an error and exit immediately. | ||
set -o errexit # exit immediately when a command fails. | ||
set -E # needs to be set if we want the ERR trap | ||
set -o pipefail # prevents errors in a pipeline from being masked | ||
|
||
VERSION=$1 | ||
IMG="europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${VERSION}" make render-manifest |
This file was deleted.
This file was deleted.
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this always return the workflow yaml from
main~head
? If yes, can we use a commit SHA instead, just in casemain~head
gets broken for some reason.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, I am happy that you ask about this because I was also between these two options.
Sticking this to a commit sha makes this less susceptible to bug, so yes, let's do that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed it to a commit sha everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to update the
create-draft-release
workflow sha, after this PR got merged.