Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use reusable workflows in create-release workflow. #454

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
a34787a
add a reusable release workflow
friedrichwilken Jan 31, 2024
99654f5
add scripts
friedrichwilken Feb 1, 2024
d0f972c
remove old script
friedrichwilken Feb 1, 2024
deb19cb
add new needs
friedrichwilken Feb 1, 2024
655263b
remove obsolete scripts
friedrichwilken Feb 1, 2024
7dd2fac
fix spelling
friedrichwilken Feb 1, 2024
cd49f77
pin workflows to commit sha
friedrichwilken Feb 1, 2024
a78a69b
fix typo
friedrichwilken Feb 1, 2024
7c15fad
rename manifest files
friedrichwilken Feb 1, 2024
29df00a
Update hack/scripts/create_changelog.sh
friedrichwilken Feb 1, 2024
2698b0d
fix typos in create-changelog
friedrichwilken Feb 2, 2024
01c0742
simplify variables
friedrichwilken Feb 2, 2024
336e3c7
Update hack/scripts/render-sec-scanners-config.sh
friedrichwilken Feb 2, 2024
4491d35
new to authors
friedrichwilken Feb 2, 2024
5eb07cb
remove removal of temp files
friedrichwilken Feb 2, 2024
f5c245a
update bump-sec-scanners-config sha
friedrichwilken Feb 7, 2024
78e1ae7
add persmissions to scripts
friedrichwilken Feb 7, 2024
98b0ee2
update sha for trigger-prow-build-job
friedrichwilken Feb 7, 2024
5738fa5
bump trigger-prow-build-job
friedrichwilken Feb 7, 2024
3f88f1e
change gh token
friedrichwilken Feb 7, 2024
440aafa
auto-bump sec-scanners-config: 2024-02-07-10-07-42 (#464)
kyma-eventing-bot Feb 7, 2024
51d7832
change token
friedrichwilken Feb 7, 2024
9235769
inherit secrets to trigger-prow=build-job-reusable
friedrichwilken Feb 7, 2024
6c27b43
bump create-draft-release
friedrichwilken Feb 7, 2024
fe5e1bc
bump sha for create draft release
friedrichwilken Feb 7, 2024
302370e
bump redener manifests
friedrichwilken Feb 7, 2024
3720d1f
use old create_changelog.sh
friedrichwilken Feb 8, 2024
0b0d1e1
add step to update kusomization
friedrichwilken Feb 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 64 additions & 83 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
@@ -1,96 +1,77 @@
name: "Create release"
name: Create release

on:
workflow_dispatch:
inputs:
DRY_RUN:
type: boolean
default: false
description: "Dry run: setting this to `true` will skip the final step of publishing the release, leaving it as a draft-release. This can be helpful for testing and debugging. Note that the tag and release are still getting generated and the next run of this result in incremented release version. To avoid this, do not forget to delete the resulting release and tag."

jobs:
verify-release:
name: Verify release
runs-on: ubuntu-latest
outputs:
version: ${{ steps.gen-version.outputs.VERSION }}
gen-version:
name: Generate semantic version from branch and tags
uses: kyma-project/eventing-tools/.github/workflows/get-version-from-release-branch-reusable.yml@d491378dbbeef63bf8616f1a1b2b1a49ea5f9eeb
# outputs: VERSION

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
bump-sec-scanners-config:
name: Bump the sec-scanners-config.yaml
needs: gen-version
uses: kyma-project/eventing-tools/.github/workflows/bump-sec-scanners-config-reusable.yml@e8d1cd652dd636ad14b420da984f0bd4bf84ee0a
with:
VERSION: ${{ needs.gen-version.outputs.VERSION }}
secrets:
BOT_PAT: ${{ secrets.BOT_PAT }}

- name: Generate version number
id: gen-version
run: |
# get script
GET_VERSION=$(mktemp /tmp/get-version-from-branch.XXXXX)
curl -L https://raw.githubusercontent.com/kyma-project/eventing-tools/main/hack/scripts/get-version-from-branch.sh -o "${GET_VERSION}"
chmod +x "${GET_VERSION}"
# get version via script
VERSION=$("${GET_VERSION}")
# push version to output environment file
echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
run-unit-test:
name: Run Unit Tests
needs: [gen-version, bump-sec-scanners-config]
uses: kyma-project/eventing-tools/.github/workflows/unit-test-reusable.yml@d491378dbbeef63bf8616f1a1b2b1a49ea5f9eeb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does run-unit-test need bump-sec-scanners-config?

Copy link
Contributor Author

@friedrichwilken friedrichwilken Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not need it, it will just wait to run the unit tests after bump-sec-scanners-config (because of the PR we create in the that part). In general, all the jobs will run consecutively. Only lint and unit-test will run in parallel because they are independent of each other. That flow is controlled by the need instructions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only flow that is really needed is the gen-version because only if it get's marked as needs the output (VERSION) gets available in the other jobs.


- name: Check image Tag
env:
VERSION: ${{ steps.gen-version.outputs.VERSION }}
run: ./scripts/check_sec-scanners-config.sh $VERSION
run-golint-cli:
name: Run Golint-cli
needs: [gen-version, bump-sec-scanners-config]
uses: kyma-project/eventing-tools/.github/workflows/lint-reusable.yml@d491378dbbeef63bf8616f1a1b2b1a49ea5f9eeb

create-draft:
name: Create draft release
needs: verify-release
runs-on: ubuntu-latest
env:
VERSION: ${{ needs.verify-release.outputs.VERSION }}
outputs:
release_id: ${{ steps.create-draft.outputs.release_id }}
build-image:
name: Build image
needs: [gen-version, run-unit-test, run-golint-cli]
uses: kyma-project/eventing-tools/.github/workflows/trigger-prow-build-job-reusable.yml@f509756e135aa2255cfd9e96c833fe5c6e511b3b
with:
VERSION: ${{ needs.gen-version.outputs.VERSION }}
TIMEOUT: 1200000 # 20 minutes
INTERVAL: 60000 # 1 minute
CONTEXT: "release-eventing-manager-build"
COMMIT_REF: ${{ github.ref_name }}
secrets: inherit

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
create-draft-release:
name: Create a draft release
needs: [gen-version, run-unit-test, run-golint-cli, build-image]
uses: kyma-project/eventing-tools/.github/workflows/create-draft-release-reusable.yml@5c713986ad2249557ad12481a3608480f548975c
with:
VERSION: ${{ needs.gen-version.outputs.VERSION }}
secrets:
BOT_PAT: ${{ secrets.BOT_PAT }}
GH_TOKEN: ${{ secrets.GH_TOKEN }}

- name: Create changelog
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./scripts/create_changelog.sh $VERSION

- name: Create draft release
id: create-draft
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
RELEASE_ID=$(./scripts/create_draft_release.sh $VERSION)
echo "release_id=$RELEASE_ID" >> $GITHUB_OUTPUT

- name: Create lightweight tag
run: |
git tag $VERSION
git push origin $VERSION

- name: Verify job status
run: ./scripts/verify-status.sh ${{ github.ref_name }} 600 10 30

- name: Create and upload eventing-manager.yaml and eventing-default-cr.yaml
env:
PULL_BASE_REF: ${{ needs.verify-release.outputs.VERSION }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
IMG: "europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${VERSION}"
MODULE_REGISTRY: "europe-docker.pkg.dev/kyma-project/prod/unsigned"
KUSTOMIZE_VERSION: "v4.5.6"
run: |
./scripts/render_and_upload_manifests.sh
render-and-upload-manifest:
name: Render and upload the manifests
needs: [gen-version, create-draft-release, build-image]
uses: kyma-project/eventing-tools/.github/workflows/render-and-upload-manifests-reusable.yml@5c713986ad2249557ad12481a3608480f548975c
with:
VERSION: ${{ needs.gen-version.outputs.VERSION }}
CR_FILE: eventing-default-cr.yaml
CRD_FILE: eventing-manager.yaml
secrets:
BOT_PAT: ${{ secrets.BOT_PAT }}

publish-release:
name: Publish release
needs: [verify-release, create-draft]
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Publish release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./scripts/publish_release.sh ${{ needs.create-draft.outputs.release_id }}
if: ${{ !inputs.DRY_RUN }}
needs: [gen-version, render-and-upload-manifest, build-image]
name: Publish the release
uses: kyma-project/eventing-tools/.github/workflows/publish-release-reusable.yml@d491378dbbeef63bf8616f1a1b2b1a49ea5f9eeb
with:
VERSION: ${{ needs.gen-version.outputs.VERSION }}
secrets:
BOT_PAT: ${{ secrets.BOT_PAT }}
51 changes: 51 additions & 0 deletions hack/scripts/create_changelog.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
Copy link
Contributor

@marcobebway marcobebway Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to move this script to the common tools repo so other repositories may use it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this. My intention was to give more flexibility with the changelog (similar to the sec-scanners-config and render-manifest). But you are right, in this case it is not needed at all. I added this PR to do so.


PREVIOUS_RELEASE=$2 # for testability

# standard bash error handling
set -o nounset # treat unset variables as an error and exit immediately.
set -o errexit # exit immediately when a command fails.
set -E # needs to be set if we want the ERR trap
set -o pipefail # prevents errors in a pipeline from being masked

RELEASE_TAG=$1

REPOSITORY=${REPOSITORY:-kyma-project/eventing-manager}
GITHUB_URL=https://api.github.com/repos/${REPOSITORY}
GITHUB_AUTH_HEADER="Authorization: token ${GITHUB_TOKEN}"
CHANGELOG_FILE="CHANGELOG.md"

if [ "${PREVIOUS_RELEASE}" == "" ]; then
PREVIOUS_RELEASE=$(git describe --tags --abbrev=0)
fi

echo "## What has changed" >>${CHANGELOG_FILE}

git log ${PREVIOUS_RELEASE}..HEAD --pretty=tformat:"%h" --reverse | while read -r commit; do
COMMIT_AUTHOR=$(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/commits/${commit}" | jq -r '.author.login')
if [ "${COMMIT_AUTHOR}" != "kyma-bot" ]; then
git show -s ${commit} --format="* %s by @${COMMIT_AUTHOR}" >>${CHANGELOG_FILE}
fi
done

NEW_CONTRIB=$$.new
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) $$.new -> $$.authors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


join -v2 \
<(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/compare/$(git rev-list --max-parents=0 HEAD)...${PREVIOUS_RELEASE}" | jq -r '.commits[].author.login' | sort -u) \
<(curl -H "${GITHUB_AUTH_HEADER}" -sS "${GITHUB_URL}/compare/${PREVIOUS_RELEASE}...HEAD" | jq -r '.commits[].author.login' | sort -u) >${NEW_CONTRIB}

if [ -s ${NEW_CONTRIB} ]; then
echo -e "\n## New contributors" >>${CHANGELOG_FILE}
while read -r user; do
REF_PR=$(grep "@${user}" ${CHANGELOG_FILE} | head -1 | grep -o " (#[0-9]\+)" || true)
if [ -n "${REF_PR}" ]; then #reference found
REF_PR=" in ${REF_PR}"
fi
echo "* @${user} made first contribution${REF_PR}" >>${CHANGELOG_FILE}
done <${NEW_CONTRIB}
fi

echo -e "\n**Full changelog**: https://github.com/$REPOSITORY/compare/${PREVIOUS_RELEASE}...${RELEASE_TAG}" >>${CHANGELOG_FILE}

# cleanup
rm ${NEW_CONTRIB} || echo "cleaned up"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) IMO, no need to echo anything here, because removing the file is an internal housekeeping.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the whole section. It is just not needed.

47 changes: 47 additions & 0 deletions hack/scripts/render-sec-scanners-config.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env bash

set -e

# This scrpit generates the sec-scanners-config by fetching all relevant images.

TAG=$1
OUTPUT_FILE=${2:-"sec-scanners-config.yaml"}
WEBHOOK_FILE=${3-"config/webhook/kustomization.yaml"}
PUBLISHER_FILE=${4-"config/manager/manager.yaml"}
KUSTOMIZATION_FILE=${5-"config/manager/kustomization.yaml"}
# Fetch Webhook Image.
echo "fetching webhook image from ${WEBHOOK_FILE}"
WEBHOOK_IMAGE=$(yq eval '.images[0].newName' <"$WEBHOOK_FILE")
WEBHOOK_TAG=$(yq eval '.images[0].newTag' <"$WEBHOOK_FILE")
WEBHOOK_IMAGE="${WEBHOOK_IMAGE}:$WEBHOOK_TAG"
echo -e "webhook image is ${WEBHOOK_IMAGE} \n"

# Fetch Publisher Image.
echo "fetching publisher image from ${PUBLISHER_FILE}"
PUBLISHER_IMAGE=$(yq eval '.spec.template.spec.containers[0].env[] | select(.name == "PUBLISHER_IMAGE") | .value' <"${PUBLISHER_FILE}")
echo -e "publisher image is ${PUBLISHER_IMAGE} \n"

# Generating File.
echo -e "generating to ${OUTPUT_FILE} \n"
cat <<EOF | tee "${OUTPUT_FILE}"
# Dont edit this file; it is autogenerated by github action 'Create release'.
# The value for the publisher image is extracted from ${PUBLISHER_FILE}.
# The value for the webhook image is extracted from ${WEBHOOK_FILE}.
module-name: eventing
rc-tag: ${TAG}
protecode:
- europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${TAG}
- ${PUBLISHER_IMAGE}
- ${WEBHOOK_IMAGE}
whitesource:
language: golang-mod
subprojects: false
exclude:
- "**/test/**"
- "**/*_test.go"
- "/hack/**"
EOF

# Bump kustomization file
sed -i "/images.newTag/c\images.newTag: ${VERSION}" "${KUSTOMIZATION_FILE}"
awk -v ntv="$VERSION" '/newTag:/ {print $1 ":" " " ntv} !/newTag:/' "$KUSTOMIZATION_FILE" >tmp_file && mv tmp_file "$KUSTOMIZATION_FILE"
10 changes: 10 additions & 0 deletions hack/scripts/render_crd.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash

# standard bash error handling
set -o nounset # treat unset variables as an error and exit immediately.
set -o errexit # exit immediately when a command fails.
set -E # needs to be set if we want the ERR trap
set -o pipefail # prevents errors in a pipeline from being masked

VERSION=$1
IMG="europe-docker.pkg.dev/kyma-project/prod/eventing-manager:${VERSION}" make render-manifest
14 changes: 0 additions & 14 deletions scripts/check_release_tag.sh

This file was deleted.

34 changes: 0 additions & 34 deletions scripts/check_sec-scanners-config.sh

This file was deleted.

56 changes: 0 additions & 56 deletions scripts/create_changelog.sh

This file was deleted.

Loading
Loading