Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(operator): build, release, and productization preparation #5465

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
+7,940 −177
Open

feat(operator): build, release, and productization preparation #5465

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
a2132ed
feat(operator): build, release, and productization preparation
jsenko Nov 4, 2024
cd85037
refactor(operator): code improvements
jsenko Nov 7, 2024
16c465d
chore(operator): update install file
jsenko Nov 13, 2024
d72bf91
chore(operator): cleanup
jsenko Nov 14, 2024
2f7ada4
chore(operator): configure published image tags for snapshot versions
jsenko Nov 14, 2024
04bc419
docs(operator): update CSV description
jsenko Nov 14, 2024
75b5a2c
refactor(operator): centralize configuration access
jsenko Nov 14, 2024
b37d54b
chore(operator): rebind env. variable
jsenko Nov 14, 2024
cd31cab
chore(operator): remove separate test application.properties
jsenko Nov 14, 2024
c9d4aae
test(operator): do not run smoke tests with OLM, just deploy
jsenko Nov 14, 2024
07009ee
chore(operator): cleanup
jsenko Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 138 additions & 27 deletions .github/workflows/operator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,72 +1,183 @@
name: Operator tests Workflow
name: Operator Test Workflow
on:
push:
branches:
- main
paths-ignore:
- '.gitignore'
- 'LICENSE'
- 'README*'
- 'docs/**'
- '.github/workflows/**'
branches: [ main ]
- 'operator/install/**'

pull_request:
branches:
- main
paths-ignore:
- '.gitignore'
- 'LICENSE'
- 'README*'
- 'docs/**'
branches: [ main ]
- '.github/workflows/**'
- 'operator/install/**'

concurrency:
# Only run once for latest commit per ref and cancel other (previous) runs.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
tests:
name: Operator Basic tests
runs-on: ubuntu-22.04

operator-build-test:
name: Build and Test Operator
runs-on: ubuntu-latest
if: github.repository_owner == 'Apicurio' && !contains(github.event.*.labels.*.name, 'DO NOT MERGE')
steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Configure env. variables 1
run: |
echo "UUID=$(uuidgen)" >> $GITHUB_ENV
# We need tho push the bundle image because opm always pulls; the catalog image because OLM
# deploys it with `imagePullPolicy: Always`; and also the operator image itself for the same reason.
- name: Configure env. variables 2
run: |
echo "IMAGE=ttl.sh/apicurio-registry-operator-$UUID:8h" >> $GITHUB_ENV
echo "BUNDLE_IMAGE=ttl.sh/apicurio-registry-operator-bundle-$UUID:8h" >> $GITHUB_ENV
andreaTP marked this conversation as resolved.
Show resolved Hide resolved
echo "CATALOG_IMAGE=ttl.sh/apicurio-registry-operator-catalog-$UUID:8h" >> $GITHUB_ENV
echo "ADDITIONAL_CATALOG_IMAGE=" >> $GITHUB_ENV
- name: Checkout ${{ github.ref }}
uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: temurin
cache: maven

- name: Enable port-forwarding
run: |
sudo apt-get -y install socat
sudo apt-get -y install socat
- name: Setup Minikube-Kubernetes
- name: Setup Minikube
uses: manusa/[email protected]
with:
minikube version: v1.28.0
minikube version: v1.33.1
kubernetes version: v1.25.0
github token: ${{ secrets.GITHUB_TOKEN }}
start args: '--force'
andreaTP marked this conversation as resolved.
Show resolved Hide resolved

- name: Enable minikube ingress
- name: Enable minikube features
run: |
minikube addons enable ingress
minikube addons enable olm
- name: Setup minikube tunnel
run: |
minikube tunnel &
minikube tunnel &
- name: Build and run local tests on Minikube
run: |
cd operator
make BUILD_OPTS="--no-transfer-progress" build
- name: Build temporary operator image
run: |
cd operator
make image-build image-push
- name: Build temporary operator bundle
run: |
cd operator
make bundle
- name: Build temporary operator catalog
run: |
cd operator
make catalog
- name: Run remote tests on Minikube
run: |
cd operator
mvn clean verify --no-transfer-progress -DskipOperatorTests=false -Dtest.operator.deployment=remote \
-Dtest.operator.deployment-target=minikube -Dtest.operator.catalog-image=$(make catalog-image-get)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of flags to enable/disable stuff when running the OLM Test, my proposal is to move the OLM test to a separate Maven sub-module and execute it in isolation.
wdyt?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's necessary. We have only two flags - one for ingresses and a second for OLM. If you want to separate it, let's do it in a separate PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to keep looking up at the implementations to understand "what is enabled with which flag".

It also mixes concerns, moving to a separate sub-module makes everything cleaner and logically separated.

operator-publish:
name: Publish Operator
runs-on: ubuntu-latest
needs: operator-build-test
if: github.event_name == 'push'
steps:

# Do not publish (version)-snapshot tag, instead use `latest-snapshot`.
- name: Configure env. variables
run: |
echo "IMAGE_TAG=latest-snapshot" >> $GITHUB_ENV
echo "BUNDLE_IMAGE_TAG=latest-snapshot" >> $GITHUB_ENV
- name: Checkout ${{ github.ref }}
uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'
distribution: temurin
cache: maven

- name: Build
run: |
cd operator
make BUILD_OPTS="--no-transfer-progress" SKIP_TESTS=true build
- name: Login to quay.io registry
run: |
docker login -u "${{ secrets.QUAY_USERNAME }}" -p "${{ secrets.QUAY_PASSWORD }}" quay.io
- name: Workaround jackson-coreutils
- name: Build and publish operator image # TODO: Also push to DockerHub Registry
run: |
# upstream issue: https://github.com/java-json-tools/jackson-coreutils/issues/59
rm -rf ~/.m2/repository/com/github/java-json-tools
mkdir -p /tmp/coreutils-workaround
( cd /tmp/coreutils-workaround && mvn dependency:get -DremoteRepositories=https://repo1.maven.org/maven2 -Dartifact=com.github.java-json-tools:jackson-coreutils:2.0 )
cd operator
make image-build image-push
- name: Run the tests in local mode
run: ./mvnw clean verify -P '!external_repos' -DskipOperatorTests=false -pl operator/controller -am
- name: Build and publish operator bundle
run: |
cd operator
make bundle
andreaTP marked this conversation as resolved.
Show resolved Hide resolved
operator-check-install-file:
name: Check Operator Install file
runs-on: ubuntu-latest
needs: operator-publish
steps:

- name: Checkout Code
uses: actions/checkout@v3

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: '17'
distribution: temurin
cache: maven

- name: Build
run: |
cd operator
make BUILD_OPTS="--no-transfer-progress" SKIP_TESTS=true build
- name: Update install file
run: |
cd operator
make INSTALL_FILE=install/install.yaml dist-install-file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have a job to compare the resulting yaml with the expected and fail instead of automatically pushing it back.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly this step and the following can be added to operator-build-test and we can remove the operator-check-install-file job

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job can stay separate, it serves a specific and separate purpose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can for sure, but it shouldn't, the CI is already pretty heavy, and rebuilding the Operator just to run a diff is not worth it in my opinion.

Happy to ask another pair of eyes in case is necessary.

- name: Run the tests in remote mode
- name: Check install file
run: |
./mvnw clean package -P '!external_repos' -pl operator/controller -am -Dquarkus.container-image.build=true
./mvnw clean verify -P '!external_repos' -DskipOperatorTests=false -pl operator/controller -am -Dtest.operator.deployment=remote -Dtest.operator.deployment-target=minikube
git add operator/install/install.yaml || true
if ! git diff --cached --exit-code &> /dev/null; then
echo 'Install file needs to be updated. Please run "cd operator; make INSTALL_FILE=install/install.yaml dist-install-file" and commit the result.';
exit 1;
else
echo "No changes to the install file.";
fi
2 changes: 2 additions & 0 deletions operator/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
controller/src/main/deploy/crd/apicurioregistries3.registry.apicur.io-v1.yml
install/apicurio-registry-operator-*-SNAPSHOT.yaml
Loading
Loading