feat: validate kustomize build of tasks and pipelines #1792

tisutisu · 2025-01-07T07:59:15Z

This PR adds a CI check to validate kustomize build is successful and also checks that PR authors are committing built manifests files of task/pipeline change with the help of hack/build-manifests.sh script with their task/pipeline changes.
Part of story: https://issues.redhat.com/browse/STONEBLD-3042

hack/build-manifests.sh

yftacherzog · 2025-01-09T12:58:04Z

task/generate-odcs-compose/0.1/generate-odcs-compose.yaml

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

Will ignore building manifest for this task then

yftacherzog · 2025-01-09T12:58:36Z

task/provision-env-with-ephemeral-namespace/0.1/provision-env-with-ephemeral-namespace.yaml

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

same for this task as well

yftacherzog · 2025-01-09T12:59:22Z

task/verify-signed-rpms/0.1/verify-signed-rpms.yaml

This task is mirrored from another repo, so changes made directly will eventually be overwritten by changes made there.

same for this task as well

tkdchen

Could you please describe the solution in the commit message?

hack/build-manifests.sh

tkdchen · 2025-01-13T04:30:05Z

hack/verify-manifests.sh

+            echo "Please run ./hack/build-manifests.sh and update your PR" >&2;
+            exit 1;
+        }
+    fi


Looks like ./hack/build-manifests.sh becomes a mandatory part of the development workflow when working with tasks and pipelines. Can you describe this? Probably inside the build-manifests.sh script itself at least.

Described it in the README change here. Do you think it is sufficient or need to be added to the build-manifests.sh script as well?

The content linked tells what need to do for task and pipeline authors, it is not a description to the whole solution. In my mind, the description could tell, for example, what the task and pipeline authors should do, what the maintainers should do, how the kustomized manifests are validated, whether this validation changes the build-definitions task and pipelines contribution workflow, even the build-and-push, etc.

Updated the README now. We don't need to change anything on the build-and-push script.

tisutisu · 2025-01-15T06:15:21Z

/retest

tisutisu · 2025-01-15T07:35:11Z

/retest

README.md

tkdchen · 2025-01-21T07:42:33Z

hack/build-manifests.sh

+# with the task name separated by a space, for example:
+# SKIP_TASKS="git-clone init"
+
+SKIP_TASKS="generate-odcs-compose provision-env-with-ephemeral-namespace verify-signed-rpms"


Why allowing to skip tasks and pipelines?

These tasks are mirrored from another repo, there is some automation which syncs them. If we change anything manually, that will be reverted. So skipped them.

hack/build-manifests.sh

tkdchen · 2025-01-21T08:05:51Z

hack/verify-manifests.sh

+            echo "Please run ./hack/build-manifests.sh and update your PR" >&2;
+            exit 1;
+        }
+    fi


The content linked tells what need to do for task and pipeline authors, it is not a description to the whole solution. In my mind, the description could tell, for example, what the task and pipeline authors should do, what the maintainers should do, how the kustomized manifests are validated, whether this validation changes the build-definitions task and pipelines contribution workflow, even the build-and-push, etc.

pipelines/template-build/template-build.yaml

kdudka

There is nothing left to review in this pull request from me as code owner of the sast-* tasks.

task/sast-coverity-check/0.2/sast-coverity-check.yaml

chmeliik

LGTM, just minor details

task/fbc-validation/0.1/fbc-validation.yaml

hack/build-manifests.sh

kdudka

The change in sast-coverity-check looks good.

tisutisu · 2025-02-03T10:38:10Z

@ligangty @yma96 @gbenhaim @amisstea @Omeramsc @avi-biton @konflux-ci/integration-service-maintainers @konflux-ci/ec could you please help with review/approve this PR?

chmeliik · 2025-02-03T10:40:58Z

After #1792 (comment), we don't need most of those reviews anymore 🙂

Just these

@konflux-ci/integration-service-maintainers
@ligangty or @yma96

dirgim

LGTM

yma96

LGTM

tisutisu force-pushed the kustomize-build branch from fdbe145 to 3033272 Compare January 7, 2025 08:08

tisutisu marked this pull request as ready for review January 7, 2025 08:11

tisutisu requested a review from a team as a code owner January 7, 2025 08:11

tisutisu force-pushed the kustomize-build branch 3 times, most recently from 29e718a to fe32af3 Compare January 7, 2025 12:45

chmeliik mentioned this pull request Jan 7, 2025

extend the sast-coverity-check CI task to support buildful scanning #1653

Merged

chmeliik reviewed Jan 7, 2025

View reviewed changes

hack/build-manifests.sh Outdated Show resolved Hide resolved

tisutisu force-pushed the kustomize-build branch from fe32af3 to c9884f3 Compare January 9, 2025 12:27

tisutisu requested review from ligangty, yma96, amisstea, avi-biton, gbenhaim, yftacherzog, Omeramsc and a team as code owners January 9, 2025 12:27

tisutisu marked this pull request as draft January 9, 2025 12:32

yftacherzog reviewed Jan 9, 2025

View reviewed changes

tisutisu force-pushed the kustomize-build branch 2 times, most recently from 55f6f38 to 04430c6 Compare January 13, 2025 03:51

tisutisu marked this pull request as ready for review January 13, 2025 03:53

tisutisu force-pushed the kustomize-build branch from 04430c6 to fb458d7 Compare January 13, 2025 04:06

tkdchen reviewed Jan 13, 2025

View reviewed changes

tisutisu force-pushed the kustomize-build branch from fb458d7 to 7aaed3c Compare January 15, 2025 05:28

tkdchen reviewed Jan 21, 2025

View reviewed changes

tisutisu force-pushed the kustomize-build branch 4 times, most recently from d3a6320 to d03d2ea Compare January 27, 2025 08:22

chmeliik reviewed Jan 27, 2025

View reviewed changes

pipelines/template-build/template-build.yaml Outdated Show resolved Hide resolved

tisutisu force-pushed the kustomize-build branch from d03d2ea to 022edac Compare January 31, 2025 06:44

tisutisu requested a review from kdudka as a code owner January 31, 2025 06:44

tisutisu force-pushed the kustomize-build branch from 022edac to 93d0038 Compare January 31, 2025 07:10

kdudka reviewed Jan 31, 2025

View reviewed changes

task/sast-coverity-check/0.2/sast-coverity-check.yaml Outdated Show resolved Hide resolved

tisutisu force-pushed the kustomize-build branch from 93d0038 to c8cb354 Compare January 31, 2025 12:40

chmeliik reviewed Jan 31, 2025

View reviewed changes

task/fbc-validation/0.1/fbc-validation.yaml Outdated Show resolved Hide resolved

hack/build-manifests.sh Outdated Show resolved Hide resolved

chmeliik previously approved these changes Feb 3, 2025

View reviewed changes

kdudka previously approved these changes Feb 3, 2025

View reviewed changes

tisutisu force-pushed the kustomize-build branch from de28fc0 to e800ade Compare February 3, 2025 07:53

Validate kustomize build in CI

f5ac2f6

tisutisu dismissed stale reviews from kdudka and chmeliik via f5ac2f6 February 3, 2025 08:35

tisutisu force-pushed the kustomize-build branch from e800ade to f5ac2f6 Compare February 3, 2025 08:35

chmeliik approved these changes Feb 3, 2025

View reviewed changes

tisutisu enabled auto-merge February 3, 2025 09:19

tisutisu disabled auto-merge February 3, 2025 10:20

dirgim approved these changes Feb 4, 2025

View reviewed changes

yma96 approved these changes Feb 5, 2025

View reviewed changes

tisutisu added this pull request to the merge queue Feb 5, 2025

Merged via the queue into konflux-ci:main with commit 1f7ea18 Feb 5, 2025
17 checks passed

rh-tap-build-team bot mentioned this pull request Feb 5, 2025

build-definitions update redhat-appstudio/infra-deployments#5462

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: validate kustomize build of tasks and pipelines #1792

feat: validate kustomize build of tasks and pipelines #1792

tisutisu commented Jan 7, 2025 •

edited

Loading

yftacherzog Jan 9, 2025

tisutisu Jan 10, 2025

yftacherzog Jan 9, 2025

tisutisu Jan 10, 2025

yftacherzog Jan 9, 2025

tisutisu Jan 10, 2025

tkdchen left a comment

tkdchen Jan 13, 2025

tisutisu Jan 13, 2025

tkdchen Jan 21, 2025 •

edited

Loading

tisutisu Jan 22, 2025

tisutisu commented Jan 15, 2025

tisutisu commented Jan 15, 2025

tkdchen Jan 21, 2025

tisutisu Jan 22, 2025

tkdchen Jan 21, 2025 •

edited

Loading

kdudka left a comment

chmeliik left a comment

kdudka left a comment

tisutisu commented Feb 3, 2025

chmeliik commented Feb 3, 2025 •

edited

Loading

dirgim left a comment

yma96 left a comment

feat: validate kustomize build of tasks and pipelines #1792

feat: validate kustomize build of tasks and pipelines #1792

Conversation

tisutisu commented Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkdchen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkdchen Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tisutisu commented Jan 15, 2025

tisutisu commented Jan 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkdchen Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

kdudka left a comment

Choose a reason for hiding this comment

chmeliik left a comment

Choose a reason for hiding this comment

kdudka left a comment

Choose a reason for hiding this comment

tisutisu commented Feb 3, 2025

chmeliik commented Feb 3, 2025 • edited Loading

dirgim left a comment

Choose a reason for hiding this comment

yma96 left a comment

Choose a reason for hiding this comment

tisutisu commented Jan 7, 2025 •

edited

Loading

tkdchen Jan 21, 2025 •

edited

Loading

tkdchen Jan 21, 2025 •

edited

Loading

chmeliik commented Feb 3, 2025 •

edited

Loading