Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add kyverno to staging #5203

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@ resources:
- konflux-rbac
- konflux-info
- vector-tekton-logs-collector
- kyverno
components:
- ../../../k-components/inject-infra-deployments-repo-details
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- kyverno.yaml
47 changes: 47 additions & 0 deletions argo-cd-apps/base/member/infra-deployments/kyverno/kyverno.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some Notes on using ArgoCD. If I'm getting it correctly, we should either enable Replace in the syncOptions, and probably ignore diff in aggregated cluster roles or use server side apply.

Also, to avoid ArgoCD's Dasbhoard to be polluted with Kyverno's reports, we should tell ArgoCD to ignore reports globally by adding them under the resource.exclusions stanza in the ArgoCD ConfigMap.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ed5b523.

Do you know where the configmap is controlled or if it can be overridden by infra-deployments? I see an owner reference to argocd on the live versions of it (probably from the helm chart behind openshift-gitops), but I don't find a reference to it in this repository.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about it. @hugares @manish-jangra @bamachrn can you help here?

Copy link
Contributor

@amisstea amisstea Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@filariow @sadlerap the configs are stored on the internal gitlab and they utilize the ArgoCD custom resource. See:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not have time to look into this but the replace sync option is at the ArgoCD applications level so in infra-deployments. For the other "resource.exclusions", is it possible to also configure this at application level rather that globally?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed https://gitlab.cee.redhat.com/service/app-interface/-/merge_requests/129507 for staging, will file a separate MR for prod. Local testing by both @filariow and myself seemed to indicate that this will be safe to add, even to clusters that won't have kyverno installed.

I didn't come across a way to do resource exclusions on a per-application basis.

Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: kyverno
labels:
noSourceTransform: "true"
spec:
generators:
- merge:
mergeKeys:
- nameNormalized
generators:
- clusters:
values:
sourceRoot: components/kyverno
environment: staging
clusterDir: ""
- list:
elements: []
template:
metadata:
name: kyverno-{{nameNormalized}}
spec:
project: default
sources:
- chart: kyverno
repoURL: https://kyverno.github.io/kyverno/
targetRevision: 3.3.4
helm:
valueFiles:
- $values/{{values.sourceRoot}}/{{values.environment}}/values.yaml
destination:
namespace: konflux-kyverno
server: '{{server}}'
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- Replace=true # Recommended by upstream
retry:
limit: 50
backoff:
duration: 10s
factor: 2
maxDuration: 3m
6 changes: 6 additions & 0 deletions argo-cd-apps/overlays/development/delete-applications.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,9 @@ kind: ApplicationSet
metadata:
name: konflux-info
$patch: delete
---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: kyverno
$patch: delete
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,9 @@ kind: ApplicationSet
metadata:
name: nvme-storage-configurator
$patch: delete
---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: kyverno
$patch: delete
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,9 @@ $patch: delete
# metadata:
# name: nvme-storage-configurator
# $patch: delete
---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: kyverno
$patch: delete
8 changes: 8 additions & 0 deletions components/kyverno/staging/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
admissionController:
replicas: 3
backgroundController:
replicas: 3
cleanupController:
replicas: 3
reportsController:
replicas: 3
Loading