feat: add best practices policies in CEL expressions (#925)

* copy restrict-node-port Signed-off-by: Chandan-DK <[email protected]> * convert restrict-node-port to cel Signed-off-by: Chandan-DK <[email protected]> * move resource files to test folders to avoid cross referencing Signed-off-by: Chandan-DK <[email protected]> * copy require-labels Signed-off-by: Chandan-DK <[email protected]> * convert require-labels to cel Signed-off-by: Chandan-DK <[email protected]> * copy restrict-service-external-ips Signed-off-by: Chandan-DK <[email protected]> * convert restrict-service-external-ips to cel Signed-off-by: Chandan-DK <[email protected]> * copy require-ro-rootfs Signed-off-by: Chandan-DK <[email protected]> * convert require-ro-rootfs to cel Signed-off-by: Chandan-DK <[email protected]> * copy restrict-image-registries Signed-off-by: Chandan-DK <[email protected]> * convert restrict-image-registries to cel Signed-off-by: Chandan-DK <[email protected]> * copy disallow-latest-tag Signed-off-by: Chandan-DK <[email protected]> * convert disallow-latest-tag to cel Signed-off-by: Chandan-DK <[email protected]> * copy disallow-default-namespace Signed-off-by: Chandan-DK <[email protected]> * convert disallow-default-namespace to cel Signed-off-by: Chandan-DK <[email protected]> * copy disallow-helm-tiller Signed-off-by: Chandan-DK <[email protected]> * convert disallow-helm-tiller to cel Signed-off-by: Chandan-DK <[email protected]> * copy disallow-empty-ingress-host Signed-off-by: Chandan-DK <[email protected]> * set original disallow-empty-ingress-host to Audit Signed-off-by: Chandan-DK <[email protected]> * convert disallow-empty-ingress-host to cel Signed-off-by: Chandan-DK <[email protected]> * patch cel policy to set it to Enforce in chainsaw test Signed-off-by: Chandan-DK <[email protected]> * fix: update semantically wrong chainsaw test resources in original require-drop-all policy Signed-off-by: Chandan-DK <[email protected]> * copy require-drop-all Signed-off-by: Chandan-DK <[email protected]> * convert require-drop-all to cel Signed-off-by: Chandan-DK <[email protected]> * update workflow to test policies in best-practices-cel folder Signed-off-by: Chandan-DK <[email protected]> * fix duplicate container names in require-probes chainsaw test Signed-off-by: Chandan-DK <[email protected]> * copy require-probes Signed-off-by: Chandan-DK <[email protected]> * convert require-probes to cel Signed-off-by: Chandan-DK <[email protected]> * require-ro-rootfs: fix selector does not match template labels Signed-off-by: Chandan-DK <[email protected]> * require-ro-rootfs: fix duplicate container names Signed-off-by: Chandan-DK <[email protected]> * disallow-helm-tiller: fix invalid container naming Signed-off-by: Chandan-DK <[email protected]> * require-labels: fix selector does not match template labels Signed-off-by: Chandan-DK <[email protected]> * restrict-image-registries: fix selector does not match template labels Signed-off-by: Chandan-DK <[email protected]> * rename file for clarity Signed-off-by: Chandan-DK <[email protected]> * copy disallow-cri-sock-mount Signed-off-by: Chandan-DK <[email protected]> * convert disallow-cri-sock-mount to cel Signed-off-by: Chandan-DK <[email protected]> * remove duplicate expressins in require-drop-all Signed-off-by: Chandan-DK <[email protected]> * rename file for clarity Signed-off-by: Chandan-DK <[email protected]> * require-drop-cap-net-raw: fix duplicate container names Signed-off-by: Chandan-DK <[email protected]> * copy require-drop-cap-net-raw Signed-off-by: Chandan-DK <[email protected]> * rename pods to distinguish them Signed-off-by: Chandan-DK <[email protected]> * convert require-drop-cap-net-raw to cel Signed-off-by: Chandan-DK <[email protected]> * copy require-pod-requests-limits Signed-off-by: Chandan-DK <[email protected]> * convert require-pod-requests-limits to cel Signed-off-by: Chandan-DK <[email protected]> * rename files for clarity Signed-off-by: Chandan-DK <[email protected]> * add new line at end of file where not present Signed-off-by: Chandan-DK <[email protected]> * calculate digests Signed-off-by: Chandan-DK <[email protected]> * add new lines Signed-off-by: Chandan-DK <[email protected]> * update digests Signed-off-by: Chandan-DK <[email protected]> * remove celPreconditions until it behaves as expected Related to issue kyverno/kyverno#9884 Signed-off-by: Chandan-DK <[email protected]> * update digests Signed-off-by: Chandan-DK <[email protected]> * remove wrong test step The update to goodpod01 fails not due to Kyverno blocking it, but rather because Kubernetes doesn't permit such modifications on pods. Signed-off-by: Chandan-DK <[email protected]> * use variables to remove duplicate logic Signed-off-by: Chandan-DK <[email protected]> * remove unnecessary whitespace in require-ro-rootfs Signed-off-by: Chandan-DK <[email protected]> * use namespaceObject variable Signed-off-by: Chandan-DK <[email protected]> * Combine expressions into 1 rule to generate VAPs Signed-off-by: Chandan-DK <[email protected]> * copy kyverno tests for disallow-default-namespace Signed-off-by: Chandan-DK <[email protected]> * fix issue caused in cel policies tests due to chainsaw templating Signed-off-by: Chandan-DK <[email protected]> --------- Signed-off-by: Chandan-DK <[email protected]> Co-authored-by: Chip Zoller <[email protected]> Co-authored-by: Mariam Fahmy <[email protected]> Co-authored-by: Jim Bugwadia <[email protected]>
kyverno · Jun 3, 2024 · 7908f02 · 7908f02
1 parent a556d71
commit 7908f02
Show file tree

Hide file tree

Showing 197 changed files with 5,354 additions and 46 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -32,6 +32,7 @@ jobs:
           - ^argo$
           - ^aws$
           - ^best-practices$
+          - ^best-practices-cel$
           - ^castai$
           - ^cert-manager$
           - ^consul$

diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/chainsaw-test.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/chainsaw-test.yaml
@@ -0,0 +1,51 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  creationTimestamp: null
+  name: disallow-cri-sock-mount
+spec:
+  steps:
+  - name: step-01
+    try:
+    - apply:
+        file: ../disallow-cri-sock-mount.yaml
+    - patch:
+        resource:
+          apiVersion: kyverno.io/v1
+          kind: ClusterPolicy
+          metadata:
+            name: disallow-container-sock-mounts
+          spec:
+            validationFailureAction: Enforce
+    - assert:
+        file: policy-ready.yaml
+  - name: step-02
+    try:
+    - apply:
+        file: good-pod.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: pod-containerd-sock.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: pod-docker-sock.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: pod-crio-sock.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: pod-cri-dockerd-sock.yaml
+    - apply:
+        file: pod-emptydir-vol.yaml
+    - apply:
+        file: pod-no-volumes.yaml
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/good-pod.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/good-pod.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: goodpod01
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: data
+    hostPath:
+      path: /data
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-containerd-sock.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-containerd-sock.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-containerd-sock-mount
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: dockersock
+    hostPath:
+      path: /var/run/containerd/containerd.sock
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-cri-dockerd-sock.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-cri-dockerd-sock.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-cri-dockerd-sock-mount
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: dockersock
+    hostPath:
+      path: /var/run/cri-dockerd.sock
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-crio-sock.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-crio-sock.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-crio-sock-mount
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: dockersock
+    hostPath:
+      path: /var/run/crio/crio.sock
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-docker-sock.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-docker-sock.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-docker-sock-mount
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: dockersock
+    hostPath:
+      path: /var/run/docker.sock
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-emptydir-vol.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-emptydir-vol.yaml
@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-emptydir-volume
+spec:
+  containers:
+  - name: busybox
+    image: busybox:1.35
+    command:
+    - sleep
+    - "3600"
+  volumes:
+    - name: mydir
+      emptyDir: {}
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-no-volumes.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/pod-no-volumes.yaml
@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-no-volumes
+spec:
+  automountServiceAccountToken: false
+  containers:
+  - name: busybox
+    image: busybox:1.35
+    command:
+    - sleep
+    - "3600"
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/policy-ready.yaml b/best-practices-cel/disallow-cri-sock-mount/.chainsaw-test/policy-ready.yaml
@@ -0,0 +1,7 @@
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: disallow-container-sock-mounts
+status:
+  ready: true
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.kyverno-test/kyverno-test.yaml b/best-practices-cel/disallow-cri-sock-mount/.kyverno-test/kyverno-test.yaml
@@ -0,0 +1,22 @@
+apiVersion: cli.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  name: disallow-cri-sock-mount
+policies:
+- ../disallow-cri-sock-mount.yaml
+resources:
+- resource.yaml
+results:
+- kind: Pod
+  policy: disallow-container-sock-mounts
+  resources:
+  - pod-with-docker-sock-mount
+  result: fail
+  rule: validate-socket-mounts
+- kind: Pod
+  policy: disallow-container-sock-mounts
+  resources:
+  - goodpod01
+  result: pass
+  rule: validate-socket-mounts
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/.kyverno-test/resource.yaml b/best-practices-cel/disallow-cri-sock-mount/.kyverno-test/resource.yaml
@@ -0,0 +1,32 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod-with-docker-sock-mount
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: dockersock
+    hostPath:
+      path: /var/run/docker.sock
+---
+apiVersion: v1
+kind: Pod
+metadata:
+  name: goodpod01
+spec:
+  containers:
+  - name: myshell
+    image: "ubuntu:18.04"
+    command:
+    - /bin/sleep
+    - "300"
+  volumes:
+  - name: data
+    hostPath:
+      path: /data
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/artifacthub-pkg.yml b/best-practices-cel/disallow-cri-sock-mount/artifacthub-pkg.yml
@@ -0,0 +1,25 @@
+name: disallow-cri-sock-mount-cel
+version: 1.0.0
+displayName: Disallow CRI socket mounts in CEL expressions
+description: >-
+  Container daemon socket bind mounts allows access to the container engine on the node. This access can be used for privilege escalation and to manage containers outside of Kubernetes, and hence should not be allowed. This policy validates that the sockets used for CRI engines Docker, Containerd, and CRI-O are not used. In addition to or replacement of this policy, preventing users from mounting the parent directories (/var/run and /var) may be necessary to completely prevent socket bind mounts.
+install: |-
+  ```shell
+  kubectl apply -f https://raw.githubusercontent.com/kyverno/policies/main/best-practices-cel/disallow-cri-sock-mount/disallow-cri-sock-mount.yaml
+  ```
+keywords:
+  - kyverno
+  - Best Practices
+  - EKS Best Practices
+  - CEL Expressions
+readme: |
+  Container daemon socket bind mounts allows access to the container engine on the node. This access can be used for privilege escalation and to manage containers outside of Kubernetes, and hence should not be allowed. This policy validates that the sockets used for CRI engines Docker, Containerd, and CRI-O are not used. In addition to or replacement of this policy, preventing users from mounting the parent directories (/var/run and /var) may be necessary to completely prevent socket bind mounts.
+  
+  Refer to the documentation for more details on Kyverno annotations: https://artifacthub.io/docs/topics/annotations/kyverno/
+annotations:
+  kyverno/category: "Best Practices, EKS Best Practices in CEL"
+  kyverno/kubernetesVersion: "1.26-1.27"
+  kyverno/subject: "Pod"
+digest: 0b91de77f8a6da0cafea457e0ba9eb14f0b8eb6bbcb56419a4e9de09c860753d
+createdAt: "2024-03-14T15:59:52Z"
+
diff --git a/best-practices-cel/disallow-cri-sock-mount/disallow-cri-sock-mount.yaml b/best-practices-cel/disallow-cri-sock-mount/disallow-cri-sock-mount.yaml
@@ -0,0 +1,58 @@
+apiVersion: kyverno.io/v1
+kind: ClusterPolicy
+metadata:
+  name: disallow-container-sock-mounts
+  annotations:
+    policies.kyverno.io/title: Disallow CRI socket mounts in CEL expressions
+    policies.kyverno.io/category: Best Practices, EKS Best Practices in CEL 
+    policies.kyverno.io/severity: medium
+    policies.kyverno.io/subject: Pod
+    policies.kyverno.io/minversion: 1.11.0
+    kyverno.io/kubernetes-version: "1.26-1.27"
+    policies.kyverno.io/description: >-
+      Container daemon socket bind mounts allows access to the container engine on the
+      node. This access can be used for privilege escalation and to manage containers
+      outside of Kubernetes, and hence should not be allowed. This policy validates that
+      the sockets used for CRI engines Docker, Containerd, and CRI-O are not used. In addition
+      to or replacement of this policy, preventing users from mounting the parent directories
+      (/var/run and /var) may be necessary to completely prevent socket bind mounts.
+spec:
+  validationFailureAction: Audit
+  background: true
+  rules:
+  - name: validate-socket-mounts
+    match:
+      any:
+      - resources:
+          kinds:
+          - Pod
+    validate:
+      cel:
+        variables:
+          - name: hasVolumes
+            expression: "!has(object.spec.volumes)"
+          - name: volumes
+            expression: "object.spec.volumes"
+          - name: volumesWithHostPath
+            expression: "variables.volumes.filter(volume, has(volume.hostPath))"
+        expressions:
+          - expression: >-
+              variables.hasVolumes || 
+              variables.volumesWithHostPath.all(volume, !volume.hostPath.path.matches('/var/run/docker.sock'))
+            message: "Use of the Docker Unix socket is not allowed."
+
+          - expression: >-
+              variables.hasVolumes || 
+              variables.volumesWithHostPath.all(volume, !volume.hostPath.path.matches('/var/run/containerd/containerd.sock'))
+            message: "Use of the Containerd Unix socket is not allowed."
+          
+          - expression: >-
+              variables.hasVolumes || 
+              variables.volumesWithHostPath.all(volume, !volume.hostPath.path.matches('/var/run/crio/crio.sock'))
+            message: "Use of the CRI-O Unix socket is not allowed."
+          
+          - expression: >-
+              variables.hasVolumes || 
+              variables.volumesWithHostPath.all(volume, !volume.hostPath.path.matches('/var/run/cri-dockerd.sock'))
+            message: "Use of the Docker CRI socket is not allowed."
+
diff --git a/best-practices-cel/disallow-default-namespace/.chainsaw-test/chainsaw-test.yaml b/best-practices-cel/disallow-default-namespace/.chainsaw-test/chainsaw-test.yaml
@@ -0,0 +1,56 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/kyverno/chainsaw/main/.schemas/json/test-chainsaw-v1alpha1.json
+apiVersion: chainsaw.kyverno.io/v1alpha1
+kind: Test
+metadata:
+  creationTimestamp: null
+  name: disallow-default-namespace
+spec:
+  steps:
+  - name: step-01
+    try:
+    - apply:
+        file: ../disallow-default-namespace.yaml
+    - patch:
+        resource:
+          apiVersion: kyverno.io/v1
+          kind: ClusterPolicy
+          metadata:
+            name: disallow-default-namespace
+          spec:
+            validationFailureAction: Enforce
+    - assert:
+        file: policy-ready.yaml
+  - name: step-02
+    try:
+    - apply:
+        file: ns.yaml
+  - name: step-03
+    try:
+    - apply:
+        file: good-resources.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: pod-default.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: ds-default.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: job-default.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: ss-default.yaml
+    - apply:
+        expect:
+        - check:
+            ($error != null): true
+        file: deploy-default.yaml
+