Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A vulnerabilityreport scan job cannot be started due to volume name duplicates in GKE #2407

Open
Effaceurs opened this issue Jan 30, 2025 · 0 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@Effaceurs
Copy link

What steps did you take and what happened:
Hello everyone, I have faced the following issue:

Trivy operator is unable to start a vulnerabilityreport scan job in GKE for pods with more than 1 container, due to duplicate volume name gcrvol. However, it works fine with only one container in a pod.

ReplicaSet":{"name":"redacted","namespace":"redacted"},"namespace":"redacted","name":"redacted","reconcileID":"419f6ca1-6242-4661-acf3-17827f37a045","error":"creating scan job failed: trivy-system/scan-vulnerabilityreport-6647b9b9bf: Job.batch "scan-vulnerabilityreport-6647b9b9bf" is invalid: [spec.template.spec.volumes[3].name: Duplicate value: "gcrvol", spec.template.spec.volumes[4].name: Duplicate value: "gcrvol", spec.template.spec.volumes[5].name: Duplicate value: "gcrvol", spec.template.spec.containers[1].volumeMounts[3].mountPath: Invalid value: "/cred": must be unique, spec.template.spec.containers[2].volumeMounts[3].mountPath: Invalid value: "/cred": must be unique, spec.template.spec.containers[2].volumeMounts[4].mountPath: Invalid value: "/cred": must be unique, spec.template.spec.containers[3].volumeMounts[3].mountPath: Invalid value: "/cred": must be unique, spec.template.spec.containers[3].volumeMounts[4].mountPath: Invalid value: "/cred": must be unique, spec.template.spec.containers[3].volumeMounts[5].mountPath: Invalid value: "/cred": must be unique]","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:263\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:224"}

What did you expect to happen:

In GKE cluster, a scan job successfully started for a pod with more then 1 container.

Anything else you would like to add:

A pod that could not be scanned contains 3 containers, one container image is fetched from docker hub and another 2 from are from private GCR. To fetch GCR there is imagePullSecrets secret mentioned in pod definition for these containers, to fetch an image from docker hub we do not mention anything in imagePullSecrets.

However, in Trivy Operator settings we specified privateRegistryScanSecretsNames for images taken from docker hub, in order not to breach docker hub rate-limits.

privateRegistryScanSecretsNames: {"redacted": "dockerconfigjson-dockerhub-com"}

I tried to set the following parameter

scanJob.useGCRServiceAccount: 'false'

After which the scan job started, but I am getting another error

Registry flag error: the number of usernames and passwords must match.

To address this, I tried to add gcr secret key to this variable

privateRegistryScanSecretsNames: {"redacted": "dockerconfigjson-dockerhub-com,gcr-key"}

But it did not help, and the error above persists.

Helm chart values are below


apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: trivy-operator
  namespace: flux-system
spec:
  chart:
    spec:
      version: 0.25.0
  values:
    nodeCollector:
      useNodeSelector: false
      excludeNodes: kubernetes.io/arch=amd64

    targetNamespaces: "redacted"

    operator:
      privateRegistryScanSecretsNames: {"redacted": "dockerconfigjson-dockerhub-com"}
      scanJobTimeout: 10m
      builtInTrivyServer: true
      scanJobsConcurrentLimit: 3

    trivy:
      mode: ClientServer
      severity: HIGH,CRITICAL

      server:
        resources:
          limits:
            memory: 2Gi

Environment:

  • Trivy-Operator version: 0.23.0
  • Kubernetes version: v1.30.5-gke.1713000
  • OS: Container-Optimized OS 113
@Effaceurs Effaceurs added the kind/bug Categorizes issue or PR as related to a bug. label Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

1 participant