Scan multiple Helm Charts at once

[yannispgs]

Question

Hi,

I am having some trouble adapting the Trivy CLI to easily scan all my Helm Charts in a single command. The repository I am scanning has the following structure :

• .helm/:
  • microservice1/:
    • templates/: all the Kubernetes manifests
    • values.yaml: values common to all environments
    • values-env.yaml: values specific to an environment
  • microservice2/: same as microservice1
  • ...
• services/: the source code for every microservice

I have only managed to scan each Helm Chart one-by-one because the helm-values option asks for relative path from Shell current directory. Thus, I need 1 Trivy command per Helm Chart.

Is there a way to tell Trivy to look for values.yaml and values-env.yaml files at the root folder of every Helm Chart ?

Otherwise, I can render the Helm templates one by one in an output dir and then Trivy will be able to scan them all at once, but I was wondering if there could be a direct workaround.

Target

Git Repository

Scanner

Misconfiguration

Output Format

Table

Mode

Standalone

Operating System

macOS Sonoma 14.5

Version

2024/08/27 11:06:26 INFO Loaded file_path=trivy.yaml
Version: 0.54.1
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-08-27 06:12:05.011405225 +0000 UTC
  NextUpdate: 2024-08-27 12:12:05.011404834 +0000 UTC
  DownloadedAt: 2024-08-27 08:09:34.000391 +0000 UTC
Check Bundle:
  Digest: sha256:ef2d9ad4fce0f933b20a662004d7e55bf200987c180e7f2cd531af631f408bb3
  DownloadedAt: 2024-08-27 08:41:07.606058 +0000 UTC

[nikpivkin]

@yannispgs Unfortunately, Trivy does not now support autodetection of values files.

[nikpivkin]

@simar7 What do you think about adding loading values files from charts like Helm does?

This object provides access to values passed into the chart. Its contents come from multiple sources:

• The values.yaml file in the chart
• If this is a subchart, the values.yaml file of a parent chart

[simar7]

My only gripe with this is we will end up reimplementing a lot of the priority logic that helm has when determining which values file will take precedence.

From the docs:

The list above is in order of specificity: values.yaml is the default, which can be overridden by a parent chart's values.yaml, which can in turn be overridden by a user-supplied values file, which can in turn be overridden by --set parameters.

Maybe Helm exposes this logic as a library that we could use?

[nikpivkin]

@simar7 I'll take a look.

[nikpivkin]

Helm itself resolves the priority. On our side it is necessary to add values.yaml from the chart to the filesystem of the scanner.

[simar7]

I've created #7459 to track implementing it.

[yannispgs]

And I thank you for that 😁

[nikpivkin]

@yannispgs Sorry if I was misleading, Trivy supports values.yaml, and above in the comments I meant values-env.yaml. What is that file? I looked at the documentation more closely and there is no mention of it in the documentation.

[yannispgs]

I should have used a placeholder: it is rather values-{env}.yaml. As many variables differ between test and production envs, we use a common values file (values.yaml), plus an environment-related values file (values-{env}.yaml).

The issue with the current version is that the trivy config . --helm-set-file <path> option is not recursive. If I want to check misconfigurations for my 2 microservices, in the Test AND Production environments, I would need to run :

trivy config .helm/microservice1 --helm-values .helm/microservice1/values.yaml --helm-values .helm/microservice1/values-test.yaml
trivy config .helm/microservice1 --helm-values .helm/microservice1/values.yaml --helm-values .helm/microservice1/values-prod.yaml

trivy config .helm/microservice2 --helm-values .helm/microservice2/values.yaml --helm-values .helm/microservice2/values-test.yaml
trivy config .helm/microservice2 --helm-values .helm/microservice2/values.yaml --helm-values .helm/microservice2/values-prod.yaml

The value provided to --helm-values is not related to the targeted scan path (here it is .helm/microserviceX) but relative to the CWD.

The most convenient way for me to run Trivy would be something like trivy config . --helm-values values.yaml --helm-values values-test.yaml followed by the same command for another environment. When detecting a Helm Chart, it would look for the values file whose paths were provided through --helm-values option.

But the more I talk about it, the more I understand the mess it would be to be implemented. The workaround I use to generate the Kubernetes manifest with helm template and then scan them all with Trivy is quite alright 😬

[nikpivkin]

With what arguments do you run helm template?

[yannispgs]

I run :

helm template .helm/${chart} -f .helm/${chart}/values.yaml -f .helm/${chart}/values-staging.yaml --output out/staging
helm template .helm/${chart} -f .helm/${chart}/values.yaml -f .helm/${chart}/values-production.yaml --output out/prod

[simar7]

@yannispgs thanks for your explanation. Initially I had misunderstood your request. What you are trying accomplish today goes against the design of the Trivy scanner which focusses on a single target at a time. Scanning multiple targets is better suited for a use case with further automation by the user. This is because of the complexities that can arise when scanning multiple targets, such as this case.

As you described above generating the manifest and then doing a helm template might be easier to do so. Let us know if you still need more help.