Webhook Timeout

[jarzuaga-va]

Hello.

We are running into some strange behavior when trying to validate signatures on deployment. We are signing our images with cosign using priv/pub key pairs and validating images using the pub key. Our images and signatures are stored in GitHub Packages Registry. We are seeing random timeouts if we have more than 8 or so images in a single deployment. Connaisseur times out and the helm deployment fails with this error:

helm.go:84: [debug] Internal error occurred: failed calling webhook "connaisseur-svc.connaisseur.svc": failed to call webhook: Post "https://connaisseur-svc.connaisseur.svc:443/mutate?timeout=30s": context deadline exceeded

Is there a limitation with how many images can be validated at one time? any other ideas on what would be causing this?

[xopham]

Hey @jarzuaga-va ,

there is no hard limit from Connaisseur-side on the number of images that can be validated. Kubernetes however requires that admission controllers (such as Connaisseur) have to pass within 30s. This puts a bit of stress on the validation speed. That being set, let's see.

There is however a few knobs you can tune to improve the situation:

• number of pod replicas
• available resources

I presume you have a single deployment.yaml that contains 8 or more images, right? In this case, the admission request will go to a single connaisseur pod and consequently your best chance may be to go for the available cpu and memory. We have tried to parallelize connaisseur as much as possible, but there may still be some limitations.

Also, could you share a (redacted) version of your Connaisseur helm/values.yaml and let me know which versions you are using?