Feature Request: Generate Secret if it does not exist

[victorgetz]

First of all thanks for your amazing work.

Description:

What i would propose is a feature which can automatically create the a secret if it does not exist.
With a identifier (for example generate@...) the webhook will now that it should take a look if the secret exists.
If it exists just take it. If not autogenerate a password/certificate.

What problem does it solve
Inside my helm chart i would like to be able to define everything for my service. At the moment we need somehow to create the secret upfront. In our case we create it upfront with terraform.

I need 2 steps to use my secret with two different technologies inside two different repositories.

Lets demonstrate it based on prometheus helm chart and admin credentials.

Example (Current):

Terraform

resource "random_password" "grafana_admin_pw" {
  length  = 32
  special = true
}

resource "vault_generic_secret" "grafana_admin_credentials" {
  path         = "kvEngine/prod/grafana"
  disable_read = false
  data_json = jsonencode({
    admin_password       = random_password.grafana_admin_pw.result
  })
}

Helm Chart

grafana:
  adminPassword: "vault:/kvEngine/prod/grafana#admin_password"

Example (Solution):

Helm Chart

grafana:
  adminPassword: "generate@vault:/kvEngine/prod/grafana#admin_password"

There is a topic about Write a value into Vault
but this one does not work with KV Engine and is really complicated.

[e-desouza]

There is a topic about Write a value into Vault
but this one does not work with KV Engine and is really complicated.

Through trial and error I found a way to write to kv (assuming role, auth and sa is set correctly):

envName : '>>vault:secret/data/test/app#TEST_ACCESS_KEY_ID#{"data":{"TEST_ACCESS_KEY_ID": "42"}}'

or

envName : '>>vault:secret/data/test/app##{"data":{"TEST_ACCESS_KEY_ID": "42"}}'

The issue is that the webhook will log an error in the pod with that env about the path not existing but it does indeed write to Vault as can be tested via cli with kv get to that path. I think fixing that incorrect error in the webhook is a good first step, though I would like an inbuilt generate if none exists feature too.

[ramizpolic]

Thank you for the submission @victorgetz! I can see how this could be a useful feature.

To enable customised secret generation, it would require two things:

• enable silencing not-found error in https://github.com/bank-vaults/vault-secrets-webhook when we are providing the default value ourselves. Thanks @e-desouza for raising this.
• extending the relevant components in injector from https://github.com/bank-vaults/internal when we are passing additional data (generate struct in the example)

This way, we would be able to preserve the same syntax and interface, whilst also allowing us to:

1. Provide default secret value myself

envName : '>>vault:secret/data/test/app#TEST_ACCESS_KEY_ID#{"data":{"TEST_ACCESS_KEY_ID": "42"}}'

In case this secret is not found in Vault, it would be created with TEST_ACCESS_KEY_ID=42.
This is supported, but needs to be verified and tested.

2. Provide the configuration to generate secret value

envName : '>>vault:secret/data/test/app#TEST_ACCESS_KEY_ID#{"generate":{"special": true, "length": 10}}'

In case this secret is not found in Vault, it would be created with TEST_ACCESS_KEY_ID=<SomeRandom10CharString>
Note that we have additional settings passed which can control how to generate the secret (e.g. generate a private key option). This needs to be implemented.

I am not sure when we will be able to add this feature to our backlog, but it is definitely on our radar. In the meantime, feel free to add more suggestions/concerns/contributions regarding this feature.

[e-desouza]

Hello @ramizpolic, is there any possibility of getting this feature in 1H'24?

[victorgetz]

For us option 2 would be amazing.
It would remove for our projects the dependency to terraform.
It would make our life easier in dozens of projects.

Can i help somehow @ramizpolic with funding or something like that?

[ramizpolic]

I am glad to see additional interest around this feature. We didn't have much time in December to cover this, but once I am back from vacation (next week), I will gather more details and check potential paths we can take here.
Personally, I am in favor of this feature, but we still need to figure out how to address the caveats around generating cryptographically secure data (do we want to rely on Vault or something else?), defining interfaces, ensuring reproducibility, etc.

@victorgetz Thanks a lot for your interest in supporting us ❤️! Although I don't have much info on the specifics yet, I'd be more than happy to explore options. Will have an update on this too in the upcoming weeks.

[ramizpolic]

Hello @ramizpolic, is there any possibility of getting this feature in 1H'24?

We will add templating options for generating secrets (alpha and alphanum for starters, but will make it easy to integrate new custom generators). Expect it by March. Will keep the details here. Thanks for the suggestions everyone!

[Tearix]

Hey @ramizpolic, this is a really cool feature we'd like to see. Is there any news or maybe a new assessment?

[ramizpolic]

Hi @Tearix, we are actively working on expanding Bank Vaults by making it more generic in terms of secret store provider support instead of only supporting Hashicorps Vault. We have been really busy on this so we paused most of other feature development. Once we finish up with these items, in a week or two, we will prioritise our feature commitments.

[4FunAndProfit]

Hello @ramizpolic, any news on this amazing feature? 😍😍
Would be GREAT GREAT GREAT to be able to generate in StartupSecret too. Maybe it is more faisable in a first version?? 🥹🥹