Google Secret Manager as Source #82
Conversation
There was a problem hiding this comment.
So overall I think this may work, but I see many problems:
- performance issues in case of many secrets in your secret manager
- every 10 seconds you will request the secrets, which is not efficient and may cost a lot ($0.03 per 10,000 operations)
- not following the concept of "check if it has changed and then request".
For this use case, I feel that Cloud Storage: https://cloud.google.com/storage is a much better place for the Kafka ACL, will contain the API to see if a file has changed, and can still have security and encryption of the Kafka ACL if wanted.
You can keep your module as a community module in your own GitHub, but unfortunately I don't think GCP secrets manager is an efficient fit for KSM
| @@ -4,6 +4,9 @@ All notable changes to this project will be documented in this file. | |||
| The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) | |||
| and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). | |||
|
|
|||
| ## [0.10 - SNAPSHOT] | |||
| - Added Google Secret Manager as an ACL source | |||
There was a problem hiding this comment.
we're still in 0.9 - SNAPSHOT
|
Secret Manager is already part of the ecosystem here, the cost isn't really an issue (and I configured KSM to refresh each 2 minutes). I understand. I'll keep in other repo... |
|
Thanks for understanding @brunodomenici , sorry for not including it in KSM, but if you have it as a community module under your own repo, I will link to it in the README :) |
Hi,
As mentioned in #81 I needed to implement Google Secret Manager as Source.
I'm not a Scala developer so, I'm sorry if I did anything wrong 🙈
closes #81