Support for AWS Parameters and Secrets Lambda Extension?

[ericbn]

Are there plans to support the recently announced AWS Parameters and Secrets Lambda Extension as an additional layer, and then have the Lambda Powertools Parameters utility integrate with it?

Or is this redundant, meaning all features in the mentioned Lambda Extension are already implemented in Lambda Powertools? Its not clear to me if the Extension is using a longer lived type of cache than the Powertools implementation.

[heitorlessa]

We haven't forgotten your question - we've just launched V2, and will provide a detailed answer to you later today

[heitorlessa]

hey Eric, thank you for your patience ;)

For your first question, would you mind raising a feature request so we can gauge interest for this?

As to your second question, at the time of writing, it's redundant. Powertools offers those features since 2020 (caching, invalidation, etc.). Here's what the Extension differs though:

• Language agnostic. As an extension, it can run for any managed runtime except Container Images (OCI).
  • Parameters feature is available in Python, Java, and being implemented in .NET as we speak.
• SDK agnostic. These extensions launch a HTTP server, use the SDK to fetch those parameters, and cache them in-memory.
  • Parameters feature is a tiny wrapper on top of AWS SDK as we optimize for the Lambda runtime (e.g., caching, cold start, built-in authorization, etc.)
• Operations. As an extension requires a Lambda Layer, you need to know which version you must update it to. You also need to manually look into the documentation for updates (if any) and then update your Extension in case of bug fixing, new features, etc.
  • You can install Powertools via PyPi (no layers, no extension), via Lambda Layer (similar issue here), or SAR (semantic version with Layer optimizations). There is no additional overhead as we don't need to initiate a HTTP server but use the SDK, since we optimize for even the smallest memory size setting - it's our tenet.

There are some other differences like testing or running your function locally (extensions aren't supported except SAM CLI), but this will prolong the discussion (happy to dig otherwise).

As you already use Powertools, this is redundant. You also benefit from (missing from the extension):

• (1) cache invalidation per param/secret/config as opposed to globally
• (2) auto-deserialization with the transform parameter
• (3) opt-out of caching for a sensitive param/secret/config
• (4) multiple providers and bring your own provider like Redis, HashiCorp Vault/Consul, etc. (something we want to provide support too)
• (5) simplified operations even if you use our Lambda Layer (with its perf/size optimizations), since you can easily watch for updates in this GitHub repo (docs follow semantic version and so its installation options)

When would we recommend the extension?

In our experience, the extension is great for customers looking for (1) simple parameter/secret retrieval, (2) retrieving several parameters in the background without creating threads/AsyncIO themselves, and (3) have restrictions in what dependencies they can bring (some can't bring any).

Where do we go from here?

We'd be happy to reconsider adding support for Extensions as a provider in Parameters if there is customer demand. Until then, we'd prefer investing in additional parameter providers, and ton of other improvements we plan in the next few months to continue with "batteries included" motion.

[ericbn]

Oi @heitorlessa! Thanks a lot for the detailed answer. I'm using the Powertools already and super happy with it, even more after understanding more about its benefits as you described.

I don't feel like requesting the Extension support as a feature after learning the details from your answer, also so you can keep the focus on the improvements already planned. Thanks and keep up the great work!

[heitorlessa]

Awesome to hear. Ping us anytime!

[heitorlessa]

Additional data point as this feature got even more powerful a few minutes ago

You can now fetch multiple parameters from distinct paths, and tweak per parameter caching, data transformation, and decryption.... or globally!

We'll choose the most optimal way to fetch parameters based on those settings.

https://github.com/awslabs/aws-lambda-powertools-python/releases/tag/v2.2.0