Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MemorySize validation for function build #6679

Closed
wants to merge 4 commits into from

Conversation

bentvelj
Copy link
Contributor

Validate function MemorySize to be an integer within range [128, 10240] as per docs.

Also had to fix a bunch of unit tests that previously had fake MemorySize.

Which issue(s) does this change fix?

#6510

Why is this change necessary?

Customers were permitted to build Lambda functions with invalid MemorySize, causing late/unexpected failure

How does it address the issue?

Adds a validation step to tell the customer the MemorySize is invalid (not an integer, or outside the range [128,10240])

What side effects does this change have?

N/A

Mandatory Checklist

PRs will only be reviewed after checklist is complete

  • Add input/output type hints to new functions/methods
  • Write design document if needed (Do I need to write a design document?)
  • Write/update unit tests
  • Write/update integration tests
  • Write/update functional tests if needed
  • make pr passes
  • make update-reproducible-reqs if dependencies were changed
  • Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@bentvelj bentvelj requested a review from a team as a code owner February 13, 2024 01:30
@bentvelj bentvelj requested review from hawflau and lucashuy February 13, 2024 01:30
@github-actions github-actions bot added area/build sam build command pr/internal labels Feb 13, 2024
Copy link
Contributor

@hawflau hawflau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding validation in build doesn't seem to me that it will resolve the issue customer described. sam build is not a prerequisite of sam local invoke (e.g. when the Lambda code has no external dependency at all or when customer has their own external build steps). sam local * seems to be the place where an incorrect config would fail invocation and thus needs validation.

I also wonder if we want to add our own validation logic. It's a duplication of knowledge as we already have sam validate --lint which would catch config errors. We can update sam local * to run sam validate --lint before starting to do the local stuff.

Comment on lines 293 to 307
MIN_MEMORY_SIZE = 128
MAX_MEMORY_SIZE = 10240

if function.memory:
try:
function_memory = int(function.memory)
except ValueError as ex:
raise InvalidFunctionDefinition(
f'Function {function.function_id} has invalid MemorySize "{function.memory}". Must be an integer between 128 and 10240.'
) from ex

if function_memory < MIN_MEMORY_SIZE or function_memory > MAX_MEMORY_SIZE:
raise InvalidFunctionDefinition(
f"Function {function.function_id} has MemorySize {function.memory}, but it must be an integer between 128 and 10240."
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we give a warning message rather than failing with the exception here? My concern is if this is changed and we missed it, then customers will be blocked building their lambda functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, however with Wing's comment we'll only be relying on cfnlint which we can expect to stay in sync with the CFN rules?

@mndeveci
Copy link
Contributor

Adding validation in build doesn't seem to me that it will resolve the issue customer described. sam build is not a prerequisite of sam local invoke (e.g. when the Lambda code has no external dependency at all or when customer has their own external build steps). sam local * seems to be the place where an incorrect config would fail invocation and thus needs validation.

In align with @hawflau's feedback here, I think this validation should be done at sam local command so that we won't spin up docker container with invalid memory configuration. Or we could catch this error and report it back to customer with a detailed message.

I wouldn't add a validation logic to sam build and let sam validate --lint to verify template is valid and deployable.

@bentvelj
Copy link
Contributor Author

bentvelj commented Feb 14, 2024

I've refactored to run the sam validate command in the __entry__ of InvokeContext, which is run before sam local invoke, start-api, start-lambda. The caveat to this is that we need command context passed into InvokeContext. This is an optional parameter, to avoid breaking all the isolated InvokeContext unit tests, and if the ctx is not provided, the sam validate step is skipped.

Integ tests to follow.

@@ -19,6 +19,13 @@
from ..invoke.layer_utils import LayerUtils


# class TestInvalidTempate(StartApiIntegBaseClass):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove the lines if they are not needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, didn't mean to commit this

@bentvelj
Copy link
Contributor Author

Closing in favour of new PR on separate branch

@bentvelj bentvelj closed this Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build sam build command pr/internal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants