diff --git a/README.md b/README.md index 10b5a18488b..9d0140dc049 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ ![Build](https://github.com/awslabs/aws-lambda-powertools/workflows/Powertools%20Python/badge.svg?branch=master) ![PythonSupport](https://img.shields.io/static/v1?label=python&message=3.6%20|%203.7|%203.8&color=blue?style=flat-square&logo=python) ![PyPI version](https://badge.fury.io/py/aws-lambda-powertools.svg) ![PyPi monthly downloads](https://img.shields.io/pypi/dm/aws-lambda-powertools) -A suite of utilities for AWS Lambda functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier. +A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. **[📜Documentation](https://awslabs.github.io/aws-lambda-powertools-python/)** | **[API Docs](https://awslabs.github.io/aws-lambda-powertools-python/api/)** | **[🐍PyPi](https://pypi.org/project/aws-lambda-powertools/)** | **[Feature request](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=)** | **[🐛Bug Report](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)** | **[Kitchen sink example](https://github.com/awslabs/aws-lambda-powertools-python/tree/develop/example)** | **[Detailed blog post](https://aws.amazon.com/blogs/opensource/simplifying-serverless-best-practices-with-lambda-powertools/)** diff --git a/docs/content/index.mdx b/docs/content/index.mdx index ec2dd862e38..0c46af2958a 100644 --- a/docs/content/index.mdx +++ b/docs/content/index.mdx @@ -5,7 +5,7 @@ description: AWS Lambda Powertools Python import Note from "../src/components/Note" -Powertools is a suite of utilities for AWS Lambda functions that makes tracing with AWS X-Ray, structured logging and creating custom metrics asynchronously easier. +A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, and more. Looking for a quick run through of the core utilities?

@@ -24,12 +24,6 @@ Powertools is available in PyPi. You can use your favourite dependency managemen ```bash:title=hello_world.sh sam init --location https://github.com/aws-samples/cookiecutter-aws-sam-python ``` -* [Tracing](./core/tracer) - Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions -* [Logging](./core/logger) - Structured logging made easier, and decorator to enrich structured logging with key Lambda context details -* [Metrics](./core/metrics) - Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) -* [Bring your own middleware](./utilities/middleware_factory) - Decorator factory to create your own middleware to run logic before, and after each Lambda invocation -* [Parameters utility](./utilities/parameters) - Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time -* [Batch utility](./utilities/batch) - Batch processing for AWS SQS, handles partial failure. ### Lambda Layer @@ -59,9 +53,10 @@ This will add a nested app stack with an output parameter `LayerVersionArn`, tha You can fetch the available versions via the API with: - ```bash - aws serverlessrepo list-application-versions --application-id arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer - ``` +```bash +aws serverlessrepo list-application-versions \ + --application-id arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer +``` ## Features @@ -71,8 +66,12 @@ Utility | Description [Logging](./core/logger) | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details [Metrics](./core/metrics) | Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) [Bring your own middleware](.//utilities/middleware_factory) | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation -[Parameters utility](./utilities/parameters) | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time -[Typing utility](./utilities/typing) | Static typing classes to speedup development in your IDE +[Parameters](./utilities/parameters) | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time +[Typing](./utilities/typing) | Static typing classes to speedup development in your IDE +[Batch](./utilities/batch) | Handle partial failures for AWS SQS batch processing +[Validation](./utilities/validation) | JSON Schema validator for inbound events and responses +[Event source data classes](./utilities/data_classes) | Data classes describing the schema of common Lambda event triggers + ## Environment variables diff --git a/docs/content/media/utilities_data_classes.png b/docs/content/media/utilities_data_classes.png new file mode 100644 index 00000000000..94ed83bde97 Binary files /dev/null and b/docs/content/media/utilities_data_classes.png differ diff --git a/docs/content/utilities/data_classes.mdx b/docs/content/utilities/data_classes.mdx new file mode 100644 index 00000000000..22c6f4ec856 --- /dev/null +++ b/docs/content/utilities/data_classes.mdx @@ -0,0 +1,240 @@ +--- +title: Event Source Data Classes +description: Utility +--- + +import Note from "../../src/components/Note" + +The event source data classes utility provides classes describing the schema of common Lambda events triggers. + +**Key Features** + +* Type hinting and code completion for common event types +* Helper functions for decoding/deserializing nested fields +* Docstrings for fields contained in event schemas + +**Background** + +When authoring Lambda functions, you often need to understand the schema of the event dictionary which is passed to the +handler. There are several common event types which follow a specific schema, depending on the service triggering the +Lambda function. + + +## Utilizing the data classes + +The classes are initialized by passing in the Lambda event object into the constructor of the appropriate data class. +For example, if your Lambda function is being triggered by an API Gateway proxy integration, you can use the +`APIGatewayProxyEvent` class. + +![Utilities Data Classes](../media/utilities_data_classes.png) + + +## Supported event sources + +Event Source | Data_class +------------------------------------------------- | --------------------------------------------------------------------------------- +[API Gateway Proxy](#api-gateway-proxy) | `APIGatewayProxyEvent` +[API Gateway Proxy event v2](#api-gateway-proxy-v2) | `APIGatewayProxyEventV2` +[CloudWatch Logs](#cloudWatch-logs) | `CloudWatchLogsEvent` +[Cognito User Pool](#cognito-user-pool-triggers) | Multiple available under `cognito_user_pool_event` +[DynamoDB streams](#dynamoDB-streams) | `DynamoDBStreamEvent`, `DynamoDBRecordEventName` +[EventBridge](#eventbridge) | `EventBridgeEvent` +[Kinesis Data Stream](#kinesis-streams) | `KinesisStreamEvent` +[S3](#S3) | `S3Event` +[SES](#SES) | `SESEvent` +[SNS](#SNS) | `SNSEvent` +[SQS](#SQS) | `SQSEvent` + + + + The examples provided below are far from exhaustive - the data classes themselves are designed to provide a form of + documentation inherently (via autocompletion, types and docstrings). + + + +## API Gateway Proxy + +Typically used for API Gateway REST API or HTTP API using v1 proxy event. + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEvent + +def lambda_handler(event, context): + event: APIGatewayProxyEvent = APIGatewayProxyEvent(event) + request_context = event.request_context + identity = request_context.identity + + if 'helloworld' in event.path && event.http_method == 'GET': + user = identity.user + do_something_with(event.body, user) +``` + +## API Gateway Proxy v2 + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEventV2 + +def lambda_handler(event, context): + event: APIGatewayProxyEventV2 = APIGatewayProxyEventV2(event) + request_context = event.request_context + query_string_parameters = event.query_string_parameters + + if 'helloworld' in event.raw_path && request_context.http.method == 'POST': + do_something_with(event.body, query_string_parameters) +``` + +## CloudWatch Logs + +CloudWatch Logs events by default are compressed and base64 encoded. You can use the helper function provided to decode, +decompress and parse json data from the event. + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import CloudWatchLogsEvent + +def lambda_handler(event, context): + event: CloudWatchLogsEvent = CloudWatchLogsEvent(event) + + decompressed_log = event.parse_logs_data + log_events = decompressed_log.log_events + for event in log_events: + do_something_with(event.timestamp, event.message) +``` + +## Cognito User Pool + +Cognito User Pools have several [different Lambda trigger sources](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-identity-pools-working-with-aws-lambda-trigger-sources), all of which map to a different data class, which +can be imported from `aws_lambda_powertools.data_classes.cognito_user_pool_event`: + +Trigger/Event Source | Data Class +------------------------------------------------- | ------------------------------------------------- +Custom message event | `data_classes.cognito_user_pool_event.CustomMessageTriggerEvent` +Post authentication | `data_classes.cognito_user_pool_event.PostAuthenticationTriggerEvent` +Post confirmation | `data_classes.cognito_user_pool_event.PostConfirmationTriggerEvent` +Pre authentication | `data_classes.cognito_user_pool_event.PreAuthenticationTriggerEvent` +Pre sign-up | `data_classes.cognito_user_pool_event.PreSignUpTriggerEvent` +Pre token generation | `data_classes.cognito_user_pool_event.PreTokenGenerationTriggerEvent` +User migration | `data_classes.cognito_user_pool_event.UserMigrationTriggerEvent` + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.cognito_user_pool_event import PostConfirmationTriggerEvent + +def lambda_handler(event, context): + event: PostConfirmationTriggerEvent = PostConfirmationTriggerEvent(event) + + user_attributes = user_attributes = event.request.user_attributes + do_something_with(user_attributes) +``` + +## DynamoDB Streams + +The DynamoDB data class utility provides the base class for `DynamoDBStreamEvent`, a typed class for +attributes values (`AttributeValue`), as well as enums for stream view type (`StreamViewType`) and event type +(`DynamoDBRecordEventName`). + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import DynamoDBStreamEvent, DynamoDBRecordEventName + +def lambda_handler(event, context): + event: DynamoDBStreamEvent = DynamoDBStreamEvent(event) + + # Multiple records can be delivered in a single event + for record in event.records: + if record.event_name == DynamoDBRecordEventName.MODIFY: + do_something_with(record.dynamodb.new_image) + do_something_with(record.dynamodb.old_image) +``` + +## EventBridge + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import EventBridgeEvent + +def lambda_handler(event, context): + event: EventBridgeEvent = EventBridgeEvent(event) + do_something_with(event.detail) + +``` + +## Kinesis streams + +Kinesis events by default contain base64 encoded data. You can use the helper function to access the data either as json +or plain text, depending on the original payload. + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import KinesisStreamEvent + +def lambda_handler(event, context): + event: KinesisStreamEvent = KinesisStreamEvent(event) + + # if data was delivered as json + data = event.data_as_text() + + # if data was delivered as text + data = event.data_as_json() + + do_something_with(data) + +``` + +## S3 + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import S3Event + +def lambda_handler(event, context): + event: S3Event = S3Event(event) + bucket_name = event.bucket_name + + # Multiple records can be delivered in a single event + for record in event.records: + object_key = record.s3.get_object.key + + do_something_with(f'{bucket_name}/{object_key}') + +``` + +## SES + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import SESEvent + +def lambda_handler(event, context): + event: SESEvent = SESEvent(event) + + # Multiple records can be delivered in a single event + for record in event.records: + mail = record.ses.mail + common_headers = list(mail.common_headers) + + do_something_with(common_headers.to, common_headers.subject) + +``` + +## SNS + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import SNSEvent + +def lambda_handler(event, context): + event: SNSEvent = SNSEvent(event) + + # Multiple records can be delivered in a single event + for record in event.records: + message = record.sns.message + subject = record.sns.subject + + do_something_with(subject, message) +``` + +## SQS + +```python:title=lambda_app.py +from aws_lambda_powertools.utilities.data_classes import SQSEvent + +def lambda_handler(event, context): + event: SQSEvent = SQSEvent(event) + + # Multiple records can be delivered in a single event + for record in event.records: + do_something_with(record.body) +``` diff --git a/docs/gatsby-config.js b/docs/gatsby-config.js index 171499c3a22..087f23a9634 100644 --- a/docs/gatsby-config.js +++ b/docs/gatsby-config.js @@ -34,7 +34,8 @@ module.exports = { 'utilities/parameters', 'utilities/batch', 'utilities/typing', - 'utilities/validation' + 'utilities/validation', + 'utilities/data_classes' ], }, navConfig: {