-
Notifications
You must be signed in to change notification settings - Fork 59
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add sample for lambda event filtering with dynamodb and sqs (#149)
* add sample for lambda event filtering with dynamodb and sqs * add more context in docs
- Loading branch information
1 parent
4b5ab00
commit 523d21f
Showing
4 changed files
with
191 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
# AWS Lambda event filtering with DynamoDB and SQS | ||
|
||
Simple demo application illustrating AWS Lambda event source filtering with DynamoDB and SQS. For this demo, we will use AWS Serverless Application Model (SAM), and a thin LocalStack wrapper `samlocal` to create our infrastructure through SAM on LocalStack. | ||
|
||
## Prerequisites | ||
|
||
* LocalStack | ||
* Docker | ||
* `make` | ||
* [`awslocal`](https://github.com/localstack/awscli-local) | ||
* [`samlocal`](https://github.com/localstack/aws-sam-cli-local) | ||
* NodeJS 14.x | ||
* [`ulid`](https://www.npmjs.com/package/ulid) | ||
|
||
## Installing | ||
|
||
Setup [Serverless Application Model (SAM)](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) and [AWS SAM CLI Local](https://github.com/localstack/aws-sam-cli-local) on your local machine. We also recommend using NodeJS 14.x alongside a [Node Version Manager](https://github.com/nvm-sh/nvm) to manage your NodeJS versions. | ||
|
||
|
||
Start LocalStack via: | ||
|
||
```sh | ||
localstack start -d | ||
``` | ||
|
||
## Deploy the application | ||
|
||
Let us first install the local dependencies: | ||
|
||
```sh | ||
npm install --save ulid | ||
``` | ||
|
||
To setup the infrastructure on LocalStack, run: | ||
|
||
```sh | ||
samlocal deploy -g | ||
``` | ||
|
||
You will be prompted to enter a name for the stack. Use the default options for the prompts and fill `Y` (`Yes`) for the confirmation prompt. The stack will be created and the output will be printed to the console. | ||
|
||
If you have made any changes to the application, you can update the stack by running: | ||
|
||
```sh | ||
samlocal deploy | ||
``` | ||
|
||
After deploying you can send a SQS message to the queue and see the Lambda function being triggered: | ||
|
||
```sh | ||
awslocal sqs send-message --queue-url http://localhost:4566/000000000000/MyQueue --message-body "{ "data" : "A" }" --delay-seconds 10 | ||
``` | ||
|
||
You will see a JSON output similar to the following: | ||
|
||
```json | ||
{ | ||
"MD5OfMessageBody": "64dfee8647a8264b25b01b7f22d72d3a", | ||
"MessageId": "22fbddd2-5add-4a03-a850-152780d786c1" | ||
} | ||
``` | ||
|
||
In the `template.yaml` we have defined the DynamoDB table and the Stream function with a filtering criteria. We instruct the Stream function to trigger the Lambda function only when the filtering criteria is satisfied. | ||
|
||
Using the SQS, we send a message body to the DynamoDB stream to match the specific filtering criteria. After the message is sent, we can see the Lambda function being triggered and you can check the logs to verify it. | ||
|
||
## Destroy the application | ||
|
||
To destroy the infrastructure on LocalStack, run: | ||
|
||
```sh | ||
samlocal delete | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
'use strict'; | ||
|
||
const AWS = require('aws-sdk'); | ||
const ULID = require('ulid'); | ||
const dynamo = new AWS.DynamoDB.DocumentClient(); | ||
|
||
const TABLE_NAME = process.env.TABLE_NAME | ||
|
||
exports.save = async (event) => { | ||
console.log(event); | ||
|
||
const object = event.body; | ||
|
||
const item = { | ||
id: ULID.ulid(), | ||
object, | ||
date: Date.now() | ||
} | ||
|
||
console.log(item); | ||
|
||
const savedItem = await saveItem(item); | ||
|
||
return { | ||
statusCode: 200, | ||
body: JSON.stringify(savedItem), | ||
} | ||
} | ||
|
||
exports.processDynamo = async (event) => { | ||
console.log(event); | ||
} | ||
|
||
exports.processASQS = async(event) => { | ||
console.log('Process A'); | ||
console.log(event); | ||
} | ||
|
||
async function saveItem(item) { | ||
const params = { | ||
TableName: TABLE_NAME, | ||
Item: item | ||
}; | ||
|
||
console.log(params) | ||
|
||
return dynamo.put(params).promise().then(() => { | ||
return item; | ||
}); | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
{ | ||
"name": "lambda-event-filtering", | ||
"version": "1.0.0", | ||
"description": "", | ||
"main": "handler.js", | ||
"scripts": { | ||
"test": "echo \"Error: no test specified\" && exit 1" | ||
}, | ||
"dependencies": { | ||
"ulid": "^2.3.0" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
AWSTemplateFormatVersion: '2010-09-09' | ||
Transform: AWS::Serverless-2016-10-31 | ||
|
||
Resources: | ||
|
||
ProcessDynamoDBStreamFunction: | ||
Type: AWS::Serverless::Function | ||
Properties: | ||
Handler: handler.processDynamo | ||
CodeUri: . | ||
Runtime: nodejs14.x | ||
Events: | ||
Stream: | ||
Type: DynamoDB | ||
Properties: | ||
Stream: !GetAtt DynamoDBTable.StreamArn | ||
BatchSize: 1 | ||
StartingPosition: TRIM_HORIZON | ||
FilterCriteria: | ||
Filters: | ||
- Pattern: "{ \"eventName\" : [\"INSERT\"] }" | ||
|
||
ProcessASQSFunction: | ||
Type: AWS::Serverless::Function | ||
Properties: | ||
Handler: handler.processASQS | ||
CodeUri: . | ||
Runtime: nodejs14.x | ||
Events: | ||
MySQSEvent: | ||
Type: SQS | ||
Properties: | ||
Queue: !GetAtt MyQueue.Arn | ||
BatchSize: 1 | ||
MaximumBatchingWindowInSeconds: 0 | ||
FilterCriteria: | ||
Filters: | ||
- Pattern: "{\"body\": { \"data\" : [\"A\"] }}" | ||
|
||
DynamoDBTable: | ||
Type: AWS::DynamoDB::Table | ||
Properties: | ||
AttributeDefinitions: | ||
- AttributeName: id | ||
AttributeType: S | ||
KeySchema: | ||
- AttributeName: id | ||
KeyType: HASH | ||
BillingMode: PAY_PER_REQUEST | ||
StreamSpecification: | ||
StreamViewType: NEW_IMAGE | ||
|
||
MyQueue: | ||
Type: AWS::SQS::Queue | ||
Properties: | ||
QueueName: MyQueue |