Skip to content

Commit a182db9

Browse files
julianwoodjulianwood
authored
Merge pull request #2519 from jack-lebon/jlebon-textract-lambda-sam-python
New serverless pattern - textract-lambda-sam-python
2 parents ebfb092 + f9a3d7b commit a182db9

File tree

5 files changed

+343
-0
lines changed

5 files changed

+343
-0
lines changed

‎textract-lambda-sam-python/README.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Automatically Detect Text with Amazon Textract and AWS Lambda
2+
3+
This pattern explains how to deploy an AWS SAM application with Amazon S3, AWS Lambda, and Amazon DynamoDB to detect text stored within pdf or image files. When an image file is uploaded to Amazon S3, the event-driven workflow begins, sending an event to AWS Lambda. This Lambda function, written in Python, invokes the Amazon Textract `DetectDocumentText` function using boto3. Once Textract returns the response, Lambda stores the detected text in a DynamoDB table.
4+
5+
Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/textract-lambda-sam-python](https://serverlessland.com/patterns/textract-lambda-sam-python)
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
15+
16+
## Deployment Instructions
17+
18+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
19+
```
20+
git clone https://github.com/aws-samples/serverless-patterns
21+
```
22+
2. Change directory to the pattern directory:
23+
```
24+
cd textract-lambda-sam-python
25+
```
26+
3. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
27+
```
28+
sam deploy --guided
29+
```
30+
4. During the prompts:
31+
* Enter a stack name
32+
* Enter the desired AWS Region
33+
* Allow SAM CLI to create IAM roles with the required permissions.
34+
35+
Once you run `sam deploy --guided` for the first time, and save the arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future deployments to use these defaults.
36+
37+
5. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
38+
39+
## How it works
40+
41+
- This pattern is designed to create all services required to run this workflow.
42+
- The workflow begins with an Amazon S3 bucket.
43+
- When an object is created within the S3 bucket, it sends an event to an AWS Lambda function.
44+
- This Lambda function invokes Amazon Textracts's [DetectDocumentText](https://docs.aws.amazon.com/textract/latest/dg/API_DetectDocumentText.html) function, which synchronously analyses the newly stored file.
45+
- Once this task is complete, the Lambda function stores the results in an Amazon DynamoDB table.
46+
47+
## Testing
48+
1. Upload a test image, or pdf file, containing text to the Amazon S3 bucket created during the deployment step. You can either do this via the console, or by running the following command. Replace `<your_file>` with your image or pdf file name. Replace `<S3BucketName>` with the name of the S3 bucket generated from the AWS SAM deployment outputs.
49+
```
50+
aws s3 cp <your_file> s3://<S3BucketName>
51+
```
52+
3. Wait for the lambda-start-detect-document-text-textract Lambda function to complete, then retrieve the record output from the DynamoDB table. Replace `<DDBTableName>` with the name of the DynamoDB table generated by the AWS SAM deployment output, in this case, it will be `TextractResultsTable`. You can use the following command:
53+
```
54+
aws dynamodb scan --table-name <DDBTableName>
55+
```
56+
For example: aws dynamodb scan --table-name TextractResultsTable
57+
4. There should now be a newly uploaded record within DynamoDB displayed, depending on the file, the key 'DetectedText' will either be empty, or contain values indicating what was found in the uploaded file.
58+
59+
## Cleanup
60+
61+
1. Delete the stack
62+
```
63+
sam delete
64+
```
65+
66+
⚠️ **IMPORTANT** - The above command `sam delete` does not delete the **Amazon S3 bucket** if there are still objects stored within it. Please [empty the bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/empty-bucket.html) before running the above command.
67+
----
68+
Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
69+
70+
SPDX-License-Identifier: MIT-0

‎textract-lambda-sam-python/example-pattern.json

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
{
2+
"title": "Automatic Text Detection with Amazon Textract",
3+
"description": "An event-driven workflow to automatically detect and store text found within pdf files by leveraging Amazon Textract, AWS Lambda, and Amazon DynamoDB.",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This sample project demonstrates how to deliver an event-driven architecture to detect text within pdf files, while storing the results in Amazon DynamoDB.",
11+
"This pattern allows you to store image files in an Amazon S3 bucket, which triggers the workflow. Upon an object being created in the S3 bucket, a Lambda function is invoked, which initiates Amazon Textracts's DetectDocumentText function. Once the function call is finished and it has retrieved any text found within the file, the Lambda function stores this information in our DynamoDB table.",
12+
"This pattern deploys 1 S3 bucket, 1 Lambda Function, and 1 DynamoDB Table."
13+
]
14+
},
15+
"gitHub": {
16+
"template": {
17+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/textract-lambda-sam-python",
18+
"templateURL": "https://github.com/aws-samples/serverless-patterns/main/textract-lambda-sam-python/template.yaml",
19+
"projectFolder": "textract-lambda-sam-python",
20+
"templateFile": "template.yaml"
21+
}
22+
},
23+
"resources": {
24+
"bullets": [
25+
{
26+
"text": "Amazon Simple Storage Service (S3)",
27+
"link": "https://aws.amazon.com/s3/"
28+
},
29+
{
30+
"text": "AWS Lambda",
31+
"link": "https://aws.amazon.com/lambda/"
32+
},
33+
{
34+
"text": "Amazon Textract",
35+
"link": "https://aws.amazon.com/textract/"
36+
},
37+
{
38+
"text": "Amazon DynamoDB",
39+
"link": "https://aws.amazon.com/dynamodb/"
40+
}
41+
]
42+
},
43+
"deploy": {
44+
"text": [
45+
"sam deploy"
46+
]
47+
},
48+
"testing": {
49+
"text": [
50+
"See the GitHub repo for detailed testing instructions."
51+
]
52+
},
53+
"cleanup": {
54+
"text": [
55+
"Delete the stack: sam delete"
56+
]
57+
},
58+
"authors": [
59+
{
60+
"name": "Jack Le Bon",
61+
"image": "https://serverlessland.com/assets/images/resources/contributors/ext-jack-le-bon.jpg",
62+
"bio": "AWS Solutions Architect",
63+
"linkedin": "jack-le-bon"
64+
}
65+
]
66+
}
67+

‎textract-lambda-sam-python/src/lambda-start-detect-document-text-textract.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import json
2+
import boto3
3+
import os
4+
import decimal
5+
6+
client = boto3.client('textract')
7+
dynamodb = boto3.resource('dynamodb')
8+
table = dynamodb.Table(os.environ.get('dynamoDBTableName'))
9+
10+
def lambda_handler(event, context):
11+
bucket = event['Records'][0]['s3']['bucket']['name']
12+
key = event['Records'][0]['s3']['object']['key']
13+
record = {'Image': key, 'Bucket': bucket}
14+
response = client.detect_document_text(
15+
Document={
16+
'S3Object': {
17+
'Bucket': bucket,
18+
'Name': key
19+
}
20+
}
21+
)
22+
record['DetectedText'] = response
23+
dynamodb_response = table.put_item(Item=json.loads(json.dumps(record), parse_float=decimal.Decimal))
24+
return {
25+
'statusCode': 200,
26+
'body': json.dumps('Textract complete.')
27+
}

‎textract-lambda-sam-python/template.yaml

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
Transform: AWS::Serverless-2016-10-31
3+
Description: >
4+
An Amazon S3 bucket for the storing of Image files.
5+
Upon uploading a new object, triggers an AWS Lambda function which consumes the event and leverages Amazon Textract's DetectDocumentText function to find text within an image or file.
6+
The Lambda function then stores the result in Amazon DynamoDB. (uksb-1tthgi812) (tag:textract-lambda-sam-python)
7+
8+
Resources:
9+
# S3 bucket to store Image files from the user.
10+
ImageFileBucket:
11+
Type: AWS::S3::Bucket
12+
13+
# Define the DynamoDB table
14+
DynamoDBTable:
15+
Type: AWS::DynamoDB::Table
16+
Properties:
17+
AttributeDefinitions:
18+
- AttributeName: "Image"
19+
AttributeType: "S"
20+
- AttributeName: "Bucket"
21+
AttributeType: "S"
22+
KeySchema:
23+
- AttributeName: "Image"
24+
KeyType: HASH
25+
- AttributeName: "Bucket"
26+
KeyType: RANGE
27+
BillingMode: PAY_PER_REQUEST
28+
TableName: "TextractResultsTable"
29+
30+
# Lambda Function to begin Amazon Rekognition's DetectLabels function.
31+
StartProcessingFunction:
32+
Type: AWS::Serverless::Function
33+
Properties:
34+
FunctionName: lambda-start-detect-document-text-textract
35+
Runtime: python3.13
36+
Handler: src/lambda-start-detect-document-text-textract.lambda_handler
37+
MemorySize: 128
38+
Timeout: 10
39+
Policies:
40+
- Version: '2012-10-17'
41+
Statement:
42+
- Effect: Allow
43+
Action:
44+
- "s3:GetObject"
45+
- "s3:PutObject"
46+
Resource: "*"
47+
- Effect: Allow
48+
Action:
49+
- "textract:DetectDocumentText"
50+
Resource: "*"
51+
- Effect: Allow
52+
Action:
53+
- "dynamodb:PutItem"
54+
Resource: !GetAtt DynamoDBTable.Arn
55+
Events:
56+
S3Event:
57+
Type: S3
58+
Properties:
59+
Bucket:
60+
Ref: ImageFileBucket
61+
Events: s3:ObjectCreated:*
62+
Environment:
63+
Variables:
64+
dynamoDBTableName: !Ref DynamoDBTable
65+
66+
Outputs:
67+
ImageFileBucket:
68+
Value: !Ref ImageFileBucket
69+
Description: S3 Bucket for object storage
70+
DynamoDBTable:
71+
Value: !Ref DynamoDBTable
72+
Description: DynamoDB table containing Textract Results

‎textract-lambda-sam-python/textract-lambda-sam-python.json

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
{
2+
"title": "Automatic Text Detection with Amazon Textract",
3+
"description": "An event-driven workflow to automatically detect and store text found within pdf files by leveraging Amazon Textract, AWS Lambda, and Amazon DynamoDB.",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This sample project demonstrates how to deliver an event-driven architecture to detect text within pdf files, while storing the results in Amazon DynamoDB.",
11+
"Upon an object creation in the S3 bucket, a Lambda function is invoked, which initiates Amazon Textracts's DetectDocumentText function. Textract returns the results to the Lambda function which stores this information in the DynamoDB table.",
12+
"This pattern deploys 1 S3 bucket, 1 Lambda Function, and 1 DynamoDB Table."
13+
]
14+
},
15+
"gitHub": {
16+
"template": {
17+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/textract-lambda-sam-python",
18+
"templateURL": "https://github.com/aws-samples/serverless-patterns/main/textract-lambda-sam-python/template.yaml",
19+
"projectFolder": "textract-lambda-sam-python",
20+
"templateFile": "template.yaml"
21+
}
22+
},
23+
"resources": {
24+
"bullets": [
25+
{
26+
"text": "Amazon Simple Storage Service (S3)",
27+
"link": "https://aws.amazon.com/s3/"
28+
},
29+
{
30+
"text": "AWS Lambda",
31+
"link": "https://aws.amazon.com/lambda/"
32+
},
33+
{
34+
"text": "Amazon Textract",
35+
"link": "https://aws.amazon.com/textract/"
36+
},
37+
{
38+
"text": "Amazon DynamoDB",
39+
"link": "https://aws.amazon.com/dynamodb/"
40+
}
41+
]
42+
},
43+
"deploy": {
44+
"text": [
45+
"sam deploy"
46+
]
47+
},
48+
"testing": {
49+
"text": [
50+
"See the GitHub repo for detailed testing instructions."
51+
]
52+
},
53+
"cleanup": {
54+
"text": [
55+
"Delete the stack: sam delete"
56+
]
57+
},
58+
"authors": [
59+
{
60+
"name": "Jack Le Bon",
61+
"image": "https://serverlessland.com/assets/images/resources/contributors/ext-jack-le-bon.jpg",
62+
"bio": "AWS Solutions Architect",
63+
"linkedin": "jack-le-bon"
64+
}
65+
],
66+
"patternArch": {
67+
"icon1": {
68+
"x": 10,
69+
"y": 50,
70+
"service": "s3",
71+
"label": "Amazon S3"
72+
},
73+
"icon2": {
74+
"x": 40,
75+
"y": 50,
76+
"service": "lambda",
77+
"label": "AWS Lambda"
78+
},
79+
"icon3": {
80+
"x": 80,
81+
"y": 25,
82+
"service": "textract",
83+
"label": "Amazon Textract"
84+
},
85+
"icon4": {
86+
"x": 80,
87+
"y": 70,
88+
"service": "dynamodb",
89+
"label": "Amazon DynamoDB"
90+
},
91+
"line1": {
92+
"from": "icon1",
93+
"to": "icon2",
94+
"label": "Object Created"
95+
},
96+
"line2": {
97+
"from": "icon2",
98+
"to": "icon3",
99+
"label": "Document"
100+
},
101+
"line3": {
102+
"from": "icon2",
103+
"to": "icon4",
104+
"label": "Results"
105+
}
106+
}
107+
}

0 commit comments

Comments
 (0)