Skip to content

Commit ec58bd9

Browse files
christophgysinchristophgysin
committed
add aws-node-upload-to-s3-and-postprocess
Example to demonstrate how to upload a file to S3 using a HTML form, and have an S3 event trigger a lambda function. fixes #16
1 parent 044adb8 commit ec58bd9

File tree

8 files changed

+208
-0
lines changed

8 files changed

+208
-0
lines changed

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Have an example? Submit a PR or [open an issue](https://github.com/serverless/ex
5252
| [Serverless Single Page App Plugin](https://github.com/serverless/examples/tree/master/aws-node-single-page-app-via-cloudfront/serverless-single-page-app-plugin) <br/> A plugin to simplify deploying Single Page Application using S3 and CloudFront | nodeJS |
5353
| [Aws Text Analysis Via Sns Post Processing](https://github.com/serverless/examples/tree/master/aws-node-text-analysis-via-sns-post-processing) <br/> Example demonstrates how to setup a simple data processing pipeline | nodeJS |
5454
| [Aws Node Twilio Send Text Message](https://github.com/serverless/examples/tree/master/aws-node-twilio-send-text-message) <br/> Send a text message via twilio from aws lambda. [See live demo](http://twilio-serverless-example.surge.sh) | nodeJS |
55+
| [Upload To S3 And Postprocess](https://github.com/serverless/examples/tree/master/aws-node-upload-to-s3-and-postprocess) <br/> Upload files to S3 using a HTML form, then trigger a lambda function to postprocess. | nodeJS |
5556
| [Aws Alexa Skill](https://github.com/serverless/examples/tree/master/aws-python-alexa-skill) <br/> This example demonstrates how to use an AWS Lambdas for your custom Alexa skill. | python |
5657
| [Aws Rest With Dynamodb](https://github.com/serverless/examples/tree/master/aws-python-rest-api-with-dynamodb) <br/> Serverless CRUD service exposing a REST HTTP interface | python |
5758
| [Aws Scheduled Cron](https://github.com/serverless/examples/tree/master/aws-python-scheduled-cron) <br/> Example of creating a function that runs as a cron job using the serverless `schedule` event | python |

‎aws-node-upload-to-s3-and-postprocess/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules
2+
.serverless
3+
index.html

‎aws-node-upload-to-s3-and-postprocess/README.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Upload a file to S3 to trigger a lambda function
2+
3+
This example shows how to upload a file to S3 using a HTML form, and have S3
4+
trigger a lambda function.
5+
6+
## Use-cases
7+
8+
- Postprocess files uploaded to an S3 bucket.
9+
10+
## Setup
11+
12+
- Edit `serverless.yml` and choose a unique S3 bucket name.
13+
- Edit `generate-form.js` and fill in your `aws_access_key_id`,
14+
`aws_secret_access_key` and `bucket_name`.
15+
- Generate the HTML form:
16+
17+
18+
```bash
19+
node generate-form.js
20+
```
21+
22+
## Deploy
23+
24+
In order to deploy the example, simply run:
25+
26+
```bash
27+
serverless deploy
28+
```
29+
30+
The output should look similar to:
31+
32+
```bash
33+
Serverless: Creating Stack...
34+
Serverless: Checking Stack create progress...
35+
.....
36+
Serverless: Stack create finished...
37+
Serverless: Packaging service...
38+
Serverless: Uploading CloudFormation file to S3...
39+
Serverless: Uploading service .zip file to S3 (3.85 MB)...
40+
Serverless: Updating Stack...
41+
Serverless: Checking Stack update progress...
42+
........................
43+
Serverless: Stack update finished...
44+
Service Information
45+
service: upload-to-s3-and-postprocess
46+
stage: dev
47+
region: us-east-1
48+
api keys:
49+
None
50+
endpoints:
51+
None
52+
functions:
53+
upload-to-s3-and-postprocess-dev-postprocess
54+
55+
```
56+
57+
## Usage
58+
59+
Open the generated `frontend/index.html` in your browser, or run:
60+
61+
```bash
62+
xdg-open frontend/index.html
63+
```
64+
65+
Select a PNG image smaller than 1Mb, and click "Upload File to S3".
66+
67+
You should get an XML response similar to:
68+
69+
```xml
70+
<PostResponse>
71+
<Location>https://serverless-fetch-file-and-store-in-s3.s3.amazonaws.com/uploads%2Fimage.png</Location><
72+
Bucket>serverless-fetch-file-and-store-in-s3</Bucket>
73+
<Key>uploads/image.png</Key>
74+
<ETag>"08c03c6a24e5058b9f3556981a23b1d7"</ETag>
75+
</PostResponse>
76+
```
77+
78+
After a while, the postprocess function gets triggerd by an S3 event:
79+
80+
```bash
81+
serverless logs --function postprocess
82+
```
83+
84+
```
85+
START RequestId: e2decc94-f2a0-11e6-b641-e3fbcfad7d8c Version: $LATEST
86+
2017-02-14 12:32:30.350 (+02:00) e2decc94-f2a0-11e6-b641-e3fbcfad7d8c New .png object has been created: uploads/image.png (23975 bytes)
87+
END RequestId: e2decc94-f2a0-11e6-b641-e3fbcfad7d8c
88+
REPORT RequestId: e2decc94-f2a0-11e6-b641-e3fbcfad7d8c Duration: 2.84 ms Billed Duration: 100 msMemory Size: 1024 MB Max Memory Used: 29 MB
89+
```

‎aws-node-upload-to-s3-and-postprocess/frontend/index.template.html

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<html>
2+
<head>
3+
<title>S3 POST Form</title>
4+
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
5+
</head>
6+
7+
<body>
8+
<form action="%BUCKET_URL%" method="post" enctype="multipart/form-data">
9+
<input type="hidden" name="Content-Type" value="image/png">
10+
<input type="hidden" name="AWSAccessKeyId" value="%AWS_ACCESS_KEY%">
11+
<input type="hidden" name="acl" value="public-read">
12+
<input type="hidden" name="success_action_status" value="201">
13+
<input type="hidden" name="policy" value="%POLICY_BASE64%">
14+
<input type="hidden" name="signature" value="%SIGNATURE%">
15+
16+
Destination filename: <input type="text" name="key" value="uploads/image.png">
17+
<br>
18+
File to upload to S3:
19+
<input name="file" type="file">
20+
<br>
21+
<input type="submit" value="Upload File to S3">
22+
</form>
23+
</body>
24+
</html>

‎aws-node-upload-to-s3-and-postprocess/generate-form.js

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#!/usr/bin/env node
2+
3+
const crypto = require('crypto');
4+
const fs = require('fs');
5+
6+
const awsAccessKeyId = '<your access key id>';
7+
const awsSecretAccessKey = '<your secret access key>';
8+
const bucketName = '<your bucket name>';
9+
10+
const msPerDay = 24 * 60 * 60 * 1000;
11+
const expiration = new Date(Date.now() + msPerDay).toISOString();
12+
const bucketUrl = `https://${bucketName}.s3.amazonaws.com`;
13+
14+
const policy = {
15+
expiration,
16+
conditions: [
17+
['starts-with', '$key', 'uploads/'],
18+
{ bucket: bucketName },
19+
{ acl: 'public-read' },
20+
['starts-with', '$Content-Type', 'image/png'],
21+
{ success_action_status: '201' },
22+
],
23+
};
24+
25+
const policyB64 = Buffer(JSON.stringify(policy), 'utf-8').toString('base64');
26+
27+
const hmac = crypto.createHmac('sha1', awsSecretAccessKey);
28+
hmac.update(new Buffer(policyB64, 'utf-8'));
29+
30+
const signature = hmac.digest('base64');
31+
32+
fs.readFile('frontend/index.template.html', 'utf8', (err, input) => {
33+
if (err) {
34+
console.log(err);
35+
}
36+
37+
const data = input
38+
.replace(/%BUCKET_URL%/g, bucketUrl)
39+
.replace(/%AWS_ACCESS_KEY%/g, awsAccessKeyId)
40+
.replace(/%POLICY_BASE64%/g, policyB64)
41+
.replace(/%SIGNATURE%/g, signature);
42+
43+
fs.writeFile('frontend/index.html', data, 'utf8', (e) => {
44+
if (e) {
45+
console.log(e);
46+
}
47+
});
48+
});

‎aws-node-upload-to-s3-and-postprocess/handler.js

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
'use strict';
2+
3+
module.exports.postprocess = (event) => {
4+
event.Records.forEach((record) => {
5+
const filename = record.s3.object.key;
6+
const filesize = record.s3.object.size;
7+
console.log(`New .png object has been created: ${filename} (${filesize} bytes)`);
8+
});
9+
};

‎aws-node-upload-to-s3-and-postprocess/package.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"name": "upload-to-s3-and-postprocess",
3+
"description": "Upload a files to S3 to trigger a lambda function.",
4+
"version": "1.0.0",
5+
"author": "Christoph Gysin <[email protected]>",
6+
"license": "MIT",
7+
"dependencies": {
8+
}
9+
}

‎aws-node-upload-to-s3-and-postprocess/serverless.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
service: upload-to-s3-and-postprocess
2+
3+
frameworkVersion: ">=1.1.0"
4+
5+
custom:
6+
bucket: <your-bucket-name>
7+
8+
provider:
9+
name: aws
10+
runtime: nodejs4.3
11+
iamRoleStatements:
12+
- Effect: Allow
13+
Action:
14+
- s3:*
15+
Resource: "*"
16+
17+
functions:
18+
postprocess:
19+
handler: handler.postprocess
20+
events:
21+
- s3:
22+
bucket: ${self:custom.bucket}
23+
event: s3:ObjectCreated:*
24+
rules:
25+
- suffix: .png

0 commit comments

Comments
 (0)