Developing with AWS Step Functions

The Scenario

To illustrate the use of AWS Step Functions I have created a scenario that describes a process where university students caught plagiarising on exams and/or assignments are required to take a test to assess their knowledge of the universities referencing standards.

Visually, the process looks like this:

The process starts by:

1. Registering the plagiarism incident
2. Scheduling an exam. Students have one week to complete the test.
3. Send the student an email notification to inform them of the requirement
4. The process waits for the exam dedline to complete, before
5. Validating the results
6. Determining whether or not the student has sat the exam, or passed. Students get three attempts to pass the exam before...
   • The incident is resolved, or
   • Administrative action is taken.

The Architecture

This example uses a simple architecture, hosting a static website in Amazon Simple Storage Service (Amazon S3) using Vue.js to invoke an Amazon API Gateway API. The API is configured with an AWS Service integration which invokes the AWS Step Function state machine for the plagiarism workflow. Along the way, the task states in the state machine are executing AWS Lambda functions which are periodically persisting data to Amazon DynamoDB and send messages via Amazon Simple Notification Service.

The Deployment Pipeline

All resources for this architecture are deployed using Amazon CloudFormation and the Serverless Application Model (SAM). SAM is a superset of CloudFormation that make deploying Serverless applications easier by providing transformations for Functions, API's and DynamoDB.

To deploy the template we simply run two AWS CLI commands in the CloudFormation API. The package takes our SAM template as an input and we specify an file name as an output. CloudFormation will package all our function code and upload it to the specified S3 bucket, substituting the Code Uri property values with the location of the zipped up functions in the S3 bucket.

aws cloudformation package \
    --template-file template.yaml \
    --output-template-file packaged.yaml \
    --s3-bucket developingwithstepfunctions

Once the package has been created, execute the deploy command to create the application stack

aws cloudformation deploy \
    --template-file packaged.yaml \
    --stack-name developing-with-step-functions \
    --capabilities CAPABILITY_IAM

Using this approach allow development teams to automate their deployments through their CI/CD pipelines and easily create discreet stacks of resources that align to environments (DEV TEST and PROD).

Services used in the pipeline include:

• AWS Code Pipeline
• AWS Code Build
• AWS CloudFormation

Using Code Deploy allows us to use on-demand build environments, which are configured using a buildspec.yaml file.

Credits

Cookiecutter SAM for Python Lambda

The application was initialised using the Cookiecutter SAM for Python Lambda. The cookiecutter template provides a wizard like command line experience to create a Serverless app based on SAM and Python 3.6.

For more details check out Heitor Lessa's repository at heitorlessa/cookiecutter-aws-sam-python

Also, a great video tutorial on how to use the Cookiecutter (and more) can be found on the AWS Twitch channel - Build on Serverless | Building the "Simple Crypto Service" Python Project

Resources

Step Functions

• AWS Step Functions
• AWS Step Functions Developer Guide
• statelint
• Amazon States Language
• Step Functions Ruby Activity Worker example

Reference Architectures

• Serverless Reference Architecture: Image Recognition and Processing Backend
• Serverless Reference Architecture: Snapshot Management
• Batch Architecture for Life Sciences Workloads
• S3 Bucket Sync Reference
• Step Functions Plug-In for Serverless Framework

AWS Developer Resources

• Serverless Application Model
• DevOps and AWS
• AWS Developer Tools