Home
We provide templates that help with the process of deploying an Authoritative Data Lookup Service to Amazon Web Services (AWS). The Docker Components will be deployed to various AWS services. Prior to beginning the deploy process, you will need to have basic knowledge of AWS services being used.
There is an expectation that you have general knowledge of AWS services and an AWS account. Some of the services you will need to be familiar with and set up in advance are:
- VPC (Virtual Private Cloud) - Within the VPC, you define a public subnet that provides outside access to the application. You will also define a private subnet(s) that provides access to ECS, S3, and EFS components. The setup process will include configurations for things like firewall settings within a security group and ports. (It is common for a VPC to be shared among applications at an institution.)
- ECR (Elastic Container Registry) - The qa_authority_lookup container is stored in the AWS container registry. It is also available through Docker Hub. TODO: Add links to the containers.
- ECS (Elastic Container Service) - The application running as a Docker container is deployed through ECS on an EC2 cluster within your VPC. (The templates will provision and set this up.)
- S3 file storage (Simple Storage Service) - Environment variable configurations are stored in S3. (The templates will provision and set this up.)
- EFS (Elastic File System) - The database and authority configurations are each stored in EFS which allows them to have mountable drives for access through Docker volumes. (The templates will provision and set these up.)
- CloudFormation - CloudFormation processes templates and provisions the ECS, S3, and EFS resources based on the configurations in the templates.
The architecture follows Docker micro-services paradigm with the application and the database each set up as a service. This is the same whether on AWS or another deployment platform.
This primary deployment components are:
- Hosted on EC2 through Amazon Elastic Container Service (ECS)
- The application (app) running within a container.
- The MySQL database (mysql) running within a container.
- Stored on S3 (Simple Storage Service)
- An environment file defining secure environment variables.
- Mounted on Elastic File System (EFS) or Elastic Block Store (EBS)
- A volume holding charts generated by the application's performance monitoring module.
- A volume holding authority configurations that determine which Authoritative Data is available for lookup.
- A volume holding the database files.
This section describes structures used by the templates that cannot be created or setup by a template. The results of these steps will be recorded as ParameterValues in prerequisites.env. The env file is read by aws-prerequisites.yaml to complete the setup process. For all these steps, see Required Background Knowledge for more information and links to documentation.
Copy prerequisites.env.example and name it prerequisites.env. Edit this file and update parameter values as instructed in the following steps.
Locate your AWS VPC ID. For many organizations, this will already exist. If it does not exist, it will need to be created. This is typically setup by your administrator or central IT service.
- Record your organizations AWS VPC ID as the
"ParameterValue"for"ParameterKey": "AWSVpcId"(ex. "vpc-4d4bec2b"). This is required for load balancer configuration and security group rules. NOTE: You may have to create the AWS VPC if this is the first time your organization is using AWS. This is typically setup by your administrator or central IT service. - Select 2 Subnets from your VPC and record the IDs as a comma separated list in the
"ParameterValue"for"ParameterKey": "SubnetIds"(ex. "subnet-d7ad4ca1,subnet-83a890da") - Create an S3 bucket and record the bucket name as the
"ParameterValue"for"ParameterKey": "S3BucketName"(ex. TODO) - Create an EFS database access point and record the TODO as the
"ParameterValue"for"ParameterKey": "EFSDatabaseAccessPoint"(ex. TODO)
The person running the templates must have permissions that allow them to provision the resources created by the templates. Specifically, the user must be able to...
- create EC2 cluster through ECS (EC2 console)
- request an S3 bucket (S3 console)
- create EFS mounts (EFS console)
- set up IAM access roles (IAM dashboard)
Upload the application and database docker images through ECR console
TBD - tentatively - grab from Docker hub - need instructions
To run the scripts, you must have enough admin privileges to be able to provision all the resources in the template. This deployment assumes that you have an AWS account, and an Identity and Access Management (IAM) account within your AWS account that allows enough access to create, update, and destroy all the necessary resources. If your IAM user lacks sufficient privileges, your deployment will stop with an error message telling you so.
To use this deployment, you will need the following pieces of infrastructure in place in your AWS account before you begin:
- A repo containing a successfully built Docker image. This can be an Elastic Container Registry (ECR) repository or a public or private repo hosted elsewhere. Building the application image and pushing it to the repo should already be done, and you will need the full path to the image, including the repository URL and image tag (usually :latest). Full documentation on this process is available at .
- A Simple Storage Service (S3) bucket containing an environment file that has been populated with valid values. The file contains secrets and other parameters necessary for the application to run, an .env.example file has been provided as a template. The S3 bucket must contain this file, and the bucket permissions must be configured to allow access from the Amazon ECS task execution IAM role, but it should not be open to the public. Full documentation on setup is available at https://docs.aws.amazon.com/AmazonECS/latest/developerguide/taskdef-envfiles.html.
- A filesystem that the containers can mount. At this point, only EFS volumes are supported, EBS will be supported in the future. This deployment assumes one EFS filesystem with two separate access points. The first access point contains the authority files, and it should be pre-populated with the files you want to use. The second will contain the database storage files, and this should provision itself on the first run. While it's not strictly required, you should also have control over a domain space where you are able to create and update DNS records.
Two templates are provided, a CloudFormation template and a standalone ECS task definition. Both require customization to your environment by populating various values; in the CloudFormation version these values can be passed in as parameters when the stack is created or updated.