SRE Capstone Project

The goal for this repository is to show the knowledge acquired during the execution of the SRE Bootcamp in particular with this major areas

1. Demonstrate the CI/CD, testing and clean code knowledge you acquired from the Bootcamp first modules.
2. Demonstrate the adaptability to learning new technologies.
3. Demonstrate the knowledge of infrastructure and code you acquired from the Bootcamp last modules.

Project Status

Metric	development	staging	main (Production)
CI/CD

Endpoints Implemented

Endpoint	Description
/	Homepage
/version	Reports the Container SHA
/_health	Application health
/login	Returns a JSON Web Token (JWT)
/cidr-to-mask	Converts from CIDR notation to Sub-Network Mask
/mask-to-cidr	Converts form Sub-Network Mask notation to CIDR

Practices Implemented

• Software Engineering and Testing Best Practices
  • SDLC implemented via TDD
• Computing AWS
  • EC2
  • ECS
• AWS Data Storage
  • S3
  • DynamoDB
  • RDS (mysql)
• Cloud Security Best Practices
  • Single entry to the application via ELB
  • Bastion Hosts to reach the VPC
  • Application and DB allocated in private networks.
  • NACs on Public Network
  • Security Groups in All network
  • Roles and Policies
• DevOps & CI/CD Fundamentals
  • Docker + DockerCompose Workflow
    • 2 Services enabled
      1. Flask Application
      2. DataStore (mysql) with Test Data
  • Single CI/CD pipeline with 4 gates
    1. Static Code Verification (pylint)
    2. Unit Test (pytest)
    3. Code Coverage up to 100%
• Infrastructure as Code (IaC)
  • State file stored via S3
  • Lock system implemented via DynamoDB
  • Three environments defined
    1. Development
    2. Staging
    3. Production

Docker image(s)

DockerHub

docker pull manquintero/academy-sre-bootcamp-manuel-quintero
docker run -d -p 8000:8000 manquintero/academy-sre-bootcamp-manuel-quintero

Elastic Container Registry

docker pull 664624836310.dkr.ecr.us-east-2.amazonaws.com/academy-sre-bootcamp-manuel-quintero
docker run -d -p 8000:8000 664624836310.dkr.ecr.us-east-2.amazonaws.com/academy-sre-bootcamp-manuel-quintero

Container Registry

docker pull gcr.io/academy-sre-bootcamp/academy-sre-bootcamp-manuel-quintero
docker run -d -p 8000:8000 gcr.io/academy-sre-bootcamp/academy-sre-bootcamp-manuel-quintero

Development environment

In order to bring the development area execute:

docker-compose up --build

This will bring the application container along with its database with sample data

Evaluating Static Code

pip install pylint
[[ -f "./python/requirements.txt" ]] && pip install -r "./python/requirements.txt"
pylint python/

Running Unit Test & Coverage

pip install pytest pytest-cov mock
[[ -f "./python/requirements.txt" ]] && pip install -r "./python/requirements.txt"
pytest python
pytest --cov-config=python/.coveragerc --cov python > pytest-coverage.txt

Although described as UT these runs rely on an external DB provided by the original project

Endpoints

Homepage

$ curl -s localhost:8000
OK

Healthcheck

$ curl -s localhost:8000/_health
HEALTHY

Version

$ curl -s localhost:8000/version
HEAD

Login

$ curl -s -d "username=antonio&password=cotorro" http://127.0.0.1:8000/login
{"data":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoidmlld2VyIn0._k6kmfmdOoKWWMT4qk9nFTz-7k-X_0UdS8tByaCaye8"}

cidr-to-mask

$ TOKEN="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoidmlld2VyIn0._k6kmfmdOoKWWMT4qk9nFTz-7k-X_0UdS8tByaCaye8"
$ curl -s -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" localhost:8000/cidr-to-mask?value=18
{"function":"cidrToMask","input":"18","output":"255.255.192.0"}

mask-to-cidr

$ TOKEN="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJyb2xlIjoidmlld2VyIn0._k6kmfmdOoKWWMT4qk9nFTz-7k-X_0UdS8tByaCaye8"
$ curl -s -H 'Accept: application/json' -H "Authorization: Bearer ${TOKEN}" localhost:8000/mask-to-cidr?value=255.128.0.0
{"function":"maskToCidr","input":"255.128.0.0","output":"9"}

Architecture

Branching Strategy

A simple branch strategy based on the mythical GitGlow with a few tailoring to fit this application lifecycle. These are the key points:

1. The branches mimic the target environments:
2. Any number of commits to the development branch can happen.
3. Changes from development shall be promoted to a staging via GitHub pull requests merge.
4. Any number of commits to the staging branch can happen.
5. Only pull request merges are allowed on Production
6. So far, no hot fixes are allowed in Production

CI/CD

The CI/CD process has been merged into a single pipeline for convenience. A few adaptations depends on the type of trigger.

1. Build
   1. Checks for SCCD in Python. [merge && pull_request]
2. Test
   1. Checks for Unit Testing via pytest. [merge && pull_request]
   2. Checks for Code Coverage. [merge && pull_request]
3. Container
   1. Builds the containers. [merge && pull_request]
   2. Tags and Pushes to three different Cloud Registries. [merge]
4. Terraform
   1. Initialize the workspace. [merge && pull_request]
   2. Checks for formatting. [merge && pull_request]
   3. Plan the provisioning. [merge]
   4. Apply the provisioning plan. [merge]

Systems with a lock mechanism in place, such as Terraform, are protected via GitHub environment concurrency

Deployment Strategy

Three Environments have been designated based on the Branching Strategy:

Environment	URL
Development	http://sre-bootcamp-development-198286806.us-east-2.elb.amazonaws.com
Staging	http://sre-bootcamp-staging-1328736561.us-east-2.elb.amazonaws.com
Production	http://sre-bootcamp-production-990790453.us-east-2.elb.amazonaws.com

The strategy is layered in two stages:

1. EC2 with an AutoScaling Group following a rolling-update process where wait_for_elb_capacity is defined.
2. ECS with a deployment_circuit_breaker configured with rollback.
   1. Every Service/Task feeds from the same ECR but different task definition with the SHA embedded into it.

IaC

Modules were defined for the next functionalities: