Skip to content

Commit b19fd08

Browse files
shipsships
committed
add Github Actions workflow
1 parent cd49d0d commit b19fd08

File tree

4 files changed

+193
-8
lines changed

4 files changed

+193
-8
lines changed

‎.github/workflows/awsdeploy.yml

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
# Based on https://docs.github.com/en/actions/deployment/deploying-to-your-cloud-provider/deploying-to-amazon-elastic-container-service
2+
3+
name: aws ecs deploy
4+
5+
on:
6+
push:
7+
branches:
8+
- main
9+
10+
env:
11+
AWS_REGION: us-east-1 # set this to your preferred AWS region, e.g. us-west-1
12+
ECR_REPOSITORY: pubpub-v7 # set this to your Amazon ECR repository name
13+
ECS_SERVICE: blake-core # set this to your Amazon ECS service name
14+
ECS_CLUSTER: blake-ecs-cluster-staging # set this to your Amazon ECS cluster name
15+
ECS_TASK_DEFINITION_TEMPLATE: blake-core # set this to the FAMILY of your task definition
16+
CONTAINER_NAME: core # set this to the name of the container in the
17+
# containerDefinitions section of your task definition
18+
19+
jobs:
20+
build:
21+
name: Build
22+
runs-on: ubuntu-latest
23+
environment: production
24+
25+
steps:
26+
- name: Checkout
27+
uses: actions/checkout@v4
28+
29+
- name: Configure AWS credentials
30+
uses: aws-actions/configure-aws-credentials@v4
31+
with:
32+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
33+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
34+
aws-region: ${{ env.AWS_REGION }}
35+
36+
- name: Get short sha
37+
id: shortsha
38+
run: echo "sha_short=$(git describe --always --abbrev=40 --dirty)" >> $GITHUB_OUTPUT
39+
40+
- name: Login to Amazon ECR
41+
id: login-ecr
42+
uses: aws-actions/amazon-ecr-login@v2
43+
44+
- name: Build, tag, and push image to Amazon ECR
45+
id: build-image
46+
env:
47+
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
48+
IMAGE_TAG: ${{ steps.shortsha.outputs.sha_short }}
49+
run: |
50+
# Build a docker container and
51+
# push it to ECR so that it can
52+
# be deployed to ECS.
53+
docker build \
54+
--platform linux/amd64 \
55+
--build-arg PACKAGE=core \
56+
--build-arg PACKAGE_DIR=core \
57+
-t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
58+
.
59+
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
60+
echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
61+
62+
deploy:
63+
name: Deploy
64+
runs-on: ubuntu-latest
65+
environment: production
66+
needs:
67+
- build
68+
69+
steps:
70+
- name: Checkout
71+
uses: actions/checkout@v4
72+
73+
- name: Configure AWS credentials
74+
uses: aws-actions/configure-aws-credentials@v4
75+
with:
76+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
77+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
78+
aws-region: ${{ env.AWS_REGION }}
79+
80+
- name: Get short sha
81+
id: shortsha
82+
run: echo "sha_short=$(git describe --always --abbrev=40 --dirty)" >> $GITHUB_OUTPUT
83+
84+
- name: Login to Amazon ECR
85+
id: login-ecr
86+
uses: aws-actions/amazon-ecr-login@v2
87+
88+
- name: Retrieve Task Definition contents from template
89+
id: get-taskdef
90+
run: |
91+
aws ecs describe-task-definition \
92+
--task-definition $ECS_TASK_DEFINITION_TEMPLATE \
93+
--query taskDefinition >> template_task_def.json
94+
95+
- name: Interpolate image tag
96+
id: write-tag
97+
env:
98+
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
99+
IMAGE_TAG: ${{ steps.shortsha.outputs.sha_short }}
100+
run: |
101+
echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
102+
103+
- name: Fill in the new image ID in the Amazon ECS task definition
104+
id: task-def
105+
uses: aws-actions/amazon-ecs-render-task-definition@c804dfbdd57f713b6c079302a4c01db7017a36fc
106+
with:
107+
task-definition: template_task_def.json
108+
container-name: ${{ env.CONTAINER_NAME }}
109+
image: ${{ steps.write-tag.outputs.image }}
110+
111+
- name: Deploy Amazon ECS task definition
112+
uses: aws-actions/amazon-ecs-deploy-task-definition@df9643053eda01f169e64a0e60233aacca83799a
113+
with:
114+
task-definition: ${{ steps.task-def.outputs.task-definition }}
115+
service: ${{ env.ECS_SERVICE }}
116+
cluster: ${{ env.ECS_CLUSTER }}
117+
wait-for-service-stability: true

‎README.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,3 +80,74 @@ We currently have a race condition where dev will sometimes fail because we can'
8080
- https://github.com/vercel/turbo/issues/460
8181

8282
`core` depends on `ui` which depends on `utils`. `utils` often takes longer to build than it does for `ui` to start building, which causes an error to be thrown because `utils` d.ts file has been cleared out during its build and hasn't been replaced yet. This generates an error, but is quick to resolve, so doesn't break actual dev work from beginning. It does make the console output messier though.
83+
84+
85+
## Building and deploying for AWS environments
86+
87+
All change management to Knowledge Futures' production environment is done through github actions.
88+
This environment runs on AWS ECS and leverages terraform to allow reproducible, parametric environments.
89+
90+
Services running in AWS ECS are scheduled using "Task Definitions", which are CRUDdy resources
91+
including all details for a container. We don't want to tie code releases to terraform "infrastructure" changes,
92+
but the service "declaration" relies on this Task Definition to exist.
93+
94+
Therefore based on community patterns we have seen, the flow is roughly this:
95+
1. The infrastructure code in terraform declares a "template" Task Definition.
96+
2. Terraform is told not to change the "service" based on changes to the Task Definition.
97+
3. Any changes to the template will be picked up by the next deploy, which is done outside of Terraform.
98+
4. Github Actions builds new containers on merge, and will use AWS-provided Actions to template the literal correct Task Definition and update the Service.
99+
100+
### Updating deployment topology and/or environment variables/container settings
101+
102+
To change "infrastructure settings", which include anything from networking to env vars,
103+
make changes to `./infrastructure/terraform/aws`. Use `terraform apply` there to update
104+
the infrastructure and/or Task Definition Template. See that directory for more info.
105+
106+
Then you must perform a Github Actions Deploy, either by pushing your changes to main or
107+
with local `act` CLI.
108+
109+
### Updating container versions with github actions
110+
111+
The core automation workflow can be examined in [`awsdeploy.yml`](./.github/workflows/awsdeploy.yml)
112+
113+
There is a Dockerfile in this repository that builds a container for one package. You can use it like:
114+
115+
```
116+
docker build \
117+
--platform linux/amd64 \
118+
--build-arg PACKAGE=core \
119+
--build-arg PACKAGE_DIR=core \
120+
-t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
121+
.
122+
```
123+
124+
Note how this matches the invocation used in the Actions workflow file.
125+
126+
We automatically build and push a container to AWS ECR with the github SHA as a tag.
127+
128+
### Using `act` to run the deploy events locally
129+
130+
If you need to build or validate a change and deploy to production, you can use the [`act` cli](https://github.com/nektos/act):
131+
132+
```
133+
act \
134+
-W .github/workflows/awsdeploy.yml \
135+
--container-architecture linux/amd64 \
136+
--secret-file ~/.aws/pubpub.secrets \
137+
-j deploy
138+
```
139+
140+
**Secrets:** Though you will have an `~/.aws/credentials` file, this is not the format for secrets access that
141+
`act` requires, so I copy the key-value pairs in that file into a file that matches the Github
142+
secrets called `~/.aws/pubpub.secrets`:
143+
144+
```
145+
AWS_ACCESS_KEY_ID=...
146+
AWS_SECRET_ACCESS_KEY...
147+
```
148+
149+
**Dirty worktree:** If you run `act` like this, the image will be conveniently tagged with the latest SHA plus `-dirty`.
150+
Images tagged with a SHA alone should be idempotently built, but `-dirty` can be changed/overwritten.
151+
152+
**TODO:**
153+
- [ ] allow deploying without a rebuild, so that a rollback is convenient

‎infrastructure/terraform/aws/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ module "service_core" {
4444
{name = "API_KEY", value = "undefined"},
4545
{name = "JWT_SECRET", value = "undefined"},
4646
{name = "MAILGUN_SMTP_USERNAME", value = "undefined"},
47-
{name = "NEXT_PUBLIC_PUBPUB_URL", value = "undefined"},
47+
{name = "NEXT_PUBLIC_PUBPUB_URL", value = "https://v7.pubpub.org"},
4848
{name = "NEXT_PUBLIC_SUPABASE_URL", value = "undefined"},
4949
{name = "SENTRY_AUTH_TOKEN", value = "undefined"},
5050
{name = "SUPABASE_SERVICE_ROLE_KEY", value = "undefined"},

‎infrastructure/terraform/aws/modules/ecs-service/main.tf

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,16 @@ module "ecs_service" {
55
cluster_arn = var.cluster_info.cluster_arn
66
enable_execute_command = true
77

8+
# allow github actions to update the service without confusing TF
9+
ignore_task_definition_changes = true
10+
811
cpu = var.resources.cpu
912
memory = var.resources.memory
1013
desired_count = var.resources.desired_count
1114
# execution_role_arn = aws_iam_role.ecs_task_execution_role.arn
1215
# task_role_arn = aws_iam_role.ecs_task_role.arn
1316

14-
# Container definition(s)
17+
# TEMPLATE Container definition(s).
1518
container_definitions = {
1619

1720
"${var.service_name}" = {
@@ -56,10 +59,4 @@ module "ecs_service" {
5659
Environment = "${var.cluster_info.name}-${var.cluster_info.environment}"
5760
Project = "Pubpub-v7"
5861
}
59-
60-
# this lifecycle property allows us to update the version of the container image without terraform clobbering it later
61-
# changing the container image creates a "revision" of the task definition
62-
# lifecycle {
63-
# ignore_changes = [services.core.container_definitions.core.image]
64-
# }
6562
}

0 commit comments

Comments
 (0)