feat(deployment): provision scaleway with tf #63
Commits on Oct 26, 2023
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
chore: do not require IAMMAnager permission for terraform
Unfortunately, creating an application and the associated access key requires the IAMManager permission, which is an organization-level right that gives full access to the security of the organization. So instead of automatically creating a Scaleway application for Airflow's object storage access, we now require extra variables that point to a pre-created Scaleway application dedicated to Airflow (application ID, access key and secret key).
-
chore: better name for Docker containers
Docker Compose defaults to using the name of the parent directory to name containers, which currently results in containers named `docker-api-1`, `docker-airflow-db-1`, etc. By specifying a `name` in the Compose file, we can tell Compose to create containers named `data-inclusion-api-1`, etc.
-
chore: avoid extra git clone when deploying
Instead of cloning from the source repository, we upload the relevant files using the file provisioner. Note that we have to be quite specific with the directories that we upload in order to avoid accidentally uploading sensitive files such as `terraform.tfvars.json`.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
chore: chown airflow files to group 0
According to the documentation for the Airflow Docker image ( https://airflow.apache.org/docs/docker-stack/build.html#important-notes-for-the-base-images ) files need to be owned by user `airflow` and group `root` (0). Group `airflow` doesn't exist in the image, which apparently caused Docker to fall back to group 0 anyway, so this change should not have any impact.
-
-
-
-
-
-
-
chore: fix file copying on empty instance
If the target directory does not exist Terraform won't create it. Also fix error reporting from Docker Compose.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
chore: move docker-compose.yml into stack_data module
Also upload to just `/root/data-inclusion`, the `/deployment` subdirectory is now unnecessary on the target.
-
chore: replace public_hostname with dns_zone + optional dns_subdomain
This goes part of the way to supporting review apps with each their own subdomain while sharing a parent DNS zone.
-
-
-
-
-
chore: attempt to use Scaleway Container registry
Pulls from ghcr.io seem to be awfully slow from Scaleway instances (more than 10 minutes to pull the pipeline image). Let's see if it's worth changing the registry.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
chore: update S3 backend config following tf upgrade
The latest version of terraform changed the syntax for specifying S3 endpoints; also it now queries by default IAM, STS and metadata AWS APIs which Scaleway Object Storage doesn't have.
-
Revert "chore: attempt to use Scaleway Container registry"
This reverts commit 69be53a.
-
-
-
fix: correctly create PG extensions
Because docker-entrypoint-initdb.d/10_postgis.sh was marked as executable, it was executed instead of sourced by docker-entrypoint.sh causing the "${psql[@]}" syntax to fail. This also caused the processing of the init scripts to be aborted, causing 10_unaccent.sh not to be executed at all. Note that this was not detected until one tried to use the expected extensions (plpython3u, unaccent), since only the first launch of the postgres container (where the DB was created) failed. -
fix: rename Postgres init scripts to match contents
The names of 10_plpython3u.sh and 10_postgis.sh were swapped.
