Use the latest containers

[ahosgood]

Possible replacement for #1374

Steps/things to test

Setup

• Stop and remove all services with docker compose stop && docker compose rm && docker system prune -a
• Start up all services with docker compose up -d
• Check site on http://localhost:8000/

Dev commands

• Run code formatting with either docker compose exec dev format from your host machine or simply format from inside the dev container (there should be LOTS of fixes from this!)
• Pull data from the develop environment of Platform.sh with either docker compose exec dev pull from your host machine or pull from inside the dev container (the first pull will take longer because it will install and set up the Platform.sh CLI) - ensure you have the PLATFORMSH_CLI_TOKEN env var set up in your .env file
• Try pulling just the media or data with pull-media and pull-data
• Try pulling the main branch with pull main
• Upgrade dependencies with docker compose exec dev upgrade from your host machine or upgrade from inside the dev container (if there are any upgrades you should restart the app container with docker compose up -d --build app) - there should be quite a few of these to test with
• Add a new Python dependency with docker compose exec dev poetry add emoji or poetry add emoji in dev (restart the app container once you are done) - this should also be possible in the app container with docker compose exec app poetry add emoji and you then have to restart your dev container

App commands

• Test the manage.py command with either docker compose exec app manage makemigrations on your host machine or manage makemigrations in the app container

[jamesbiggs]

Hi AJ, great work on this! - just a few small things

1. Database creation on docker-compose up -d
   I think we should avoid doing this (or add some way that we can turn it on/off) because the database generation takes quite a bit of time before the app runs. After that, I still need to pull the database, so the previous database would get removed(?) so it runs redundant code by doing this. Could there be a way that lets us pull a db before running the app? If not, it's not the end of the world, because it works fine after that!

2. upgrade and docker compose up -d --build app work great, but could we have upgrade-poetry and upgrade-node (or something similar) so that we don't need to do both at the same time?

3. I can create migrations, and edit them fine (may need a test from someone else though, I'm running WSL2 as root)
   I am however struggling to run manage startapp test which should create a new app folder in the project, but it keeps giving me this error, no matter what I try:

usage: manage.py startapp [-h] [--template TEMPLATE] [--extension EXTENSIONS] [--name FILES] [--exclude [EXCLUDE]] [--version]
                          [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color]
                          name [directory]
manage.py startapp: error: You must provide an application name.

5. Some API tests fail but that's because of the site set in the CMS. No issues there otherwise!

[ahosgood]

Awesome! Thank you for the feedback.

There is a new preview dev image (latest non-stable). You can pull it with:

docker pull ghcr.io/nationalarchives/tna-python-dev:preview

Then you can rebuild your dev container with it:

docker compose up --build -d dev

1. We need to run all the DB migrations on startup because that's what the image will do in production but this is the main reason I want to squash all the migrations when we move to AWS to decrease this load time. One option is to just start the dev container and pull everything before you start everything else:
   1. docker compose up -d dev
   2. [wait for dev image to be ready]
   3. docker compose exec dev pull
   4. docker compose up -d
2. In addition to upgrade, you should now have upgrade poetry and upgrade npm
3. I made a fix so the manage command can accept multiple arguments - previously manage startapp test would have just passed the startapp argument to the manage script
4. I fixed the site domain in this commit: 08882c0 - I didn't realise someone had changed it in the development environment! There should be a fix for this branch coming in a bit

[jamesbiggs]

Awesome work! 🚀 Thanks AJ!

[ahosgood]

@jamesbiggs - If you're happy that everything works, I need to do two more things before we merge:

1. Update the documentation
2. Run the linter to fix all the issues - I didn't want to do this before you'd reviewed it else you would have had a difficult time ascertaining which bits to actually check!

Are you happy for me to continue on with those?

[jamesbiggs]

Go for it! 😃