Simplify Docker Compose configuration

Intention is to make is as easy as possible to get started and provide
documentation for any optional configuration, and leave it up to developers to
manage their own configuration in override files that won't be committed.

README.md

@@ -5,144 +5,69 @@ Internet of Good Things ([IoGT][2]) is developed as a public good under a [BSD-2
 The development uses the Python programming language, building on the [Django][4] server-side web framework combined with the [Wagtail][5] content management system.
 
 In line with the latest Wagtail [Long Term Support release][6], IoGT 2.x supports:
-- Wagtail 2.11.x
-- Django 2.2.x, 3.0.x and 3.1.x
-- Python 3.8 and 3.9
+- Wagtail 2.15.x
+- Django 3.0.x, 3.1.x and 3.2.x
+- Python 3.8, 3.9 and 3.10
 - PostgreSQL, MySQL and SQLite as database backends
 
 ## Getting started
 
-If you'd like to contribute to IoGT development, the best way to get started is to clone a copy of this repository and run the code on your machine.
+The easiest way to get started is to run IoGT using Docker. Make sure you have Docker Desktop installed, or at the very least the Docker Engine and Docker Compose (which are included in Docker Desktop).
+
+Check out the code.
 ```
 git clone https://github.com/unicef/iogt.git
 cd iogt
 ```
 
-Create an [isolated Python environment][1], into which we will install the project's dependencies. This is highly recommended so that the requirements of this project don't interfere with any other Python environments you may have in your system, including the global Python environment. We also recommend that you create a new environment outside of the project directory.
+Create the database. By default, an SQLite database will be created. This will take a few minutes on the first run because the IoGT container image needs to be created from scratch.
 ```
-python3 -m venv <path_to_venv_dir>
-source <path_to_venv_dir>/bin/activate
-pip install -r requirements.txt
-```
-
-Create the database. By default, an SQLite database will be created.
-```
-./manage.py migrate
+docker compose run --rm django python manage.py migrate
 ```
 
 Create a superuser account for administration purposes.
 ```
-./manage.py createsuperuser
+docker compose run --rm django python manage.py createsuperuser
 ```
 
 Compile .po language files stored in locale/
 ```
-./manage.py compilemessages
+docker compose run --rm django python manage.py compilemessages
 ```
 
-Finally, start the server.
+Start the server.
 ```
-./manage.py runserver
+docker compose up -d
 ```
 
-Once running, navigate to http://localhost:8000 in your browser.
-
-
-### Running ElasticSearch (Optional)
-
-1. Set up an elastic search cluster
-2. Update local.py to use Elasticsearch as the backend. More details [here](https://docs.wagtail.io/en/stable/topics/search/backends.html#elasticsearch-backend)
-   ```
-   WAGTAILSEARCH_BACKENDS = {
-    'default': {
-        'BACKEND': 'wagtail.search.backends.elasticsearch7',
-        'URLS': ['http://localhost:9200'],
-        'INDEX': 'iogt',
-        'TIMEOUT': 5,
-        'OPTIONS': {},
-        'INDEX_SETTINGS': {},
-        'AUTO_UPDATE': True,
-        'ATOMIC_REBUILD': True
-        }
-   }
-   ```
-
-3. Run `./manage.py update_index` to update the ElasticSearch Index
-
-### Setting up test data
-
-It is possible to automatically populate the database with example data for a basic test site.
-```
-./manage.py create_initial_data
-```
-
-Optionally, create the main menu automatically as well.
+Check the logs.
 ```
-./manage.py autopopulate_main_menus
+docker compose logs -f
 ```
 
+Once running, navigate to http://localhost:8000/ in your browser. Log in to the admin UI at http://localhost:8000/admin/ - use the superuser name and password you created earlier.
 
-### Running Tests
+Shut everything down.
 ```
-./manage.py test --settings=iogt.settings.production
+docker compose down
 ```
 
-## Setup with Docker Compose
-
-You can choose to set up the project locally using Docker Compose. This setup is recommended if you want to replicate the production environment.
-
-### Steps
-
-Clone the repository
-
-```
-git clone https://github.com/unicef/iogt.git
-```
-Run setup command. This will set up the database and also update the ElasticSearch Index.
-```
-cd iogt
-make setup
-```
-
-Create a super user account for administration purposes.
-```
-docker-compose run django python manage.py createsuperuser
-```
-
-Run the compose file
-```
-make up
-```
-You're all set now. See the `Makefile` for other commands related to docker-compose
+# Optional
 
 ## Setting up test data
 
 It is possible to automatically populate the database with example data for a basic test site.
-```
-docker-compose run django python manage.py create_initial_data
-```
 
-Optionally, create the main menu automatically as well.
-```
-docker-compose run django python manage.py autopopulate_main_menus
+```sh
+docker compose run --rm django python manage.py create_initial_data
 ```
 
-## Running Tests
-
-Run the following commands:
-```
-make tests
-make selenium-test
-```
+Optionally, create the main menu automatically as well.
 
-In parallel, for example, with 4 processes:
-```
-IOGT_TEST_PARALLEL=4 make test
-IOGT_TEST_PARALLEL=4 make selenium-test
+```sh
+docker compose run --rm django python manage.py autopopulate_main_menus
 ```
 
-More details of the Selenium tests can be found in the [Selenium test README][9].
-
 ## Configuring the Chatbot
 
 Follow instructions [here](messaging/README.md)
@@ -182,12 +107,32 @@ Even though it has a column "is in use", its data is currently not updated autom
 
 Page caching is provided by Wagtail Cache and is deactivated by default. To start caching pages, follow the [activation intructions][7].
 
+## Elasticsearch
+
+To use Elasticsearch as a search backend instead of the relational database see the [setup instructions][11].
+
 ## Migrating content from IoGT v1
 Follow instructions [here](iogt_content_migration/README.md)
 
 ## Add/Update Package
 Follow instructions [here][10]
 
+# Running Tests
+
+Run the following commands:
+```
+make tests
+make selenium-test
+```
+
+In parallel, for example, with 4 processes:
+```
+IOGT_TEST_PARALLEL=4 make test
+IOGT_TEST_PARALLEL=4 make selenium-test
+```
+
+More details of the Selenium tests can be found in the [Selenium test README][9].
+
 
 [1]: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment
 [2]: https://www.unicef.org/innovation/IoGT
@@ -199,3 +144,4 @@ Follow instructions [here][10]
 [8]: ./docs/troubleshooting.md
 [9]: ./selenium_tests/README.md
 [10]: ./docs/dependency-management.md
+[11]: ./docs/elasticsearch.md

docker-compose-legacy-db.yml

@@ -2,7 +2,7 @@ version: '3.5'
 
 services:
   legacy-db:
-    image: postgres:alpine
+    image: postgres:9.6-alpine
     restart: unless-stopped
     environment:
       POSTGRES_PASSWORD: iogt

docker-compose.yml

@@ -1,70 +1,14 @@
-version: '3.5'
+version: '3'
 
 services:
-
   django:
     build:
       context: ./
       dockerfile: Dockerfile.dev
     environment:
-      DB_NAME: postgres
-      DB_USER: postgres
-      DB_PASSWORD: iogt
-      DB_HOST: database
-      DB_PORT: '5432'
-      DJANGO_SETTINGS_MODULE: iogt.settings.docker_compose_dev
-      COMMIT_HASH: asdfghjkl
-      WAGTAILTRANSFER_SECRET_KEY: wagtailtransfer-secret-key
-      WAGTAILTRANSFER_SOURCE_NAME: iogt_global
-      WAGTAILTRANSFER_SOURCE_BASE_URL: https://example.com/wagtail-transfer/
-      WAGTAILTRANSFER_SOURCE_SECRET_KEY: wagtailtransfer-source-secret-key
-      BASE_URL: 'http://localhost:8000'
-      VAPID_PUBLIC_KEY: ''
-      VAPID_PRIVATE_KEY: ''
-      VAPID_ADMIN_EMAIL: ''
-      COMMENTS_COMMUNITY_MODERATION: 'enable'
-      COMMENT_MODERATION_CLASS: 'comments.clients.AlwaysApproveModerator'
-      BLACKLISTED_WORDS: ''
-      SUPERSET_BASE_URL: ''
-      SUPERSET_DATABASE_NAME: ''
-      SUPERSET_USERNAME: ''
-      SUPERSET_PASSWORD: ''
-      PUSH_NOTIFICATION: 'enable'
-      JQUERY: 'enable'
-      MATOMO_TRACKING: 'enable'
-      MATOMO_SERVER_URL: ''
-      MATOMO_SITE_ID: ''
-      IMAGE_SIZE_PRESET: ''
+      DJANGO_SETTINGS_MODULE: iogt.settings.dev
     command: python manage.py runserver 0.0.0.0:8000
     volumes:
       - ./:/app/
     ports:
       - "8000:8000"
-    depends_on:
-      - elasticsearch
-      - database
-
-  elasticsearch:
-    image: 'docker.elastic.co/elasticsearch/elasticsearch:7.12.1'
-    environment:
-      - discovery.type=single-node
-    volumes:
-      - es-data01:/usr/share/elasticsearch/data
-    ports:
-      - "9200:9200"
-
-  database:
-    image: postgres:alpine
-    restart: unless-stopped
-    environment:
-      POSTGRES_PASSWORD: iogt
-    volumes:
-      - pg-data01:/var/lib/postgresql/data
-      - ./initdb.d:/docker-entrypoint-initdb.d
-    ports:
-      - 5432:5432
-
-
-volumes:
-  es-data01:
-  pg-data01:

docs/elasticsearch.md

@@ -0,0 +1,64 @@
+# Overview
+
+Wagtail supports the use of Elasticsearch as a search backend, to speed up searches and remove the burden of searching from the relational database.
+
+# Setup
+
+The outline of the setup process is as follows.
+
+1. Create an Elasticsearch cluster
+1. Configure Wagtail to use Elasticsearch
+1. Create the search index
+
+## Create Elasticsearch cluster
+
+Create the file `docker-compose.override.yml`. Add the following to the file.
+
+```yaml
+version: '3'
+services:
+  search:
+    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
+    environment:
+      - discovery.type=single-node
+    volumes:
+      - search:/usr/share/elasticsearch/data
+    ports:
+      - "9200:9200"
+volumes:
+  search:
+```
+
+If you already have a `docker-compose.override.yml`, you will need to merge the snippet above into it, in which case, you will probably not need the first two lines.
+
+## Configure Wagtail to use Elasticsearch
+
+Create the file `iogt/settings/local.py`. Add the following to the file.
+
+```python
+WAGTAILSEARCH_BACKENDS = {
+    'default': {
+        'BACKEND': 'wagtail.search.backends.elasticsearch7',
+        'URLS': ['http://search:9200'],
+        'INDEX': 'iogt',
+        'TIMEOUT': 5,
+        'OPTIONS': {},
+        'INDEX_SETTINGS': {},
+        'AUTO_UPDATE': True,
+        'ATOMIC_REBUILD': True
+    }
+}
+```
+
+More information about the available configuration options can be found in the [search backends] section of the Wagtail documentation.
+
+## Create the search index
+
+To create and subsequently update the ElasticSearch index.
+
+```sh
+docker compose run --rm django python manage.py update_index
+```
+
+
+[search backends]: https://docs.wagtail.org/en/v2.15.6/topics/search/backends.html

docs/envvars.md

@@ -0,0 +1,46 @@
+# Overview
+
+The IoGT app supports environment variables as a way of changing some frequently used configuration options.
+
+# Use with Docker
+
+Environment variables can be defined in `docker-compose.yml`, `docker-compose.override.yml` or any custom Docker Compose configuration file. Environment variables are specified for each service, for example.
+
+```yaml
+services:
+  django:
+    environment:
+      DB_HOST: db
+```
+
+# List of environment variables
+
+```yaml
+DB_NAME: postgres
+DB_USER: postgres
+DB_PASSWORD: iogt
+DB_HOST: db
+DB_PORT: '5432'
+COMMIT_HASH: asdfghjkl
+WAGTAILTRANSFER_SECRET_KEY: wagtailtransfer-secret-key
+WAGTAILTRANSFER_SOURCE_NAME: iogt_global
+WAGTAILTRANSFER_SOURCE_BASE_URL: https://example.com/wagtail-transfer/
+WAGTAILTRANSFER_SOURCE_SECRET_KEY: wagtailtransfer-source-secret-key
+VAPID_PUBLIC_KEY: ''
+VAPID_PRIVATE_KEY: ''
+VAPID_ADMIN_EMAIL: ''
+COMMENTS_COMMUNITY_MODERATION: enable
+COMMENT_MODERATION_CLASS: comments.clients.AlwaysApproveModerator
+BLACKLISTED_WORDS: ''
+SUPERSET_BASE_URL: ''
+SUPERSET_DATABASE_NAME: ''
+SUPERSET_USERNAME: ''
+SUPERSET_PASSWORD: ''
+PUSH_NOTIFICATION: enable
+JQUERY: enable
+MATOMO_TRACKING: enable
+MATOMO_URL: ''
+MATOMO_SITE_ID: ''
+IMAGE_SIZE_PRESET: ''
+```
+