Make network, volumes and names better manageable and more secure

[alexgarel]

Who for

Sys admin, docker developpers and CD script writers.

What

Currently

Our current docker deployment is based on docker-compose. See docs/cicd.md.
But in some prod deployment there are some fixed names:

• volumes names (eg for mongodb)
• network names (eg. webnet)

Also many things are on the same network (webnet) when they should not, as they do not need to talk to service external to their docker compose.

More over some services names clashes (eg. postgresql in off-server and robotoff).

What we should do

• isolate services that are internal to the project in default docker compose network (no need to have it private) and let docker compose manage the name
• every shared network name should be configurable from the .env
• no volume should have a static name… let docker-compose add it's prefix (we control COMPOSE_PROJECT_NAME in the .env)
  • although it's really ok to create external service for data to avoid accidental volumes removal and control their location
• avoid naming services with generic name, fix at least actual problems…

To be systematic:

• shared network name should have a prefix which reflect the environment: like stagging / prod
• COMPOSE_PROJECT_NAME should use <project_name>_: like po_stagging, po_prod, ...
• local network should be the docker-compose default one, that is be prefixed by compose project name
• volumes should also use the docker-compose default one

Why

We want to avoid any incident on production.

If we don't keep things well separated there is a big risk of merging two volumes without even noticing.

Also docker-compose is very permissive on service name and will simply round robin between services that have the same name. This leads to really nasty bugs and some time difficult to diagnose.
That's why we should keep separate networks.

Also in dev we want to be able to connect a lot of services if possible, but in a controlled way.

Actions

We have to look at currently deployed projects:

• openfoodfacts-server: off-net / mongo-dev /
• robotoff: robotoff-ml, robotoff-net, robotoff-ann-net, robotoff-org, robotoff-ann-org
• openfoodfacts-monitoring: filebeat / monitoring
• openfoodfacts-events: events-net (see also Deploy to prod openfoodfacts-events#151)
• openfoodfacts-search: off-search-net
• impactestimator: impactestimator-net, impactestimator-org (maybe just remove for now ?)

• rename webnet where it is used to use an environment prefix
• do not put every services in webnet but only those needed
• rename off-server postgresql to a more private name (to avoid clash with robotoff)
• ensure every volumes use a prefix
• switch compose_project_name to use environment name (beware volumes migration)