Skip to content

Commit

Permalink
docker: podman-friendly image dependencies
Browse files Browse the repository at this point in the history
Adds container image tags for each dependency, avoiding the use of
"latest" tag, for better dependency management.

Adds fully qualified canonical locations of all dependent container
images, making the installation and the development processes
podman-friendly.

Enriches DEVELOPING guide accordingly.
  • Loading branch information
tiborsimko committed Jul 7, 2023
1 parent 24cf794 commit fad50aa
Show file tree
Hide file tree
Showing 7 changed files with 69 additions and 28 deletions.
65 changes: 53 additions & 12 deletions DEVELOPING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,34 +9,75 @@ Installation
============

You can run a local CERN Open Data instance for development purposes using
Docker with ``docker-compose-dev.yml`` configuration. The source code directory
will be mounted in the container and the system will be ready for "live
editing". This is useful for active feature development or for pull request
integration purposes. A usage example:
container technology such as Docker or Podman.

Install with Docker
-------------------

One popular solution to develop the CERN Open Data instance locally is to use
Docker. For development purposes, please use the ``docker-compose-dev.yml``
configuration. The source code directory will be mounted in the container and
the system will be ready for "live editing". This is useful for active feature
development or for pull request integration purposes. A usage example:

.. code-block:: console
$ ./scripts/generate-localhost-certificate.sh
$ docker compose -f docker-compose-dev.yml build
$ docker compose -f docker-compose-dev.yml up
$ docker exec -i -t opendatacernch-web-1 /code/scripts/populate-instance.sh --skip-files
$ docker compose -f docker-compose-dev.yml up -d
$ docker exec -i -t opendatacernch-web-1 /code/scripts/populate-instance.sh
$ firefox http://0.0.0.0:5000/
$ docker compose -f docker-compose-dev.yml down
$ docker compose -f docker-compose-dev.yml down -v
If you want to use production-like conditions locally, you can use Docker with
``docker-compose.yml`` configuration. This is useful for tuning overall system
performance such as reverse proxy caching. The source code directory will not be
mounted in the container in this case. A usage example:
If you want to simulate production-like deployment conditions locally, please
use the ``docker-compose.yml`` configuration. This is useful for tuning overall
system performance such as reverse proxy caching. The source code directory
will not be mounted in the container in this case. A usage example:

.. code-block:: console
$ ./scripts/generate-localhost-certificate.sh
$ docker compose build
$ docker compose up
$ docker compose up -d
$ docker exec -i -t opendatacernch-web-1 /code/scripts/populate-instance.sh
$ firefox http://0.0.0.0/
$ docker compose down -v
Install with Podman
-------------------

Another possibility to develop the CERN Open Data instance locally is to use
the Podman container technology. This has an advantage that your containers
will be running in the regular user space, not requiring any superuser access.

An example of a Podman development session:

.. code-block:: console
$ ./scripts/generate-localhost-certificate.sh
$ podman-compose -f docker-compose-dev.yml build
$ podman-compose -f docker-compose-dev.yml up
$ podman exec -i -t opendatacernch_web_1 \
./scripts/populate-instance.sh --skip-docs --skip-glossary --skip-records
$ podman exec -i -t opendatacernch_web_1 \
cernopendata fixtures records --mode insert -f cernopendata/modules/fixtures/data/records/cms-primary-datasets.json
$ podman-compose -f docker-compose-dev.yml down -v
Note that if you would like to test production-like conditions with Podman, you
will have to allow the regular user processes to listen to privileged
HTTP/HTTPS ports, for example by allowing all ports from 80 up:

.. code-block:: console
$ echo 80 | sudo tee /proc/sys/net/ipv4/ip_unprivileged_port_start
Then, when you are done with the testing, you can return back to the default
operating system configuration allowing only ports 1024 and up:

.. code-block:: console
$ echo 1024 | sudo tee /proc/sys/net/ipv4/ip_unprivileged_port_start
Development tips
================

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
# as an Intergovernmental Organization or submit itself to any jurisdiction.

# Use CentOS7
FROM centos:7
FROM docker.io/library/centos:7

# Install Node.js 6 from Nodesource early. Doing so after installing EPEL7
# would make Nodesource to not recognise anymore the system as a supported
Expand Down
10 changes: 5 additions & 5 deletions docker-compose-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ services:
# wdb.set_trace()
# ```
wdb:
image: kozea/wdb-server
image: docker.io/kozea/wdb:3.3.0
ports:
- "1984:1984"

Expand Down Expand Up @@ -151,7 +151,7 @@ services:

postgresql:
restart: "unless-stopped"
image: postgres:14.8
image: docker.io/library/postgres:14.8
environment:
- POSTGRES_USER=cernopendata
- POSTGRES_DB=cernopendata
Expand All @@ -161,13 +161,13 @@ services:

redis:
restart: "unless-stopped"
image: redis
image: docker.io/library/redis:7
ports:
- "6379:6379"

elasticsearch:
restart: "unless-stopped"
image: elasticsearch:5.6.16
image: docker.io/library/elasticsearch:5.6.16
# Uncomment if DEBUG logging needs to enabled for Elasticsearch
# command: ["elasticsearch", "-Elogger.level=DEBUG"]
environment:
Expand All @@ -180,7 +180,7 @@ services:

rabbitmq:
restart: "unless-stopped"
image: rabbitmq
image: docker.io/library/rabbitmq:3.12
ports:
- "4369:4369"
- "5672:5672"
Expand Down
10 changes: 5 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ services:

postgresql:
restart: "always"
image: postgres:14.8
image: docker.io/library/postgres:14.8
environment:
- POSTGRES_USER=cernopendata
- POSTGRES_DB=cernopendata
Expand All @@ -158,13 +158,13 @@ services:

redis:
restart: "always"
image: redis
image: docker.io/library/redis:7
ports:
- "6379"

elasticsearch:
restart: "always"
image: elasticsearch:5.6.16
image: docker.io/library/elasticsearch:5.6.16
command: ["elasticsearch", "-E", "logger.org.elasticsearch.deprecation=error"]
environment:
- bootstrap.memory_lock=true
Expand All @@ -188,7 +188,7 @@ services:

rabbitmq:
restart: "always"
image: rabbitmq
image: docker.io/library/rabbitmq:3.12
depends_on:
- redis
ports:
Expand All @@ -198,7 +198,7 @@ services:
nginx:
restart: "always"
build: ./nginx
image: cernopendata/nginx
image: cernopendata/nginx:1.22
ports:
- "80:80"
volumes:
Expand Down
4 changes: 2 additions & 2 deletions elasticsearch-proxy/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# This file is part of CERN Open Data Portal.
# Copyright (C) 2017, 2022 CERN.
# Copyright (C) 2017, 2022, 2023 CERN.
#
# CERN Open Data Portal is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
Expand All @@ -22,7 +22,7 @@
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.

FROM nginx:1.22
FROM docker.io/library/nginx:1.22

RUN rm /etc/nginx/conf.d/default.conf
ADD nginx.conf /etc/nginx/conf.d/
Expand Down
4 changes: 2 additions & 2 deletions nginx/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# This file is part of CERN Open Data Portal.
# Copyright (C) 2016, 2022 CERN.
# Copyright (C) 2016, 2022, 2023 CERN.
#
# CERN Open Data Portal is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License as
Expand All @@ -22,7 +22,7 @@
# waive the privileges and immunities granted to it by virtue of its status
# as an Intergovernmental Organization or submit itself to any jurisdiction.

FROM nginx:1.22-perl
FROM docker.io/library/nginx:1.22-perl

RUN rm /etc/nginx/conf.d/default.conf
ADD cernopendata.conf /etc/nginx/conf.d/
Expand Down
2 changes: 1 addition & 1 deletion sentry/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM sentry:8.22
FROM docker.io/library/sentry:8.22

RUN apt-get update && \
apt-get install wget -y
Expand Down

0 comments on commit fad50aa

Please sign in to comment.