docs(*:skip) Update Docker documentation (#3722)

Signed-off-by: Robert Steiner <[email protected]>

doc/source/conf.py

@@ -94,6 +94,10 @@
 # The current released version
 rst_prolog = """
 .. |stable_flwr_version| replace:: 1.10.0
+.. |stable_flwr_superlink_docker_digest| replace:: 4b317d5b6030710b476f4dbfab2c3a33021ad40a0fcfa54d7edd45e0c51d889c
+.. |ubuntu_version| replace:: 22.04
+.. |setuptools_version| replace:: 70.3.0
+.. |pip_version| replace:: 24.1.2
 """
 
 # -- General configuration ---------------------------------------------------

doc/source/contributor-how-to-build-docker-images.rst

@@ -1,4 +1,4 @@
-How to build Docker Flower images locally
+How to Build Docker Flower Images Locally
 =========================================
 
 Flower provides pre-made docker images on `Docker Hub <https://hub.docker.com/u/flwr>`_
@@ -9,27 +9,22 @@ images exist and how to build them locally.
 
 Before we can start, we need to meet a few prerequisites in our local development environment.
 
-#. Clone the flower repository.
+#. Clone the ``flower`` repository.
 
     .. code-block:: bash
 
-      $ git clone https://github.com/adap/flower.git && cd flower
+      $ git clone --depth=1 https://github.com/adap/flower.git && cd flower
 
 #. Verify the Docker daemon is running.
 
-    Please follow the first section on
-    :doc:`Run Flower using Docker <how-to-run-flower-using-docker>`
-    which covers this step in more detail.
+   The build instructions that assemble the images are located in the respective Dockerfiles. You
+   can find them in the subdirectories of ``src/docker``.
 
-
-The build instructions that assemble the images are located in the respective Dockerfiles. You
-can find them in the subdirectories of ``src/docker``.
-
-Flower Docker images are configured via build arguments. Through build arguments, we can make the
-creation of images more flexible. For example, in the base image, we can specify the version of
-Python to install using the ``PYTHON_VERSION`` build argument. Some of the build arguments have
-default values, others must be specified when building the image. All available build arguments for
-each image are listed in one of the tables below.
+   Flower Docker images are configured via build arguments. Through build arguments, we can make the
+   creation of images more flexible. For example, in the base image, we can specify the version of
+   Python to install using the ``PYTHON_VERSION`` build argument. Some of the build arguments have
+   default values, others must be specified when building the image. All available build arguments for
+   each image are listed in one of the tables below.
 
 Building the base image
 -----------------------
@@ -49,19 +44,19 @@ Building the base image
    * - ``DISTRO_VERSION``
      - Version of the Linux distribution.
      - No
-     - ``22.04``
+     - :substitution-code:`|ubuntu_version|`
    * - ``PYTHON_VERSION``
      - Version of ``python`` to be installed.
      - No
      - ``3.11`` or ``3.11.1``
    * - ``PIP_VERSION``
      - Version of ``pip`` to be installed.
      - Yes
-     - ``23.0.1``
+     - :substitution-code:`|pip_version|`
    * - ``SETUPTOOLS_VERSION``
      - Version of ``setuptools`` to be installed.
      - Yes
-     - ``69.0.2``
+     - :substitution-code:`|setuptools_version|`
    * - ``FLWR_VERSION``
      - Version of Flower to be installed.
      - Yes
@@ -71,9 +66,9 @@ Building the base image
      - No
      - ``flwr`` or ``flwr-nightly``
 
-
-The following example creates a base Ubuntu/Alpine image with Python 3.11.0, pip 23.0.1,
-setuptools 69.0.2 and Flower |stable_flwr_version|:
+The following example creates a base Ubuntu/Alpine image with Python ``3.11.0``,
+pip :substitution-code:`|pip_version|`, setuptools :substitution-code:`|setuptools_version|`
+and Flower :substitution-code:`|stable_flwr_version|`:
 
 .. code-block:: bash
    :substitutions:
@@ -82,11 +77,11 @@ setuptools 69.0.2 and Flower |stable_flwr_version|:
    $ docker build \
      --build-arg PYTHON_VERSION=3.11.0 \
      --build-arg FLWR_VERSION=|stable_flwr_version| \
-     --build-arg PIP_VERSION=23.0.1 \
-     --build-arg SETUPTOOLS_VERSION=69.0.2 \
+     --build-arg PIP_VERSION=|pip_version| \
+     --build-arg SETUPTOOLS_VERSION=|setuptools_version| \
      -t flwr_base:0.1.0 .
 
-The name of image is ``flwr_base`` and the tag ``0.1.0``. Remember that the build arguments as well
+In this example, we specify our image name as ``flwr_base`` and the tag as ``0.1.0``. Remember that the build arguments as well
 as the name and tag can be adapted to your needs. These values serve as examples only.
 
 Building the SuperLink/SuperNode or ServerApp image
@@ -107,32 +102,31 @@ Building the SuperLink/SuperNode or ServerApp image
    * - ``BASE_IMAGE``
      - The Tag of the Flower base image.
      - Yes
-     - :substitution-code:`|stable_flwr_version|-py3.10-ubuntu22.04`
+     - :substitution-code:`|stable_flwr_version|-py3.11-ubuntu|ubuntu_version|`
 
-The following example creates a SuperLink/SuperNode or ServerApp image with the official Flower
-base image:
+For example, to build a SuperLink image with the latest Flower version, Python 3.11 and Ubuntu 22.04, run the following:
 
 .. code-block:: bash
+   :substitutions:
 
-  $ cd src/docker/<superlink|supernode|serverapp>/
-  $ docker build \
-    --build-arg BASE_IMAGE=<FLOWER-VERSION>-py<PY-VERSION>-<DISTRIBUTION and VERSION> \
-    -t flwr_superlink:0.1.0 .
-
+   $ cd src/docker/superlink
+   $ docker build \
+     --build-arg BASE_IMAGE=|stable_flwr_version|-py3.11-ubuntu22.04 \
+     -t flwr_superlink:0.1.0 .
 
 If you want to use your own base image instead of the official Flower base image, all you need to do
-is set the ``BASE_REPOSITORY`` build argument.
+is set the ``BASE_REPOSITORY`` build argument to ``flwr_base`` (as we've specified above).
 
 .. code-block:: bash
 
-  $ cd src/docker/superlink/
-  $ docker build \
-    --build-arg BASE_REPOSITORY=flwr_base \
-    --build-arg BASE_IMAGE=0.1.0
-    -t flwr_superlink:0.1.0 .
+   $ cd src/docker/superlink/
+   $ docker build \
+     --build-arg BASE_REPOSITORY=flwr_base \
+     --build-arg BASE_IMAGE=0.1.0
+     -t flwr_superlink:0.1.0 .
 
 After creating the image, we can test whether the image is working:
 
 .. code-block:: bash
 
-  $ docker run --rm flwr_superlink:0.1.0 --help
+   $ docker run --rm flwr_superlink:0.1.0 --help

doc/source/docker/enable-tls.rst

@@ -0,0 +1,152 @@
+Enable TLS for Secure Connections
+=================================
+
+When operating in a production environment, it is strongly recommended to enable Transport Layer
+Security (TLS) for each Flower Component to ensure secure communication.
+
+To enable TLS, you will need a PEM-encoded root certificate, a PEM-encoded private key and a
+PEM-encoded certificate chain.
+
+.. note::
+
+  For testing purposes, you can generate your own self-signed certificates. The
+  `Enable SSL connections <https://flower.ai/docs/framework/how-to-enable-ssl-connections.html#certificates>`__
+  page contains a section that will guide you through the process.
+
+
+Because Flower containers, by default, run with a non-root user ``app``, the mounted files and
+directories must have the proper permissions for the user ID ``49999``.
+
+For example, to change the user ID of all files in the ``certificates/`` directory, you can run
+``sudo chown -R 49999:49999 certificates/*``.
+
+If you later want to delete the directory, you can change the user ID back to the current user
+ID by running ``sudo chown -R $USER:$(id -gn) state``.
+
+SuperLink
+---------
+
+Assuming all files we need are in the local ``certificates`` directory, we can use the flag
+``--volume`` to mount the local directory into the ``/app/certificates/`` directory of the container:
+
+.. code-block:: bash
+   :substitutions:
+
+   $ docker run --rm \
+        --volume ./certificates/:/app/certificates/:ro \
+        flwr/superlink:|stable_flwr_version| \
+        --ssl-ca-certfile certificates/ca.crt \
+        --ssl-certfile certificates/server.pem \
+        --ssl-keyfile certificates/server.key
+
+.. dropdown:: Understanding the command
+
+   * ``docker run``: This tells Docker to run a container from an image.
+   * ``--rm``: Remove the container once it is stopped or the command exits.
+   * | ``--volume ./certificates/:/app/certificates/:ro``: Mount the ``certificates`` directory in
+     | the current working directory of the host machine as a read-only volume at the
+     | ``/app/certificates`` directory inside the container.
+     |
+     | This allows the container to access the TLS certificates that are stored in the certificates
+     | directory.
+   * | :substitution-code:`flwr/superlink:|stable_flwr_version|`: The name of the image to be run and the specific
+     | tag of the image. The tag :substitution-code:`|stable_flwr_version|` represents a specific version of the image.
+   * | ``--ssl-ca-certfile certificates/ca.crt``: Specify the location of the CA certificate file
+     | inside the container.
+     |
+     | The ``certificates/ca.crt`` file is a certificate that is used to verify the identity of the
+     | SuperLink.
+   * | ``--ssl-certfile certificates/server.pem``: Specify the location of the SuperLink's
+     | TLS certificate file inside the container.
+     |
+     | The ``certificates/server.pem`` file is used to identify the SuperLink and to encrypt the
+     | data that is transmitted over the network.
+   * | ``--ssl-keyfile certificates/server.key``: Specify the location of the SuperLink's
+     | TLS private key file inside the container.
+     |
+     | The ``certificates/server.key`` file is used to decrypt the data that is transmitted over
+     | the network.
+
+SuperNode
+---------
+
+Assuming that the ``ca.crt`` certificate already exists locally, we can use the flag ``--volume`` to mount the local
+certificate into the container's ``/app/`` directory.
+
+.. note::
+
+   If you're generating self-signed certificates and the ``ca.crt`` certificate doesn't exist
+   on the SuperNode, you can copy it over after the generation step.
+
+.. code-block:: bash
+   :substitutions:
+
+   $ docker run --rm \
+        --volume ./ca.crt:/app/ca.crt/:ro \
+        flwr/supernode:|stable_flwr_version| \
+        --root-certificates ca.crt
+
+.. dropdown:: Understanding the command
+
+   * ``docker run``: This tells Docker to run a container from an image.
+   * ``--rm``: Remove the container once it is stopped or the command exits.
+   * | ``--volume ./ca.crt:/app/ca.crt/:ro``: Mount the ``ca.crt`` file from the
+     | current working directory of the host machine as a read-only volume at the ``/app/ca.crt``
+     | directory inside the container.
+   * | :substitution-code:`flwr/supernode:|stable_flwr_version|`: The name of the image to be run and the specific
+     | tag of the image. The tag :substitution-code:`|stable_flwr_version|` represents a specific version of the image.
+   * | ``--root-certificates ca.crt``: This specifies the location of the CA certificate file
+     | inside the container.
+     |
+     | The ``ca.crt`` file is used to verify the identity of the SuperLink.
+
+
+SuperExec
+---------
+
+Assuming all files we need are in the local ``certificates`` directory where the SuperExec will be executed from, we can use the flag
+``--volume`` to mount the local directory into the ``/app/certificates/`` directory of the container:
+
+.. code-block:: bash
+   :substitutions:
+
+   $ docker run --rm \
+        --volume ./certificates/:/app/certificates/:ro \
+        flwr/superexec:|stable_flwr_version| \
+        --ssl-ca-certfile certificates/ca.crt \
+        --ssl-certfile certificates/server.pem \
+        --ssl-keyfile certificates/server.key \
+        --executor-config \
+        root-certificates=\"certificates/superlink_ca.crt\"
+
+
+.. dropdown:: Understanding the command
+
+   * ``docker run``: This tells Docker to run a container from an image.
+   * ``--rm``: Remove the container once it is stopped or the command exits.
+   * | ``--volume ./certificates/:/app/certificates/:ro``: Mount the ``certificates`` directory in
+     | the current working directory of the host machine as a read-only volume at the
+     | ``/app/certificates`` directory inside the container.
+     |
+     | This allows the container to access the TLS certificates that are stored in the certificates
+     | directory.
+   * | :substitution-code:`flwr/superexec:|stable_flwr_version|`: The name of the image to be run and the specific
+     | tag of the image. The tag :substitution-code:`|stable_flwr_version|` represents a specific version of the image.
+   * | ``--ssl-ca-certfile certificates/ca.crt``: Specify the location of the CA certificate file
+     | inside the container.
+     |
+     | The ``certificates/ca.crt`` file is a certificate that is used to verify the identity of the
+     | SuperExec.
+   * | ``--ssl-certfile certificates/server.pem``: Specify the location of the SuperExec's
+     | TLS certificate file inside the container.
+     |
+     | The ``certificates/server.pem`` file is used to identify the SuperExec and to encrypt the
+     | data that is transmitted over the network.
+   * | ``--ssl-keyfile certificates/server.key``: Specify the location of the SuperExec's
+     | TLS private key file inside the container.
+     |
+     | The ``certificates/server.key`` file is used to decrypt the data that is transmitted over
+     | the network.
+   * | ``--executor-config root-certificates=\"certificates/superlink_ca.crt\"``: Specify the
+     | location of the CA certificate file inside the container that the SuperExec executor
+     | should use to verify the SuperLink's identity.

doc/source/docker/index.rst

@@ -0,0 +1,37 @@
+Run Flower using Docker
+=======================
+
+Start your Flower journey with our pre-made Docker images on Docker Hub, supporting ``amd64``
+and ``arm64v8`` architectures.
+
+Our Quickstart guide walks you through containerizing a Flower project and running it end to
+end using Docker.
+
+Getting Started
+---------------
+
+.. toctree::
+   :maxdepth: 1
+
+   tutorial-quickstart-docker
+
+
+Running in Production
+---------------------
+
+.. toctree::
+   :maxdepth: 1
+
+   enable-tls
+   persist-superlink-state
+
+Advanced Options
+----------------
+
+.. toctree::
+   :maxdepth: 1
+
+   set-environment-variables
+   run-as-root-user
+   pin-version
+   use-a-different-version

doc/source/docker/persist-superlink-state.rst

@@ -0,0 +1,39 @@
+Persist the State of the SuperLink
+==================================
+
+By default, the Flower SuperLink keeps its state in-memory. When using the Docker flag ``--rm``, the
+state is not persisted between container starts.
+
+If you want to persist the state of the SuperLink on your host system, all you need to do is specify
+a directory where you want to save the file on your host system and a name for the database file.
+
+By default, the SuperLink container runs with a non-root user called ``app`` with the user ID
+``49999``. It is recommended to create a new directory and change the user ID of the directory to
+``49999`` to ensure the mounted directory has the proper permissions.
+
+If you later want to delete the directory, you can change the user ID back to the current user
+ID by running ``sudo chown -R $USER:$(id -gn) state``.
+
+Example
+-------
+
+In the example below, we create a new directory called ``state``, change the user ID and tell
+Docker via the flag ``--volume`` to mount the local ``state`` directory into the ``/app/state``
+directory of the container. Lastly, we use the flag ``--database`` to specify the name of the
+database file.
+
+.. code-block:: bash
+   :substitutions:
+
+   $ mkdir state
+   $ sudo chown -R 49999:49999 state
+   $ docker run --rm \
+        --volume ./state/:/app/state flwr/superlink:|stable_flwr_version| \
+        --database state.db \
+        ...
+
+As soon as the SuperLink starts, the file ``state.db`` is created in the ``state`` directory on
+your host system. If the file already exists, the SuperLink tries to restore the state from the
+file. To start the SuperLink with an empty database, ensure that there is no database
+called ``state.db`` in the ``state`` directory (``rm state.db``) before you execute the
+``docker run`` command above.