Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(framework:skip) Incorporate 1.12.0 changes/user feedback #4356

Merged
merged 4 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions doc/source/docker/run-quickstart-examples-docker-compose.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ Run the Quickstart Example
into the example directory:

.. code-block:: bash
:substitutions:

$ curl https://raw.githubusercontent.com/adap/flower/refs/heads/main/src/docker/complete/compose.yml \
$ curl https://raw.githubusercontent.com/adap/flower/refs/tags/v|stable_flwr_version|/src/docker/complete/compose.yml \
-o compose.yml

3. Build and start the services using the following command:
Expand All @@ -65,17 +66,11 @@ Run the Quickstart Example
``local-deployment`` with your chosen name in both the ``tool.flwr.federations.``
string and the corresponding ``flwr run .`` command.

5. Run the example:
5. Run the example and follow the logs of the ServerApp:

.. code-block:: bash

$ flwr run . local-deployment

6. Follow the logs of the SuperExec service:

.. code-block:: bash

$ docker compose logs superexec -f
$ flwr run . local-deployment --stream

That is all it takes! You can monitor the progress of the run through the logs of the
SuperExec.
Expand Down
45 changes: 30 additions & 15 deletions doc/source/docker/tutorial-deploy-on-multiple-machines.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,14 @@ Step 1: Set Up

.. code-block:: bash

$ docker compose -f certs.yml -f ../complete/certs.yml up --build
$ docker compose -f certs.yml -f ../complete/certs.yml run --rm --build gen-certs

Step 2: Copy the Server Compose Files
-------------------------------------

Use the method that works best for you to copy the ``server`` directory, the
certificates, and your Flower project to the remote machine.
certificates, and the ``pyproject.toml`` file of your Flower project to the remote
machine.

For example, you can use ``scp`` to copy the directories:

Expand All @@ -81,7 +82,7 @@ For example, you can use ``scp`` to copy the directories:
$ scp -r ./server \
./superexec-certificates \
./superlink-certificates \
../../../examples/quickstart-sklearn-tabular remote:~/distributed
../../../examples/quickstart-sklearn-tabular/pyproject.toml remote:~/distributed

Step 3: Start the Flower Server Components
------------------------------------------
Expand All @@ -90,17 +91,31 @@ Log into the remote machine using ``ssh`` and run the following command to start
SuperLink and SuperExec services:

.. code-block:: bash
:linenos:

$ ssh <your-remote-machine>
# In your remote machine
$ cd <path-to-``distributed``-directory>
$ export PROJECT_DIR=../quickstart-sklearn-tabular
$ docker compose -f server/compose.yml up --build -d
$ ssh <your-remote-machine>
# In your remote machine
$ cd <path-to-``distributed``-directory>
$ export PROJECT_DIR=../
$ docker compose -f server/compose.yml up --build -d

.. note::

The path to the ``PROJECT_DIR`` containing the ``pyproject.toml`` file should be
relative to the location of the server ``compose.yml`` file.

.. note::

The Path of the ``PROJECT_DIR`` should be relative to the location of the ``server``
Docker Compose files.
When working with Docker Compose on Linux, you may need to create the ``state``
directory first and change its ownership to ensure proper access and permissions.
After exporting the ``PROJECT_DIR`` (after line 4), run the following commands:

.. code-block:: bash

$ mkdir server/state
$ sudo chown -R 49999:49999 server/state

For more information, consult the following page: :doc:`persist-superlink-state`.

Go back to your terminal on your local machine.

Expand All @@ -117,8 +132,8 @@ On your local machine, run the following command to start the client components:

.. note::

The Path of the ``PROJECT_DIR`` should be relative to the location of the ``client``
Docker Compose files.
The path to the ``PROJECT_DIR`` containing the ``pyproject.toml`` file should be
relative to the location of the client ``compose.yml`` file.

Step 5: Run Your Flower Project
-------------------------------
Expand All @@ -136,14 +151,14 @@ we have named our remote federation ``remote-superexec``:

.. note::

The Path of the ``root-certificates`` should be relative to the location of the
The path of the ``root-certificates`` should be relative to the location of the
``pyproject.toml`` file.

To run the project, execute:
Run the project and follow the ServerApp logs:

.. code-block:: bash

$ flwr run ../../../examples/quickstart-sklearn-tabular remote-superexec
$ flwr run ../../../examples/quickstart-sklearn-tabular remote-superexec --stream

That's it! With these steps, you've set up Flower on two separate machines and are ready
to start using it.
Expand Down
56 changes: 23 additions & 33 deletions doc/source/docker/tutorial-quickstart-docker-compose.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Before you start, make sure that:

- The ``flwr`` CLI is :doc:`installed <../how-to-install-flower>` locally.
- The Docker daemon is running.
- Docker Compose is `installed <https://docs.docker.com/compose/install/>`_.
- Docker Compose V2 is `installed <https://docs.docker.com/compose/install/>`_.

Step 1: Set Up
--------------
Expand Down Expand Up @@ -63,12 +63,11 @@ Open your terminal and run:

.. code-block:: bash

$ docker compose -f compose.yml up --build -d
$ docker compose up --build -d

.. dropdown:: Understand the command

* ``docker compose``: The Docker command to run the Docker Compose tool.
* ``-f compose.yml``: Specify the YAML file that contains the basic Flower service definitions.
* ``--build``: Rebuild the images for each service if they don't already exist.
* ``-d``: Detach the containers from the terminal and run them in the background.

Expand All @@ -86,21 +85,16 @@ addresses in the ``pyproject.toml`` file.
.. code-block:: toml
:caption: quickstart-compose/pyproject.toml

[tool.flwr.federations.docker-compose]
[tool.flwr.federations.local-deployment]
address = "127.0.0.1:9093"
insecure = true

2. Execute the command to run the quickstart example:
2. Run the quickstart example, monitor the ServerApp logs and wait for the summary to
appear:

.. code-block:: bash

$ flwr run quickstart-compose docker-compose

3. Monitor the SuperExec logs and wait for the summary to appear:

.. code-block:: bash

$ docker compose logs superexec -f
$ flwr run quickstart-compose local-deployment --stream

Step 4: Update the Application
------------------------------
Expand Down Expand Up @@ -135,30 +129,29 @@ In the next step, change the application code.

.. code-block:: bash

$ docker compose -f compose.yml up --build -d
$ docker compose up --build -d

3. Run the updated quickstart example:

.. code-block:: bash

$ flwr run quickstart-compose docker-compose
$ docker compose logs superexec -f
$ flwr run quickstart-compose local-deployment --stream

In the SuperExec logs, you should find the ``Get weights`` line:

.. code-block::
:emphasize-lines: 9

superexec-1 | INFO : Starting Flower SuperExec
superexec-1 | WARNING : Option `--insecure` was set. Starting insecure HTTP server.
superexec-1 | INFO : Starting Flower SuperExec gRPC server on 0.0.0.0:9093
superexec-1 | INFO : ExecServicer.StartRun
superexec-1 | 🎊 Successfully installed quickstart-compose to /app/.flwr/apps/flower/quickstart-compose/1.0.0.
superexec-1 | INFO : Created run -6767165609169293507
superexec-1 | INFO : Started run -6767165609169293507
superexec-1 | WARNING : Option `--insecure` was set. Starting insecure HTTP client connected to superlink:9091.
superexec-1 | Get weights
superexec-1 | INFO : Starting Flower ServerApp, config: num_rounds=3, no round_timeout
INFO : Starting Flower SuperExec
WARNING : Option `--insecure` was set. Starting insecure HTTP server.
INFO : Starting Flower SuperExec gRPC server on 0.0.0.0:9093
INFO : ExecServicer.StartRun
🎊 Successfully installed quickstart-compose to /app/.flwr/apps/flower/quickstart-compose/1.0.0.
INFO : Created run -6767165609169293507
INFO : Started run -6767165609169293507
WARNING : Option `--insecure` was set. Starting insecure HTTP client connected to superlink:9091.
Get weights
INFO : Starting Flower ServerApp, config: num_rounds=3, no round_timeout

Step 5: Persisting the SuperLink State
--------------------------------------
Expand Down Expand Up @@ -194,7 +187,7 @@ service, ensuring that it maintains its state even after a restart.

.. code-block:: bash

$ flwr run quickstart-compose docker-compose
$ flwr run quickstart-compose local-deployment --stream

3. Check the content of the ``state`` directory:

Expand Down Expand Up @@ -227,14 +220,14 @@ Step 6: Run Flower with TLS

.. code-block:: bash

$ docker compose -f certs.yml up --build
$ docker compose -f certs.yml run --rm --build gen-certs

2. Add the following lines to the ``quickstart-compose/pyproject.toml``:

.. code-block:: toml
:caption: quickstart-compose/pyproject.toml

[tool.flwr.federations.docker-compose-tls]
[tool.flwr.federations.local-deployment-tls]
address = "127.0.0.1:9093"
root-certificates = "../superexec-certificates/ca.crt"

Expand All @@ -248,8 +241,7 @@ Step 6: Run Flower with TLS

.. code-block:: bash

$ flwr run quickstart-compose docker-compose-tls
$ docker compose logs superexec -f
$ flwr run quickstart-compose local-deployment-tls --stream

Step 7: Add another SuperNode
-----------------------------
Expand Down Expand Up @@ -376,8 +368,7 @@ To run Flower with persisted SuperLink state and enabled TLS, a slight change in

.. code-block:: bash

$ flwr run quickstart-compose docker-compose-tls
$ docker compose logs superexec -f
$ flwr run quickstart-compose local-deployment-tls --stream

Step 9: Merge Multiple Compose Files
------------------------------------
Expand All @@ -402,7 +393,6 @@ Remove all services and volumes:
.. code-block:: bash

$ docker compose down -v
$ docker compose -f certs.yml down -v

Where to Go Next
----------------
Expand Down
16 changes: 5 additions & 11 deletions doc/source/docker/tutorial-quickstart-docker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ Step 1: Set Up
flwr run

$ cd quickstart-docker
$ pip install -e .

2. Create a new Docker bridge network called ``flwr-network``:

Expand Down Expand Up @@ -324,21 +323,16 @@ Step 6: Run the Quickstart Project
.. code-block:: toml
:caption: pyproject.toml

[tool.flwr.federations.docker]
[tool.flwr.federations.local-deployment]
address = "127.0.0.1:9093"
insecure = true

2. Run the ``quickstart-docker`` project by executing the command:
2. Run the ``quickstart-docker`` project and follow the ServerApp logs to track the
execution of the run:

.. code-block:: bash

$ flwr run . docker

3. Follow the SuperExec logs to track the execution of the run:

.. code-block:: bash

$ docker logs -f superexec
$ flwr run . local-deployment --stream

Step 7: Update the Application
------------------------------
Expand Down Expand Up @@ -384,7 +378,7 @@ Step 7: Update the Application

.. code-block:: bash

$ flwr run . docker
$ flwr run . local-deployment --stream

Step 8: Clean Up
----------------
Expand Down