Merge branch 'main' into semantic-partitioner

adap · Jun 20, 2024 · 54ac47f · 54ac47f
2 parents da2f047 + e2d5163
commit 54ac47f
Show file tree

Hide file tree

Showing 132 changed files with 836 additions and 330 deletions.
diff --git a/.github/workflows/framework.yml b/.github/workflows/framework.yml
@@ -31,6 +31,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
       - name: Bootstrap
         uses: ./.github/actions/bootstrap
         with:

diff --git a/README.md b/README.md
@@ -153,6 +153,7 @@ Other [examples](https://github.com/adap/flower/tree/main/examples):
 - [Flower with KaplanMeierFitter from the lifelines library](https://github.com/adap/flower/tree/main/examples/federated-kaplan-meier-fitter)
 - [Sample Level Privacy with Opacus](https://github.com/adap/flower/tree/main/examples/opacus)
 - [Sample Level Privacy with TensorFlow-Privacy](https://github.com/adap/flower/tree/main/examples/tensorflow-privacy)
+- [Flower with a Tabular Dataset] (https://github.com/adap/flower/tree/main/examples/fl-tabular)
 
 ## Community
 

diff --git a/benchmarks/flowertune-llm/README.md b/benchmarks/flowertune-llm/README.md
@@ -0,0 +1,61 @@
+![](_static/flower_llm.jpg)
+
+# FlowerTune LLM Leaderboard
+
+This repository guides you through the process of federated LLM instruction tuning with a
+pre-trained [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.3) model across 4 domains --- general NLP, finance, medical and code.
+
+Please follow the instructions to run and evaluate the federated LLMs.
+
+## Create a new project
+
+As the first step, please register a Flower account on [Flower website](https://flower.ai/login).
+Assuming `flwr` package is already installed on your system (check [here](https://flower.ai/docs/framework/how-to-install-flower.html) for `flwr` installation).
+We provide a single-line command to create a new project directory based on your selected challenge:
+
+```shell
+flwr new --framework=flwrtune --username=your_flower_account
+```
+
+Then you will see a prompt to ask your project name and the choice of LLM challenges from the set of general NLP, finance, medical and code.
+Type your project name and select your preferred challenge,
+and then a new project directory will be generated automatically.
+
+### Structure
+
+After running `flwr new`, you will see a new directory generated with the following structure:
+
+```bash
+<project-name>
+       ├── README.md           # <- Instructions
+       ├── pyproject.toml      # <- Environment dependencies
+       └── <project_name>
+                  ├── app.py          # <- Flower ClientApp/ServerApp build
+                  ├── client.py       # <- Flower client constructor
+                  ├── server.py       # <- Sever-related functions
+                  ├── models.py       # <- Model build
+                  ├── dataset.py      # <- Dataset and tokenizer build
+                  ├── conf/config.yaml         # <- User configuration
+                  └── conf/static_config.yaml  # <- Static configuration
+```
+
+This can serve as the starting point for you to build up your own federated LLM fine-tuning methods.
+Please note that any modification to the content of `conf/static_config.yaml` is strictly prohibited for those who wish to participate in the [LLM Leaderboard](https://flower.ai/benchmarks/llm-leaderboard).
+Otherwise, the submission will not be considered.
+
+## Run FlowerTune LLM challenges
+
+With a new project directory created, running a baseline challenge can be done by:
+
+1. Navigate inside the directory that you just created.
+
+
+2. Follow the `Environments setup` section of `README.md` in the project directory to install project dependencies.
+
+
+3. Run the challenge as indicated in the `Running the challenge` section in the `README.md`.
+
+## Evaluate pre-trained LLMs
+
+After the LLM fine-tuning finished, evaluate the performance of your pre-trained LLMs
+following the `README.md` in `evaluation` directory.
diff --git a/benchmarks/flowertune-llm/_static/flower_llm.jpg b/benchmarks/flowertune-llm/_static/flower_llm.jpg
diff --git a/datasets/flwr_datasets/__init__.py b/datasets/flwr_datasets/__init__.py
@@ -23,11 +23,12 @@
 
 __all__ = [
     "FederatedDataset",
-    "partitioner",
     "metrics",
-    "visualization",
+    "partitioner",
     "preprocessor",
     "utils",
+    "visualization",
 ]
 
+
 __version__ = _package_version
diff --git a/datasets/flwr_datasets/partitioner/__init__.py b/datasets/flwr_datasets/partitioner/__init__.py
@@ -28,15 +28,15 @@
 from .semantic_partitioner import SemanticPartitioner
 
 __all__ = [
+    "DirichletPartitioner",
+    "ExponentialPartitioner",
     "IidPartitioner",
-    "Partitioner",
+    "InnerDirichletPartitioner",
+    "LinearPartitioner",
     "NaturalIdPartitioner",
-    "DirichletPartitioner",
+    "Partitioner",
+    "ShardPartitioner",
     "SizePartitioner",
-    "LinearPartitioner",
-    "InnerDirichletPartitioner",
     "SquarePartitioner",
-    "ShardPartitioner",
-    "ExponentialPartitioner",
     "SemanticPartitioner"
 ]
diff --git a/datasets/flwr_datasets/preprocessor/__init__.py b/datasets/flwr_datasets/preprocessor/__init__.py
@@ -20,7 +20,7 @@
 from .preprocessor import Preprocessor
 
 __all__ = [
+    "Divider",
     "Merger",
     "Preprocessor",
-    "Divider",
 ]
diff --git a/datasets/flwr_datasets/visualization/__init__.py b/datasets/flwr_datasets/visualization/__init__.py
@@ -19,6 +19,6 @@
 from .label_distribution import plot_label_distributions
 
 __all__ = [
-    "plot_label_distributions",
     "plot_comparison_label_distribution",
+    "plot_label_distributions",
 ]
diff --git a/datasets/flwr_datasets/visualization/bar_plot.py b/datasets/flwr_datasets/visualization/bar_plot.py
@@ -38,7 +38,6 @@ def _plot_bar(
     plot_kwargs: Optional[Dict[str, Any]],
     legend_kwargs: Optional[Dict[str, Any]],
 ) -> Axes:
-
     if axis is None:
         if figsize is None:
             figsize = _initialize_figsize(

diff --git a/datasets/flwr_datasets/visualization/heatmap_plot.py b/datasets/flwr_datasets/visualization/heatmap_plot.py
@@ -39,7 +39,6 @@ def _plot_heatmap(
     plot_kwargs: Optional[Dict[str, Any]],
     legend_kwargs: Optional[Dict[str, Any]],
 ) -> Axes:
-
     if axis is None:
         if figsize is None:
             figsize = _initialize_figsize(
@@ -92,7 +91,6 @@ def _initialize_figsize(
     num_partitions: int,
     num_labels: int,
 ) -> Tuple[float, float]:
-
     figsize = (0.0, 0.0)
     if partition_id_axis == "x":
         figsize = (3 * np.sqrt(num_partitions), np.sqrt(num_labels))

diff --git a/dev/format.sh b/dev/format.sh
@@ -3,6 +3,7 @@ set -e
 cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"/../
 
 # Python
+python -m flwr_tool.check_copyright src/py/flwr
 python -m flwr_tool.init_py_fix src/py/flwr
 python -m isort --skip src/py/flwr/proto src/py
 python -m black -q --exclude src/py/flwr/proto src/py

diff --git a/dev/test.sh b/dev/test.sh
@@ -58,6 +58,10 @@ echo "- All Markdown checks passed"
 
 echo "- Start license checks"
 
+echo "- copyright: start"
+python -m flwr_tool.check_copyright src/py/flwr
+echo "- copyright: done"
+
 echo "- licensecheck: start"
 python -m licensecheck -u poetry --fail-licenses gpl --zero
 echo "- licensecheck: done"

diff --git a/doc/source/contributor-how-to-build-docker-images.rst b/doc/source/contributor-how-to-build-docker-images.rst
@@ -65,22 +65,22 @@ Building the base image
    * - ``FLWR_VERSION``
      - Version of Flower to be installed.
      - Yes
-     - ``1.8.0``
+     - ``1.9.0``
    * - ``FLWR_PACKAGE``
      - The Flower package to be installed.
      - No
      - ``flwr`` or ``flwr-nightly``
 
 
 The following example creates a base Ubuntu/Alpine image with Python 3.11.0, pip 23.0.1,
-setuptools 69.0.2 and Flower 1.8.0:
+setuptools 69.0.2 and Flower 1.9.0:
 
 .. code-block:: bash
 
   $ cd src/docker/base/<ubuntu|alpine>
   $ docker build \
     --build-arg PYTHON_VERSION=3.11.0 \
-    --build-arg FLWR_VERSION=1.8.0 \
+    --build-arg FLWR_VERSION=1.9.0 \
     --build-arg PIP_VERSION=23.0.1 \
     --build-arg SETUPTOOLS_VERSION=69.0.2 \
     -t flwr_base:0.1.0 .
@@ -106,7 +106,7 @@ Building the SuperLink/SuperNode or ServerApp image
    * - ``BASE_IMAGE``
      - The Tag of the Flower base image.
      - Yes
-     - ``1.8.0-py3.10-ubuntu22.04``
+     - ``1.9.0-py3.10-ubuntu22.04``
 
 The following example creates a SuperLink/SuperNode or ServerApp image with the official Flower
 base image:

diff --git a/doc/source/contributor-tutorial-get-started-as-a-contributor.rst b/doc/source/contributor-tutorial-get-started-as-a-contributor.rst
@@ -17,8 +17,8 @@ supports `PEP 517 <https://peps.python.org/pep-0517/>`_.
 Developer Machine Setup
 -----------------------
 
-Preliminarities
-~~~~~~~~~~~~~~~
+Preliminaries
+~~~~~~~~~~~~~
 Some system-wide dependencies are needed.
 
 For macOS

diff --git a/doc/source/how-to-install-flower.rst b/doc/source/how-to-install-flower.rst
@@ -20,7 +20,7 @@ Stable releases are available on `PyPI <https://pypi.org/project/flwr/>`_::
 
 For simulations that use the Virtual Client Engine, ``flwr`` should be installed with the ``simulation`` extra::
 
-  python -m pip install flwr[simulation]
+  python -m pip install "flwr[simulation]"
 
 
 Using conda (or mamba)