diff --git a/doc/source/tutorial-quickstart-xgboost.rst b/doc/source/tutorial-quickstart-xgboost.rst index fe15227fdf11..2350dfecbab6 100644 --- a/doc/source/tutorial-quickstart-xgboost.rst +++ b/doc/source/tutorial-quickstart-xgboost.rst @@ -3,12 +3,6 @@ Quickstart XGBoost ================== -.. meta:: - :description: Check out this Federated Learning quickstart tutorial for using Flower with XGBoost to train classification models on trees. - -.. youtube:: AY1vpXUpesc - :width: 100% - Federated XGBoost ----------------- @@ -34,11 +28,11 @@ Given the robustness and efficiency of XGBoost, combining it with federated lear offers a promising solution for these specific challenges. In this tutorial we will learn how to train a federated XGBoost model on HIGGS dataset -using Flower and ``xgboost`` package. We use a simple example (`full code -xgboost-quickstart -`_) with two -*clients* and one *server* to demonstrate how federated XGBoost works, and then we dive -into a more complex example (`full code xgboost-comprehensive +using Flower and ``xgboost`` package to perform a binary classification task. We use a +simple example (`full code xgboost-quickstart +`_) to demonstrate +how federated XGBoost works, and then we dive into a more complex example (`full code +xgboost-comprehensive `_) to run various experiments. @@ -52,6 +46,7 @@ We first need to install Flower and Flower Datasets. You can do this by running .. code-block:: shell + # In a new Python environment $ pip install flwr flwr-datasets Since we want to use ``xgboost`` package to build up XGBoost trees, let's go ahead and @@ -61,76 +56,55 @@ install ``xgboost``: $ pip install xgboost -Flower Client -------------- +The Configurations +------------------ -*Clients* are responsible for generating individual weight-updates for the model based -on their local datasets. Now that we have all our dependencies installed, let's run a -simple distributed training with two clients and one server. +We define all required configurations/hyper-parameters in ``pyproject.toml``: -In a file called ``client.py``, import xgboost, Flower, Flower Datasets and other -related functions: +.. code-block:: toml -.. code-block:: python + [tool.flwr.app.config] + # ServerApp + num-server-rounds = 3 + fraction-fit = 0.1 + fraction-evaluate = 0.1 - import argparse - from typing import Union - from logging import INFO - from datasets import Dataset, DatasetDict - import xgboost as xgb + # ClientApp + local-epochs = 1 + params.objective = "binary:logistic" + params.eta = 0.1 # Learning rate + params.max-depth = 8 + params.eval-metric = "auc" + params.nthread = 16 + params.num-parallel-tree = 1 + params.subsample = 1 + params.tree-method = "hist" - import flwr as fl - from flwr_datasets import FederatedDataset - from flwr.common.logger import log - from flwr.common import ( - Code, - EvaluateIns, - EvaluateRes, - FitIns, - FitRes, - GetParametersIns, - GetParametersRes, - Parameters, - Status, - ) - from flwr_datasets.partitioner import IidPartitioner +The ``local-epochs`` represents the number of iterations for local tree boost. We use +CPU for the training in default. One can shift it to GPU by setting ``tree_method`` to +``gpu_hist``. We use AUC as evaluation metric. -Dataset partition and hyper-parameter selection -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The Data +-------- -Prior to local training, we require loading the HIGGS dataset from Flower Datasets and -conduct data partitioning for FL: +This tutorial uses `Flower Datasets `_ to easily +download and partition the `HIGGS` dataset. .. code-block:: python # Load (HIGGS) dataset and conduct partitioning - # We use a small subset (num_partitions=30) of the dataset for demonstration to speed up the data loading process. - partitioner = IidPartitioner(num_partitions=30) + # We use a small subset (num_partitions=20) of the dataset for demonstration to speed up the data loading process. + partitioner = IidPartitioner(num_partitions=20) fds = FederatedDataset(dataset="jxie/higgs", partitioners={"train": partitioner}) - # Load the partition for this `node_id` - partition = fds.load_partition(partition_id=args.partition_id, split="train") + # Load the partition for this `partition_id` + partition = fds.load_partition(partition_id, split="train") partition.set_format("numpy") -In this example, we split the dataset into 30 partitions with uniform distribution -(``IidPartitioner(num_partitions=30)``). Then, we load the partition for the given -client based on ``partition_id``: - -.. code-block:: python - - # We first define arguments parser for user to specify the client/partition ID. - parser = argparse.ArgumentParser() - parser.add_argument( - "--partition-id", - default=0, - type=int, - help="Partition ID used for the current client.", - ) - args = parser.parse_args() - - # Load the partition for this `partition_id`. - partition = fds.load_partition(idx=args.partition_id, split="train") - partition.set_format("numpy") +In this example, we split the dataset into 20 partitions with uniform distribution +(`IidPartitioner +`_). +Then, we load the partition for the given client based on ``partition_id``. After that, we do train/test splitting on the given partition (client's local data), and transform data format for ``xgboost`` package. @@ -151,8 +125,7 @@ as below: .. code-block:: python - # Define data partitioning related functions - def train_test_split(partition: Dataset, test_fraction: float, seed: int): + def train_test_split(partition, test_fraction, seed): """Split the data into train and validation set given split rate.""" train_test = partition.train_test_split(test_size=test_fraction, seed=seed) partition_train = train_test["train"] @@ -164,42 +137,25 @@ as below: return partition_train, partition_test, num_train, num_test - def transform_dataset_to_dmatrix(data: Union[Dataset, DatasetDict]) -> xgb.core.DMatrix: + def transform_dataset_to_dmatrix(data): """Transform dataset to DMatrix format for xgboost.""" x = data["inputs"] y = data["label"] new_data = xgb.DMatrix(x, label=y) return new_data -Finally, we define the hyper-parameters used for XGBoost training. - -.. code-block:: python - - num_local_round = 1 - params = { - "objective": "binary:logistic", - "eta": 0.1, # lr - "max_depth": 8, - "eval_metric": "auc", - "nthread": 16, - "num_parallel_tree": 1, - "subsample": 1, - "tree_method": "hist", - } - -The ``num_local_round`` represents the number of iterations for local tree boost. We use -CPU for the training in default. One can shift it to GPU by setting ``tree_method`` to -``gpu_hist``. We use AUC as evaluation metric. - -Flower client definition for XGBoost -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The ClientApp +------------- -After loading the dataset we define the Flower client. We follow the general rule to -define ``XgbClient`` class inherited from ``fl.client.Client``. +*Clients* are responsible for generating individual weight-updates for the model based +on their local datasets. Let's first see how we define Flower client for XGBoost. We +follow the general rule to define ``FlowerClient`` class inherited from +``fl.client.Client``. .. code-block:: python - class XgbClient(fl.client.Client): + # Define Flower Client and client_fn + class FlowerClient(Client): def __init__( self, train_dmatrix, @@ -216,27 +172,10 @@ define ``XgbClient`` class inherited from ``fl.client.Client``. self.num_local_round = num_local_round self.params = params -All required parameters defined above are passed to ``XgbClient``'s constructor. +All required parameters defined above are passed to ``FlowerClient``'s constructor. -Then, we override ``get_parameters``, ``fit`` and ``evaluate`` methods insides -``XgbClient`` class as follows. - -.. code-block:: python - - def get_parameters(self, ins: GetParametersIns) -> GetParametersRes: - _ = (self, ins) - return GetParametersRes( - status=Status( - code=Code.OK, - message="OK", - ), - parameters=Parameters(tensor_type="", tensors=[]), - ) - -Unlike neural network training, XGBoost trees are not started from a specified random -weights. In this case, we do not use ``get_parameters`` and ``set_parameters`` to -initialise model parameters for XGBoost. As a result, let's return an empty tensor in -``get_parameters`` when it is called by the server at the first round. +Then, we override ``fit`` and ``evaluate`` methods insides ``FlowerClient`` class as +follows. .. code-block:: python @@ -252,8 +191,7 @@ initialise model parameters for XGBoost. As a result, let's return an empty tens ) else: bst = xgb.Booster(params=self.params) - for item in ins.parameters.tensors: - global_model = bytearray(item) + global_model = bytearray(ins.parameters.tensors[0]) # Load global model into booster bst.load_model(global_model) @@ -278,7 +216,7 @@ initialise model parameters for XGBoost. As a result, let's return an empty tens In ``fit``, at the first round, we call ``xgb.train()`` to build up the first set of trees. From the second round, we load the global model sent from server to new build Booster object, and then update model weights on local training data with function -``local_boost`` as follows: +``_local_boost`` as follows: .. code-block:: python @@ -303,8 +241,7 @@ training, the last ``N=num_local_round`` trees will be extracted to send to the def evaluate(self, ins: EvaluateIns) -> EvaluateRes: # Load global model bst = xgb.Booster(params=self.params) - for para in ins.parameters.tensors: - para_b = bytearray(para) + para_b = bytearray(ins.parameters.tensors[0]) bst.load_model(para_b) # Run evaluation @@ -314,9 +251,6 @@ training, the last ``N=num_local_round`` trees will be extracted to send to the ) auc = round(float(eval_results.split("\t")[1].split(":")[1]), 4) - global_round = ins.config["global_round"] - log(INFO, f"AUC = {auc} at round {global_round}") - return EvaluateRes( status=Status( code=Code.OK, @@ -330,54 +264,25 @@ training, the last ``N=num_local_round`` trees will be extracted to send to the In ``evaluate``, after loading the global model, we call ``bst.eval_set`` function to conduct evaluation on valid set. The AUC value will be returned. -Now, we can create an instance of our class ``XgbClient`` and add one line to actually -run this client: - -.. code-block:: python - - fl.client.start_client( - server_address="127.0.0.1:8080", - client=XgbClient( - train_dmatrix, - valid_dmatrix, - num_train, - num_val, - num_local_round, - params, - ).to_client(), - ) - -That's it for the client. We only have to implement ``Client`` and call -``fl.client.start_client()``. The string ``"[::]:8080"`` tells the client which server -to connect to. In our case we can run the server and the client on the same machine, -therefore we use ``"[::]:8080"``. If we run a truly federated workload with the server -and clients running on different machines, all that needs to change is the -``server_address`` we point the client at. - -Flower Server +The ServerApp ------------- -These updates are then sent to the *server* which will aggregate them to produce a -better model. Finally, the *server* sends this improved version of the model back to -each *client* to finish a complete FL round. - -In a file named ``server.py``, import Flower and FedXgbBagging from -``flwr.server.strategy``. +After the local training on clients, the model updates are then sent to the *server* +which will aggregate them to produce a better model. Finally, the *server* sends this +improved version of the model back to each *client* to finish a complete FL round. -We first define a strategy for XGBoost bagging aggregation. +In a file named ``server_app.py``, we define a strategy for XGBoost bagging aggregation: .. code-block:: python # Define strategy strategy = FedXgbBagging( - fraction_fit=1.0, - min_fit_clients=2, - min_available_clients=2, - min_evaluate_clients=2, - fraction_evaluate=1.0, + fraction_fit=fraction_fit, + fraction_evaluate=fraction_evaluate, evaluate_metrics_aggregation_fn=evaluate_metrics_aggregation, on_evaluate_config_fn=config_func, on_fit_config_fn=config_func, + initial_parameters=parameters, ) @@ -398,21 +303,9 @@ We first define a strategy for XGBoost bagging aggregation. } return config -We use two clients for this example. An ``evaluate_metrics_aggregation`` function is -defined to collect and wighted average the AUC values from clients. The ``config_func`` -function is to return the current FL round number to client's ``fit()`` and -``evaluate()`` methods. - -Then, we start the server: - -.. code-block:: python - - # Start Flower server - fl.server.start_server( - server_address="0.0.0.0:8080", - config=fl.server.ServerConfig(num_rounds=5), - strategy=strategy, - ) +An ``evaluate_metrics_aggregation`` function is defined to collect and wighted average +the AUC values from clients. The ``config_func`` function is to return the current FL +round number to client's ``fit()`` and ``evaluate()`` methods. Tree-based bagging aggregation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -587,87 +480,71 @@ serialisation, and sending back to each client. Launch Federated XGBoost! ------------------------- -With both client and server ready, we can now run everything and see federated learning -in action. FL systems usually have a server and multiple clients. We therefore have to -start the server first: - -.. code-block:: shell - - $ python3 server.py - -Once the server is running we can start the clients in different terminals. Open a new -terminal and start the first client: +To run the project, do: .. code-block:: shell - $ python3 client.py --partition-id=0 + # Run with default arguments + $ flwr run . -Open another terminal and start the second client: +With default arguments you will see an output like this one: .. code-block:: shell - $ python3 client.py --partition-id=1 - -Each client will have its own dataset. You should now see how the training does in the -very first terminal (the one that started the server): - -.. code-block:: shell - - INFO : Starting Flower server, config: num_rounds=5, no round_timeout - INFO : Flower ECE: gRPC server running (5 rounds), SSL is disabled + Loading project configuration... + Success + INFO : Starting Flower ServerApp, config: num_rounds=3, no round_timeout + INFO : INFO : [INIT] - INFO : Requesting initial parameters from one random client - INFO : Received initial parameters from one random client - INFO : Evaluating initial global parameters + INFO : Using initial global parameters provided by strategy + INFO : Starting evaluation of initial global parameters + INFO : Evaluation returned no results (`None`) INFO : INFO : [ROUND 1] - INFO : configure_fit: strategy sampled 2 clients (out of 2) + INFO : configure_fit: strategy sampled 2 clients (out of 20) INFO : aggregate_fit: received 2 results and 0 failures - INFO : configure_evaluate: strategy sampled 2 clients (out of 2) + INFO : configure_evaluate: strategy sampled 2 clients (out of 20) INFO : aggregate_evaluate: received 2 results and 0 failures INFO : INFO : [ROUND 2] - INFO : configure_fit: strategy sampled 2 clients (out of 2) + INFO : configure_fit: strategy sampled 2 clients (out of 20) INFO : aggregate_fit: received 2 results and 0 failures - INFO : configure_evaluate: strategy sampled 2 clients (out of 2) + INFO : configure_evaluate: strategy sampled 2 clients (out of 20) INFO : aggregate_evaluate: received 2 results and 0 failures INFO : INFO : [ROUND 3] - INFO : configure_fit: strategy sampled 2 clients (out of 2) + INFO : configure_fit: strategy sampled 2 clients (out of 20) INFO : aggregate_fit: received 2 results and 0 failures - INFO : configure_evaluate: strategy sampled 2 clients (out of 2) - INFO : aggregate_evaluate: received 2 results and 0 failures - INFO : - INFO : [ROUND 4] - INFO : configure_fit: strategy sampled 2 clients (out of 2) - INFO : aggregate_fit: received 2 results and 0 failures - INFO : configure_evaluate: strategy sampled 2 clients (out of 2) - INFO : aggregate_evaluate: received 2 results and 0 failures - INFO : - INFO : [ROUND 5] - INFO : configure_fit: strategy sampled 2 clients (out of 2) - INFO : aggregate_fit: received 2 results and 0 failures - INFO : configure_evaluate: strategy sampled 2 clients (out of 2) + INFO : configure_evaluate: strategy sampled 2 clients (out of 20) INFO : aggregate_evaluate: received 2 results and 0 failures INFO : INFO : [SUMMARY] - INFO : Run finished 5 round(s) in 1.67s + INFO : Run finished 3 round(s) in 145.42s INFO : History (loss, distributed): INFO : round 1: 0 INFO : round 2: 0 INFO : round 3: 0 - INFO : round 4: 0 - INFO : round 5: 0 INFO : History (metrics, distributed, evaluate): - INFO : {'AUC': [(1, 0.76755), (2, 0.775), (3, 0.77935), (4, 0.7836), (5, 0.7872)]} + INFO : {'AUC': [(1, 0.7664), (2, 0.77595), (3, 0.7826)]} + INFO : Congratulations! You've successfully built and run your first federated XGBoost system. -The AUC values can be checked in ``metrics_distributed``. One can see that the average -AUC increases over FL rounds. +The AUC values can be checked in ``History (metrics, distributed, evaluate)``. One can +see that the average AUC increases over FL rounds. -The full `source code -`_ for this -example can be found in ``examples/xgboost-quickstart``. +You can also override the parameters defined in the ``[tool.flwr.app.config]`` section +in ``pyproject.toml`` like this: + +.. code-block:: shell + + # Override some arguments + $ flwr run . --run-config "num-server-rounds=5 params.eta=0.05" + +.. note:: + + Check the full `source code + `_ for this + example in ``examples/xgboost-quickstart`` in the Flower GitHub repository. Comprehensive Federated XGBoost ------------------------------- @@ -677,9 +554,7 @@ more comprehensive experiments by customising the experimental settings. In the xgboost-comprehensive example (`full code `_), we provide more options to define various experimental setups, including aggregation strategies, -data partitioning and centralised/distributed evaluation. We also support :doc:`Flower -simulation ` making it easy to simulate large client cohorts in -a resource-aware manner. Let's take a look! +data partitioning and centralised/distributed evaluation. Let's take a look! Cyclic training ~~~~~~~~~~~~~~~ @@ -690,7 +565,7 @@ one single client participating in the training per round in the cyclic training scenario. The trained local XGBoost trees will be passed to the next client as an initialised model for next round's boosting. -To do this, we first customise a ``ClientManager`` in ``server_utils.py``: +To do this, we first customise a ``ClientManager`` in ``server_app.py``: .. code-block:: python @@ -804,48 +679,17 @@ clients to be sequentially selected given FL round: # Return client/config pairs return [(client, fit_ins) for client in sampled_clients] - - def configure_evaluate( - self, server_round: int, parameters: Parameters, client_manager: ClientManager - ) -> List[Tuple[ClientProxy, EvaluateIns]]: - """Configure the next round of evaluation.""" - # Do not configure federated evaluation if fraction eval is 0. - if self.fraction_evaluate == 0.0: - return [] - - # Parameters and config - config = {} - if self.on_evaluate_config_fn is not None: - # Custom evaluation config function provided - config = self.on_evaluate_config_fn(server_round) - evaluate_ins = EvaluateIns(parameters, config) - - # Sample clients - sample_size, min_num_clients = self.num_evaluation_clients( - client_manager.num_available() - ) - clients = client_manager.sample( - num_clients=sample_size, - min_num_clients=min_num_clients, - ) - - # Sample the clients sequentially given server_round - sampled_idx = (server_round - 1) % len(clients) - sampled_clients = [clients[sampled_idx]] - - # Return client/config pairs - return [(client, evaluate_ins) for client in sampled_clients] - Customised data partitioning ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In ``dataset.py``, we have a function ``instantiate_partitioner`` to instantiate the -data partitioner based on the given ``num_partitions`` and ``partitioner_type``. +In ``task.py``, we have a function ``instantiate_fds`` to instantiate Flower Datasets +and the data partitioner based on the given ``partitioner_type`` and ``num_partitions``. Currently, we provide four supported partitioner type to simulate the uniformity/non-uniformity in data quantity (uniform, linear, square, exponential). .. code-block:: python + from flwr_datasets import FederatedDataset from flwr_datasets.partitioner import ( IidPartitioner, LinearPartitioner, @@ -861,22 +705,29 @@ uniformity/non-uniformity in data quantity (uniform, linear, square, exponential } - def instantiate_partitioner(partitioner_type: str, num_partitions: int): - """Initialise partitioner based on selected partitioner type and number of - partitions.""" - partitioner = CORRELATION_TO_PARTITIONER[partitioner_type]( - num_partitions=num_partitions - ) - return partitioner + def instantiate_fds(partitioner_type, num_partitions): + """Initialize FederatedDataset.""" + # Only initialize `FederatedDataset` once + global fds + if fds is None: + partitioner = CORRELATION_TO_PARTITIONER[partitioner_type]( + num_partitions=num_partitions + ) + fds = FederatedDataset( + dataset="jxie/higgs", + partitioners={"train": partitioner}, + preprocessor=resplit, + ) + return fds Customised centralised/distributed evaluation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To facilitate centralised evaluation, we define a function in ``server_utils.py``: +To facilitate centralised evaluation, we define a function in ``server_app.py``: .. code-block:: python - def get_evaluate_fn(test_data): + def get_evaluate_fn(test_data, params): """Return a function for centralised evaluation.""" def evaluate_fn( @@ -898,7 +749,6 @@ To facilitate centralised evaluation, we define a function in ``server_utils.py` iteration=bst.num_boosted_rounds() - 1, ) auc = round(float(eval_results.split("\t")[1].split(":")[1]), 4) - log(INFO, f"AUC = {auc} at round {server_round}") return 0, {"AUC": auc} @@ -910,434 +760,83 @@ loads the global model weights to it. The evaluation is conducted by calling As for distributed evaluation on the clients, it's same as the quick-start example by overriding the ``evaluate()`` method insides the ``XgbClient`` class in -``client_utils.py``. - -Flower simulation -~~~~~~~~~~~~~~~~~ - -We also provide an example code (``sim.py``) to use the simulation capabilities of -Flower to simulate federated XGBoost training on either a single machine or a cluster of -machines. - -.. code-block:: python - - from logging import INFO - import xgboost as xgb - from tqdm import tqdm - - import flwr as fl - from flwr_datasets import FederatedDataset - from flwr.common.logger import log - from flwr.server.strategy import FedXgbBagging, FedXgbCyclic - - from dataset import ( - instantiate_partitioner, - train_test_split, - transform_dataset_to_dmatrix, - separate_xy, - resplit, - ) - from utils import ( - sim_args_parser, - NUM_LOCAL_ROUND, - BST_PARAMS, - ) - from server_utils import ( - eval_config, - fit_config, - evaluate_metrics_aggregation, - get_evaluate_fn, - CyclicClientManager, - ) - from client_utils import XgbClient - -After importing all required packages, we define a ``main()`` function to perform the -simulation process: - -.. code-block:: python - - def main(): - # Parse arguments for experimental settings - args = sim_args_parser() - - # Load (HIGGS) dataset and conduct partitioning - partitioner = instantiate_partitioner( - partitioner_type=args.partitioner_type, num_partitions=args.pool_size - ) - fds = FederatedDataset( - dataset="jxie/higgs", - partitioners={"train": partitioner}, - resplitter=resplit, - ) - - # Load centralised test set - if args.centralised_eval or args.centralised_eval_client: - log(INFO, "Loading centralised test set...") - test_data = fds.load_split("test") - test_data.set_format("numpy") - num_test = test_data.shape[0] - test_dmatrix = transform_dataset_to_dmatrix(test_data) - - # Load partitions and reformat data to DMatrix for xgboost - log(INFO, "Loading client local partitions...") - train_data_list = [] - valid_data_list = [] - - # Load and process all client partitions. This upfront cost is amortized soon - # after the simulation begins since clients wont need to preprocess their partition. - for node_id in tqdm(range(args.pool_size), desc="Extracting client partition"): - # Extract partition for client with node_id - partition = fds.load_partition(node_id=node_id, split="train") - partition.set_format("numpy") - - if args.centralised_eval_client: - # Use centralised test set for evaluation - train_data = partition - num_train = train_data.shape[0] - x_test, y_test = separate_xy(test_data) - valid_data_list.append(((x_test, y_test), num_test)) - else: - # Train/test splitting - train_data, valid_data, num_train, num_val = train_test_split( - partition, test_fraction=args.test_fraction, seed=args.seed - ) - x_valid, y_valid = separate_xy(valid_data) - valid_data_list.append(((x_valid, y_valid), num_val)) - - x_train, y_train = separate_xy(train_data) - train_data_list.append(((x_train, y_train), num_train)) - -We first load the dataset and perform data partitioning, and the pre-processed data is -stored in a ``list``. After the simulation begins, the clients won't need to pre-process -their partitions again. - -Then, we define the strategies and other hyper-parameters: - -.. code-block:: python - - # Define strategy - if args.train_method == "bagging": - # Bagging training - strategy = FedXgbBagging( - evaluate_function=( - get_evaluate_fn(test_dmatrix) if args.centralised_eval else None - ), - fraction_fit=(float(args.num_clients_per_round) / args.pool_size), - min_fit_clients=args.num_clients_per_round, - min_available_clients=args.pool_size, - min_evaluate_clients=( - args.num_evaluate_clients if not args.centralised_eval else 0 - ), - fraction_evaluate=1.0 if not args.centralised_eval else 0.0, - on_evaluate_config_fn=eval_config, - on_fit_config_fn=fit_config, - evaluate_metrics_aggregation_fn=( - evaluate_metrics_aggregation if not args.centralised_eval else None - ), - ) - else: - # Cyclic training - strategy = FedXgbCyclic( - fraction_fit=1.0, - min_available_clients=args.pool_size, - fraction_evaluate=1.0, - evaluate_metrics_aggregation_fn=evaluate_metrics_aggregation, - on_evaluate_config_fn=eval_config, - on_fit_config_fn=fit_config, - ) - - # Resources to be assigned to each virtual client - # In this example we use CPU by default - client_resources = { - "num_cpus": args.num_cpus_per_client, - "num_gpus": 0.0, - } - - # Hyper-parameters for xgboost training - num_local_round = NUM_LOCAL_ROUND - params = BST_PARAMS - - # Setup learning rate - if args.train_method == "bagging" and args.scaled_lr: - new_lr = params["eta"] / args.pool_size - params.update({"eta": new_lr}) - -After that, we start the simulation by calling ``fl.simulation.start_simulation``: - -.. code-block:: python - - # Start simulation - fl.simulation.start_simulation( - client_fn=get_client_fn( - train_data_list, - valid_data_list, - args.train_method, - params, - num_local_round, - ), - num_clients=args.pool_size, - client_resources=client_resources, - config=fl.server.ServerConfig(num_rounds=args.num_rounds), - strategy=strategy, - client_manager=CyclicClientManager() if args.train_method == "cyclic" else None, - ) - -One of key parameters for ``start_simulation`` is ``client_fn`` which returns a function -to construct a client. We define it as follows: - -.. code-block:: python - - def get_client_fn( - train_data_list, valid_data_list, train_method, params, num_local_round - ): - """Return a function to construct a client. - - The VirtualClientEngine will execute this function whenever a client is sampled by - the strategy to participate. - """ - - def client_fn(cid: str) -> fl.client.Client: - """Construct a FlowerClient with its own dataset partition.""" - x_train, y_train = train_data_list[int(cid)][0] - x_valid, y_valid = valid_data_list[int(cid)][0] - - # Reformat data to DMatrix - train_dmatrix = xgb.DMatrix(x_train, label=y_train) - valid_dmatrix = xgb.DMatrix(x_valid, label=y_valid) - - # Fetch the number of examples - num_train = train_data_list[int(cid)][1] - num_val = valid_data_list[int(cid)][1] - - # Create and return client - return XgbClient( - train_dmatrix, - valid_dmatrix, - num_train, - num_val, - num_local_round, - params, - train_method, - ) - - return client_fn - -Arguments parser -~~~~~~~~~~~~~~~~ - -In ``utils.py``, we define the arguments parsers for clients, server and simulation, -allowing users to specify different experimental settings. Let's first see the sever -side: - -.. code-block:: python - - import argparse - - - def server_args_parser(): - """Parse arguments to define experimental settings on server side.""" - parser = argparse.ArgumentParser() - - parser.add_argument( - "--train-method", - default="bagging", - type=str, - choices=["bagging", "cyclic"], - help="Training methods selected from bagging aggregation or cyclic training.", - ) - parser.add_argument( - "--pool-size", default=2, type=int, help="Number of total clients." - ) - parser.add_argument( - "--num-rounds", default=5, type=int, help="Number of FL rounds." - ) - parser.add_argument( - "--num-clients-per-round", - default=2, - type=int, - help="Number of clients participate in training each round.", - ) - parser.add_argument( - "--num-evaluate-clients", - default=2, - type=int, - help="Number of clients selected for evaluation.", - ) - parser.add_argument( - "--centralised-eval", - action="store_true", - help="Conduct centralised evaluation (True), or client evaluation on hold-out data (False).", - ) - - args = parser.parse_args() - return args - -This allows user to specify training strategies / the number of total clients / FL -rounds / participating clients / clients for evaluation, and evaluation fashion. Note -that with ``--centralised-eval``, the sever will do centralised evaluation and all +``client_app.py``. + +Arguments explainer +~~~~~~~~~~~~~~~~~~~ + +We define all hyper-parameters under ``[tool.flwr.app.config]`` entry in +``pyproject.toml``: + +.. code-block:: toml + + [tool.flwr.app.config] + # ServerApp + train-method = "bagging" # Choose from [bagging, cyclic] + num-server-rounds = 3 + fraction-fit = 1.0 + fraction-evaluate = 1.0 + centralised-eval = false + + # ClientApp + partitioner-type = "uniform" # Choose from [uniform, linear, square, exponential] + test-fraction = 0.2 + seed = 42 + centralised-eval-client = false + local-epochs = 1 + scaled-lr = false + params.objective = "binary:logistic" + params.eta = 0.1 # Learning rate + params.max-depth = 8 + params.eval-metric = "auc" + params.nthread = 16 + params.num-parallel-tree = 1 + params.subsample = 1 + params.tree-method = "hist" + +On the server side, we allow user to specify training strategies / FL rounds / +participating clients / clients for evaluation, and evaluation fashion. Note that with +``centralised-eval = true``, the sever will do centralised evaluation and all functionalities for client evaluation will be disabled. -Then, the argument parser on client side: - -.. code-block:: python - - def client_args_parser(): - """Parse arguments to define experimental settings on client side.""" - parser = argparse.ArgumentParser() - - parser.add_argument( - "--train-method", - default="bagging", - type=str, - choices=["bagging", "cyclic"], - help="Training methods selected from bagging aggregation or cyclic training.", - ) - parser.add_argument( - "--num-partitions", default=10, type=int, help="Number of partitions." - ) - parser.add_argument( - "--partitioner-type", - default="uniform", - type=str, - choices=["uniform", "linear", "square", "exponential"], - help="Partitioner types.", - ) - parser.add_argument( - "--node-id", - default=0, - type=int, - help="Node ID used for the current client.", - ) - parser.add_argument( - "--seed", default=42, type=int, help="Seed used for train/test splitting." - ) - parser.add_argument( - "--test-fraction", - default=0.2, - type=float, - help="Test fraction for train/test splitting.", - ) - parser.add_argument( - "--centralised-eval", - action="store_true", - help="Conduct evaluation on centralised test set (True), or on hold-out data (False).", - ) - parser.add_argument( - "--scaled-lr", - action="store_true", - help="Perform scaled learning rate based on the number of clients (True).", - ) - - args = parser.parse_args() - return args - -This defines various options for client data partitioning. Besides, clients also have an -option to conduct evaluation on centralised test set by setting ``--centralised-eval``, -as well as an option to perform scaled learning rate based on the number of clients by -setting ``--scaled-lr``. - -We also have an argument parser for simulation: - -.. code-block:: python - - def sim_args_parser(): - """Parse arguments to define experimental settings on server side.""" - parser = argparse.ArgumentParser() - - parser.add_argument( - "--train-method", - default="bagging", - type=str, - choices=["bagging", "cyclic"], - help="Training methods selected from bagging aggregation or cyclic training.", - ) - - # Server side - parser.add_argument( - "--pool-size", default=5, type=int, help="Number of total clients." - ) - parser.add_argument( - "--num-rounds", default=30, type=int, help="Number of FL rounds." - ) - parser.add_argument( - "--num-clients-per-round", - default=5, - type=int, - help="Number of clients participate in training each round.", - ) - parser.add_argument( - "--num-evaluate-clients", - default=5, - type=int, - help="Number of clients selected for evaluation.", - ) - parser.add_argument( - "--centralised-eval", - action="store_true", - help="Conduct centralised evaluation (True), or client evaluation on hold-out data (False).", - ) - parser.add_argument( - "--num-cpus-per-client", - default=2, - type=int, - help="Number of CPUs used for per client.", - ) - - # Client side - parser.add_argument( - "--partitioner-type", - default="uniform", - type=str, - choices=["uniform", "linear", "square", "exponential"], - help="Partitioner types.", - ) - parser.add_argument( - "--seed", default=42, type=int, help="Seed used for train/test splitting." - ) - parser.add_argument( - "--test-fraction", - default=0.2, - type=float, - help="Test fraction for train/test splitting.", - ) - parser.add_argument( - "--centralised-eval-client", - action="store_true", - help="Conduct evaluation on centralised test set (True), or on hold-out data (False).", - ) - parser.add_argument( - "--scaled-lr", - action="store_true", - help="Perform scaled learning rate based on the number of clients (True).", - ) - - args = parser.parse_args() - return args - -This integrates all arguments for both client and server sides. +On the client side, we can define various options for client data partitioning. Besides, +clients also have an option to conduct evaluation on centralised test set by setting +``centralised-eval = true``, as well as an option to perform scaled learning rate based +on the number of clients by setting ``scaled-lr = true``. Example commands ~~~~~~~~~~~~~~~~ -To run a centralised evaluated experiment with bagging strategy on 5 clients with -exponential distribution for 50 rounds, we first start the server as below: +To run bagging aggregation for 5 rounds evaluated on centralised test set: .. code-block:: shell - $ python3 server.py --train-method=bagging --pool-size=5 --num-rounds=50 --num-clients-per-round=5 --centralised-eval - -Then, on each client terminal, we start the clients: + flwr run . --run-config "train-method='bagging' num-server-rounds=5 centralised-eval=true" +To run cyclic training with linear partitioner type evaluated on centralised test set: .. code-block:: shell - $ python3 clients.py --train-method=bagging --num-partitions=5 --partitioner-type=exponential --node-id=NODE_ID + flwr run . --run-config "train-method='cyclic' partitioner-type='linear' + centralised-eval-client=true" -To run the same experiment with Flower simulation: +.. note:: -.. code-block:: shell + The full `code + `_ for + this comprehensive example can be found in ``examples/xgboost-comprehensive`` in the + Flower GitHub repository. + +Video tutorial +-------------- - $ python3 sim.py --train-method=bagging --pool-size=5 --num-rounds=50 --num-clients-per-round=5 --partitioner-type=exponential --centralised-eval +.. note:: -The full `code -`_ for this -comprehensive example can be found in ``examples/xgboost-comprehensive``. + The video shown below shows how to setup a XGBoost + Flower project using our + previously recommended APIs. A new video tutorial will be released that shows the + new APIs (as the content above does) + +.. meta:: + :description: Check out this Federated Learning quickstart tutorial for using Flower with XGBoost to train classification models on trees. + +.. youtube:: AY1vpXUpesc + :width: 100%