[Q&A] Cannot load tenseal_context

[yuntianlong2002]

Python version (python3 -V)

3.8

NVFlare version (python3 -m pip list | grep "nvflare")

2.4.0rc5

NVFlare branch (if running examples, please use the branch that corresponds to the NVFlare version, git branch)

No response

Operating system

Ubuntu 22.04.3

Have you successfully run any of the following examples?

• hello-numpy-sag with simulator
• hello-pt with simulator
• hello-numpy-sag with POC
• hello-pt with POC

Please describe your question

we provisioned a real system with dashboard(1 server and two clients) and ran the XGBoost financial without problems, now we want to add homomorphic encryption to it by adding the following server configs:

{
            "id": "serialize_filter",
            "path": "nvflare.app_opt.he.model_serialize_filter.HEModelSerializeFilter",
            "args": {
            }
        },
        {
            "id": "aggregator",
            "path": "nvflare.app_opt.he.intime_accumulate_model_aggregator.HEInTimeAccumulateWeightedAggregator",
            "args": {
              "weigh_by_local_iter": false
            }
          }

and client configs

"task_result_filters": [
        {
          "tasks": ["train"],
          "filters":[
              {
                "path": "nvflare.app_opt.he.model_encryptor.HEModelEncryptor",
              "args": {
                "weigh_by_local_iter": true
              }
            }
          ]
        },
        {
          "tasks": ["submit_model"],
          "filters":[
              {
                "path": "nvflare.app_opt.he.model_encryptor.HEModelEncryptor",
              "args": {
                "weigh_by_local_iter": false
              }
            }
          ]
        }
      ],
      "task_data_filters": [
        {
          "tasks": ["train", "validate"],
          "filters":[
              {
                "path": "nvflare.app_opt.he.model_decryptor.HEModelDecryptor",
              "args": {
              }
            }
          ]
        }
      ],

but we got

2023-12-04 15:32:24,333 - HEModelSerializeFilter - ERROR - Traceback (most recent call last):
  File "/home/yun/miniconda3/envs/nvflare/lib/python3.8/site-packages/nvflare/private/event.py", line 61, in fire_event
    h.handle_event(event, ctx)
  File "/home/yun/miniconda3/envs/nvflare/lib/python3.8/site-packages/nvflare/app_opt/he/model_serialize_filter.py", line 40, in handle_event
    self.tenseal_context = load_tenseal_context_from_workspace(self.tenseal_context_file, fl_ctx)
  File "/home/yun/miniconda3/envs/nvflare/lib/python3.8/site-packages/nvflare/app_opt/he/homomorphic_encrypt.py", line 41, in load_tenseal_context_from_workspace
    raise ValueError("Cannot load tenseal_context {}: {}".format(ctx_file_name, rc))
ValueError: Cannot load tenseal_context server_context.tenseal: LoadResult.NO_SUCH_CONTENT

on the server side.

I am wondering how do I setup the server context, when I started the server, I just use the zip downloaded from dashboard and run the bash script to start the server

[yuntianlong2002]

Seems like *.tenseal mentioned here https://github.com/NVIDIA/NVFlare/blob/main/docs/real_world_fl/workspace.rst#L77 doesn't exist in the zip file downloaded from the Dashboard

[yuntianlong2002]

thanks! I will try that and will let you know!

[yuntianlong2002]

Seems like there is not a "context.save" function, so I used with open(os.path.join(".", "ctx_whole"), "wb") as f: f.write(context.serialize())

however I got Cannot load tenseal_context client_context.tenseal: LoadResult.NOT_SIGNED

[yuntianlong2002]

So loading the context is actually built-in function or nvflare, but anyway I provision the system with the command line tool instead of the dashboard, then the tenseal context is correctly generated automatically now.

[holgerroth]

Yes, you need to configure your project file to include the HEBuilder which generates these tenseal files. An example can be found here.

I would recommend trying the HE example with CIFAR-10 first.

[yuntianlong2002]

thanks! I use the command line bootstrap tool and now it works. On a separate note, will HE work on the financial example at all, or it only work in neural network?

[ZiyueXu77]

@yuntianlong2002
The operations HE supports are limited to addition and multiplication for most HE libraries (SEAL, PALISADE). Hence at current form, HE does not apply to our XGBoost examples:

• horizontal tree-based: server only perform concatenation of local tree models, since it is the model itself, there is no need to encrypt it (the model will be available to all anyways)
• horizontal histogram-based: server needs to boost a tree, which involves division and argmax operations, so HE does not function here
• vertical histogram-based: the only way to inject HE is to encrypt server->client message and let client perform G/H computation only (without finding local best splits, which involves division and argmax). This is currently not natively supported by XGBoost library, and we are investigating potential solutions.

So to your questions, HE currently does not work for XGBoost, but it can certainly work for any aggregation algorithms requiring addition / multiplication only - e.g. deep learning, regression model, k-means, etc.