Is there some guidance on using POC mode with docker-compose?

[mgd-hin]

Hi,
I have a project that runs well in simulator and POC mode (on one machine). I would now like to run this project on several different containers to try a more real-life scenario for my FL project. However, I am having trouble starting my clients. This is the docker-compose that I use:

The image I use is from the standard Dockerfile (i.e. what is created when I run the provision mode with the Docker builder component).
Paths and file names to the start.sh scripts are correct. The volume mounts to the container workspace folders are also successful. When I use docker-compose up, all containers start but exit immediately with exit code 0.

services:
  server:
    command:
    - ./workspace/startup/start.sh
    - localhost
    image: nvflare-service
    ports:
    - 8002:8002
    - 8003:8003
    volumes:
    - [absolute-path-to-host-dir]/poc/server:/workspace
    - nvflare_svc_persist:/tmp/nvflare/
 
  site-1:
    command:
    - ./workspace/startup/start.sh
    - server:8002:8003
    - client-1
    image: nvflare-service
    volumes:
    -[absolute-path-to-host-dir]/poc/site-1:/workspace
    depends_on:
    - server
 
  site-2:
    command:
    - ./workspace/startup/start.sh
    - server:8002:8003
    - client-2
    image: nvflare-service
    volumes:
    - [absolute-path-to-host-dir]/poc/site-2:/workspace
    depends_on:
    - server
 
  admin:
    command:
    - ./workspace/startup/fl_admin.sh
    image: nvflare-service
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    volumes:
    -[absolute-path-to-host-dir]/poc/admin:/workspace
    depends_on:
    - server
    
volumes:
  nvflare_svc_persist: null

The POC Readme says to use startup/start.sh server_host(name or IP). I have tried providing localhost:8002:8003, [server-container-name]: 8002: 8003, and assigning IP addresses to every container in the compose network. However, I am not sure if this is a connectivity issue first and foremost, because even if a client cannot find a server to connect to, I assume the service would keep running on that client for more than a few seconds.

Is there anything obvious that I am missing or doing wrong from NVFlare side (if this is only a docker struggle, I understand this is the wrong place to ask) ?
Is there an example or a tutorial for running POC on different containers (ideally with docker-compose)?

[YuanTingHsieh]

Hi @mgd-hin ,

This is both a docker problem and an NVFlare problem.

So the "start.sh" of NVFlare is actually running in background as daemon process.

If you use docker to run a background process it will exit right away.
You can search online for how to run docker with background process.
(one way is to use nohup)

[chesterxgchen]

you can use production model ( non-HA, secure, localhost) as well. our internal testing team is using this mode. client/servers are docker containers

[chesterxgchen]

Not sure if this is an issue or not. In 2.4.0 release candidate we improved the POC command, it is acting as secure production mode. It also allow one to simulate multiple clients/server on the same host using detached docker.