Example `gpu_device_ids` for multi-model usage?

System Info

P4D (A100 40 GB x 8)

Who can help?

@juney-nvidia @byshiue

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

I want to use multi-model TP-4 x 2 setup. How do I set gpu_device_ids to [[0,1,2,3],[4,5,6,7]]
Could you please provide an example usage of gpu_device_ids?

Expected behavior

n/a

actual behavior

n/a

additional notes

n/a

How do I use an ensemble which load balances across these 2 model instances?