-
|
How to deploy a copy of the model on different GPUs and tell the worker to schedule them? |
Beta Was this translation helpful? Give feedback.
Answered by
lkevinzc
May 23, 2022
Replies: 1 comment
-
|
Hi @pl8787 ! You could pass See https://mosecorg.github.io/mosec/example/env/ for an example. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
kemingy
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi @pl8787 ! You could pass
CUDA_VISIBLE_DEVICESenvironment variable to the worker processes, so that each worker will occupy the specified GPU device and run in parallel to serve the requests.See https://mosecorg.github.io/mosec/example/env/ for an example.