lmdeploy支持DP吗？
#2780
Replies: 1 comment 3 replies
-
|
现在需要起8个独立的服务 |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
有支持DP的打算吗？ |
Beta Was this translation helpful? Give feedback.
-
|
如果不要求服务的话，可以自行写一段代码调用 pipeline 接口。然后用 torchrun 实现 dp |
Beta Was this translation helpful? Give feedback.
-
|
这样可行，但是感觉不太优雅。 |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
8张A100用于7B推理，只能起8个独立的服务吗还是支持DP=8？
Beta Was this translation helpful? Give feedback.
All reactions