Q: Continuous Batching without Turbomind? #2029
Answered
by
zhyncs
SinanAkkoyun
asked this question in
Q&A
-
|
Hello! |
Beta Was this translation helpful? Give feedback.
Answered by
zhyncs
Jul 15, 2024
Replies: 1 comment 1 reply
-
|
PyTorch Engine also supports continuous batching :) cc @grimoire |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Great, thank you! :) |
Beta Was this translation helpful? Give feedback.
Answer selected by
SinanAkkoyun
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
PyTorch Engine also supports continuous batching :) cc @grimoire