调用vllm接口，最后一个token输出后一直收不到is_end，导致卡十秒钟 #3313
chenslcool
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
我有一个api接口，和vllm格式的返回不完全一致，我成功接入了模型，但是调用应用的时候，输出最后一个token后总要卡十秒钟才显示对话结束。接口的格式是这样的，请问这个接口是哪个字段不对呢？导致maxkb不能确认已经停止
Beta Was this translation helpful? Give feedback.
All reactions