support streaming messages

[bruffridge]

I think support for the new Assistants API streaming was added to deep-chat.

OvidijusParsiunas/deep-chat#137

[jackitaliano]

Yes, it was, but not there. That is simulating streaming. It still doesn't return anything until the whole message is received, but it then slowly "streams" the content. Results actually in a longer wait time.

However, actual streaming was added, but only in dev so far.
OvidijusParsiunas/deep-chat#155

Should be in a coming update, but will also result in some other changes that should be addressed due to deep-chat changes.

[jackitaliano]

Update on previous comment.

Tried out streaming functionality in dev, and it works well to stream text results. Some issues remain, so shouldn't be put into Bidara yet. I opened one issue so far regarding download links on deep-chat, and will look out for others.

Should wait on this issue until deep-chat's are resolved.

[jackitaliano]

Deep-chat maintainer is working on a solution to issue I mentioned. He said it would be non-trivial, so I'll keep tabs on that.

OvidijusParsiunas/deep-chat#157

[jackitaliano]

This issue seems to be resolved in deep-chat-dev v9.0.164; however, some other issues remain as mentioned #88, so I'll wait to put that in.

[bruffridge]

I was working on this a little, and one issue I encountered is I wasn't able to get the responseInterceptor to fix the image links since the function is called once for each token returned from streaming, so couldn't use the existing find and replace code. So moving on for now. I'm not sure how valuable streaming is anyways other than slightly better UX.

[jackitaliano]

Have fixed some of the issues with streaming on our end; however, one new bug presented itself on deep-chat's. Have an issue opened, but don't expect it to be resolved for another week due to the maintainer being OOO.