Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

why use websocket #93

Open
NoraEr opened this issue Sep 21, 2024 · 0 comments
Open

why use websocket #93

NoraEr opened this issue Sep 21, 2024 · 0 comments

Comments

@NoraEr
Copy link

NoraEr commented Sep 21, 2024

Hello, thanks for this amazing repo.
I am using the NALLM backend with a custom Streamlit app.
I have been facing issues with the websocket (mainly timeout issues and disconnections) implemented in api/src/main.py under @app.websocket("/text2text") endpoint.
Apologies if my question is basic but I am just trying to understand.

Is there a good reason for using websockets instead of HTTP endpoint?
My understanding is that websockets are mainly used for chat applications between users (where users don't know when they will be getting a new message). If the chat application is between a human and AI where a human always initiate the conversation by asking the question and the AI/ LLM responds with an answer, why do you use a websocket? I assume using HTTP for the "/text2text" endpoint could work as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant