You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello, thanks for this amazing repo.
I am using the NALLM backend with a custom Streamlit app.
I have been facing issues with the websocket (mainly timeout issues and disconnections) implemented in api/src/main.py under @app.websocket("/text2text") endpoint.
Apologies if my question is basic but I am just trying to understand.
Is there a good reason for using websockets instead of HTTP endpoint?
My understanding is that websockets are mainly used for chat applications between users (where users don't know when they will be getting a new message). If the chat application is between a human and AI where a human always initiate the conversation by asking the question and the AI/ LLM responds with an answer, why do you use a websocket? I assume using HTTP for the "/text2text" endpoint could work as well?
The text was updated successfully, but these errors were encountered:
Hello, thanks for this amazing repo.
I am using the NALLM backend with a custom Streamlit app.
I have been facing issues with the websocket (mainly timeout issues and disconnections) implemented in
api/src/main.py
under@app.websocket("/text2text")
endpoint.Apologies if my question is basic but I am just trying to understand.
Is there a good reason for using websockets instead of HTTP endpoint?
My understanding is that websockets are mainly used for chat applications between users (where users don't know when they will be getting a new message). If the chat application is between a human and AI where a human always initiate the conversation by asking the question and the AI/ LLM responds with an answer, why do you use a websocket? I assume using HTTP for the
"/text2text"
endpoint could work as well?The text was updated successfully, but these errors were encountered: