Open
Description
Feature Request
We can improve the AI requests with a few steps:
- Translation seems to be an easy task, we can use
meta-llama/Llama-3.1-8B-Instruct
, the request will be faster - add a
maxToken
to the call API, to do so we have to calculate the amount of token in our request, see https://github.com/belladoreai/llama-tokenizer-js or tiktoken (python) - we can improve the frontend message errors by displaying what is the real problem ?
- We can stream the response, by streaming the response we will be able to display content the the user gradually.
Metadata
Metadata
Assignees
Type
Projects
Status
No status