Skip to content

⚡️Improve AI requests #694

Open
Open
@AntoLC

Description

@AntoLC

Feature Request

We can improve the AI requests with a few steps:

  • Translation seems to be an easy task, we can use meta-llama/Llama-3.1-8B-Instruct, the request will be faster
  • add a maxToken to the call API, to do so we have to calculate the amount of token in our request, see https://github.com/belladoreai/llama-tokenizer-js or tiktoken (python)
  • we can improve the frontend message errors by displaying what is the real problem ?
  • We can stream the response, by streaming the response we will be able to display content the the user gradually.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions