Support for other Chat LLM

[othmanelhoufi]

Hello everyone,

The UI is so good and intuitive, but I think it can be better if it supports different Chat LLM (mainly open source, but also for the upcoming Anthropic 100k model).
I know each LLM provider has their own REST API, or if it's an open source LLM they are deployed by users manually in Azure or other providers.
Although, if this interface can centralize access to these Chat LLM by giving an endpoint link and dynamically calling the right API it's going to be a game changer.
Imagine that you can utilize the LLM of your choice, and with the integration of Plugins you get to experiment with any LLM through web search or other.

Let me know what you think 💬

[hzhangxyz]

it would be better to create a proxy/wrapper server to modify/forward the request, it is better to be an individual project for low coupling?

[hzhangxyz]

Btw, Fastchat from lmsys has their own gpt compatible api.

[ttimasdf]

https://github.com/songquanpeng/one-api use an self hosted API proxy.