Adding chat limits

[haydenkong]

Just got a idea to put chat limits, cause im sharing my hosted version with some of my family memebers and I don't want them to use all my API costs. Can someone please help? I want to implement something on the supabase table like: level1 and level2 and i can manage who can access it more and the chat limits (token counts).

Thankyou so much.

[spammenotinoz]

Hi, please refer to #1527

For myself I found it wasn't the # of chats\messages but the length of a specific conversation. So I did a quick code update to limit the conversation history. ChatBot-UI is API based, as API's have no memory\history, each call in a conversation repeats the entire conversation. So my family kids (were still using the original conversation), when I looked at the network for a single prompt of "what model is this?", the token count was 300k for an hour of usage. So that's when I did a deep dive, and understood how it works under the covers each prompt was a few k of tokens. (Thanks to CloudFlare API gateway, for helping me understand)
You can start with educating your family.
I don't have the coding knowledge or I would have summarised the history using a cheaper model, than discarding it.

I also changed the default model to Claude Haiku (insanely fast and cheap, you get $5 of initial credit and I am still on those). But also GPT 3.5-Turbo is fast and cheap, just make that the default.

If your users use, attachments (embeddings) then costs really compound quickly. I fixed this by @fkesheh's pull request
#1518

Result is my usage dropped from $15 per day to just under a $1 per day.

[haydenkong]

Do you know how to implement it? I am quite confused on how to implement it to chatbotui