Enhancement: Adding AI-powered routing support from Not Diamond

[acompa]

What features would you like to see added?

Hey there! I want to contribute an AI-powered routing integration to LibreChat which will be supported by Not Diamond.

Briefly, on us: we're offering AI-powered routing for LLM prompts, so that users can focus on workflows and prompts instead of integrating with the latest LLM releases or various provider libraries. You can watch Tomas walk through our SDK here, and find out more at our site.

More details

When a user submits their prompt to our modelSelect/ endpoint, we respond with the model which will produce the best prompt response. Notably, we do not proxy requests by default, and instead anticipate users will submit prompts to our recommended models. You can read more about this here.

Implementation plan

UI changes

Notably, we would add Not Diamond as a provider in the left-most dropdown, see picture 1 below.

When users select Not Diamond, they can also configure routing targets. I want to allow users to configure this through LibreChat's model dropdown as a multiselect list, see picture 2.

Some additional notes:

• Dropdown defined here
• Probably need MultiSelectDropDown to support multiple provider choices
• Want to add ND icons so that EndpointIcon can pick them up - should go here
  • Can probably add thumbs up + thumbs down in that directory too.
  • Maybe also ConvoIcon and ConvoIconURL for our logo?

LibreChat <> ND API

We'll also need to add our API as a LibreChat target, including API key support. Work required in api/server/services:

• Need to parse out ND API key in EndpointService
• Add model to ModelService
• Add endpoints to endpoint directory

Updating LibreChat provider components

Will need to add a ND component here in client/src/components/Input.

Next steps

• Does this implementation plan capture all of the elements requiring updates for UI, API and components?
• Are there other LibreChat elements which will need to change in order to support a new model provider like Not Diamond?

Thanks for taking the time to read this request!

Which components are impacted by your request?

UI, Endpoints

Pictures

Code of Conduct

• I agree to follow this project's Code of Conduct