Langchain integration

[mathieuisabel]

How can RouteLLM be used with LangChain?

[iojw]

Hi! Do you have a specific use case in mind? We don't currently have a LangChain integration but I assume RouteLLM could be used in-place of model calls, where instead of just using GPT-4, you would have a router that routed between two models and returns the response.

[mathieuisabel]

I would think more as a drop-in replacement for the models. i.e. right now you can set llm as either ChatOpenAI or ChatAnthropic and you don't have to change the definition of the chain itself. i.e.
llm = ChatOpenAI( model_name=model_name, temperature=0.0, openai_api_key=os.getenv('OpenAI__ApiKey'), max_retries=3, request_timeout=240 )
or
llm = ChatAnthropic( model_name=model_name, temperature=0.0, api_key=os.getenv('Anthropic__ApiKey'), max_retries=3, timeout=10 )
and then:

This one touches on function calling see #20 as a consideration for drop-in replacement

chain = prompt | llm.bind_tools([structure_segment_function]) response=chain.invoke(chain_payload)

[iojw]

Yes exactly, a drop-in replacement for models is what I had in mind. We'd be happy to accept any contributions here!