Support more EvalClient formats based on LLM APIs

[liwii]

There are more and more services in which you can call LLMs with HTTP requests. Integrating those services into LangCheck in the form of EvalClient could expand the userbase of LangCheck.

I did a quick research of services we can potentially support:

LLM hosting services

• Vertex AI
• Support Azure AI Foundry #177
• Groq
• Amazon Bedrock
• Amazon Sagemaker
• OpenRouter
• Hugging Face Inference API
• Sambanova Cloud
• Fireworks AI
• together.ai
• Replicate
• Hyperbolic

APIs provided by model developers

• DeepSeek
• cohere
• Mistral AI
• Grok

(I personally feel items listed on top are more important, but it is very subjective and unreliable)

We can split the work to investigate the API spec & implement the EvalClient for each service!!

[liwii]

@matchcase

If you are interested, it would be a good project to pick one of the services and implement an EvalClient for that!!

Here is the reference implementation of GeminiEvalClient: #113

[kennysong]

Should we add Vertex AI near the top of the list as well?

[liwii]

Sounds reasonable, added!!

[matchcase]

By the way, does langcheck support OpenRouter? It is a relatively popular platform for querying LLMs.

[liwii]

Wow I didn't know this service, but it looks pretty cool!!
Added OpenRouter to the list as well, I might take that in the future, but you can also create a sub-issue and work on that if you want!!