Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llm client base class and interface #9

Open
3 tasks
rlucas7 opened this issue Jan 6, 2025 · 0 comments
Open
3 tasks

llm client base class and interface #9

rlucas7 opened this issue Jan 6, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@rlucas7
Copy link
Owner

rlucas7 commented Jan 6, 2025

This task is done when a common interface is implemented for llm clients.
And that the existing code for gemini and openi (from this PR) is refactored to use the common interface.

One of the issues is that different llm providers have different results sets and different methods. Eg. gemini-1.5-pro has a bunch of harm probabilities whereas openai does not.

Also the key/value names may differ slightly for some of the things that are supported across clients.

llm client class need:

  • a method to generate relevances
  • a method to parse the output into a set of str, str key/vals
  • a method to get credentials

additional items likely to be desireable:

  • handling of retry logic
  • handling of prompts (for now keep the prompt orthogonal to client)
@rlucas7 rlucas7 added the enhancement New feature or request label Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant