Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAI API Upgrade to version >= 1.0 and added vLLM server support #122

Closed
wants to merge 2 commits into from

Conversation

taisazero
Copy link
Contributor

@taisazero taisazero commented Mar 2, 2024

🤔 What is the nature of this change?

  • New feature
  • Website, documentation, demo improvements
  • Refactor code or style

🔗 Related Issue

#110

💡 Background or solution

  • Upgraded to OpenAI API 1.0
  • Added support for vLLM endpoints.
  • Refactored LLM config to allow for LLMs served through a remote vLLM endpoint, locally hosted language models through FSChat, Azure endpoints, and OpenAI endpoints.
  • Added a default OpenAI client that's based on whichever endpoint is available (Azure or OpenAI) that's shared across the repository.

Sorry, I realized that was another PR on upgrading to the 1.0 OpenAI client: #111 after I finished implementing this. Please discard this if the other migration PR is completed. Hope this helps!

@chenweize1998
Copy link
Collaborator

Huge thanks! Will look into it these days

@yushengsu-thu yushengsu-thu self-requested a review March 21, 2024 20:38
@yushengsu-thu yushengsu-thu marked this pull request as draft March 21, 2024 20:38
@chenweize1998 chenweize1998 marked this pull request as ready for review March 24, 2024 08:37
@chenweize1998 chenweize1998 mentioned this pull request Mar 24, 2024
3 tasks
@chenweize1998
Copy link
Collaborator

Hi! I've made some minor modifications based on your PR, and have merged it with #127. Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants