Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenAI v1.x.x compatibility to embedding models and api base #641

Merged
merged 3 commits into from
Sep 3, 2024
Merged

Add OpenAI v1.x.x compatibility to embedding models and api base #641

merged 3 commits into from
Sep 3, 2024

Conversation

HollowMan6
Copy link

@HollowMan6 HollowMan6 commented Sep 1, 2024

Test code

import time, os

from gptcache import cache, Config
from gptcache.manager import manager_factory
from gptcache.embedding import OpenAI as EmbeddingModel
from gptcache.processor.post import temperature_softmax
from gptcache.similarity_evaluation.distance import SearchDistanceEvaluation
from gptcache.adapter import openai
from openai import OpenAI

os.environ["OPENAI_API_KEY"] = "sk-xxxxxx"
os.environ["OPENAI_BASE_URL"] = "https://api.openai.com/v1"

cache.set_openai_key()

client = OpenAI()

embedding_model = EmbeddingModel(model="text-embedding-3-large", client=client)
data_manager = manager_factory("sqlite,faiss", vector_params={"dimension": embedding_model.dimension})

cache.init(
    embedding_func=embedding_model.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(),
    post_process_messages_func=temperature_softmax
    )
cache.config = Config(similarity_threshold=0.9)

def openai_chat():
  question = 'what is github'
  start = time.time()
  answer = openai.cache_openai_chat_complete(
        client,
        model='gpt-4o',
        messages=[
          {
              'role': 'user',
              'content': question
          }
        ],
      )
  print(answer)
  print(time.time() - start)

openai_chat()
openai_chat()

@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: HollowMan6
To complete the pull request process, please assign cxie after the PR has been reviewed.
You can assign the PR to them by writing /assign @cxie in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot
Copy link
Collaborator

Welcome @HollowMan6! It looks like this is your first PR to zilliztech/GPTCache 🎉

@mergify mergify bot added dco-passed and removed needs-dco labels Sep 1, 2024
- Support passing the client as the parameter
- In OpenAI v1.x.x, `api_base` has been renamed into `base_url`
openai/openai-python#1051 (comment)

Signed-off-by: Hollow Man <[email protected]>
@HollowMan6
Copy link
Author

The unittest failure doesn't look like it's introduced by this PR https://github.com/zilliztech/GPTCache/actions/runs/10654173032/job/29542114701

ModuleNotFoundError: No module named 'langchain_community'

@SimFG SimFG merged commit 17646e1 into zilliztech:dev Sep 3, 2024
4 of 6 checks passed
@oussamaJmaaa
Copy link

oussamaJmaaa commented Oct 18, 2024

any updates on this ? only openai v0.28.0 worked for me , if i tried updating openai i face errors .

@SimFG
Copy link
Collaborator

SimFG commented Oct 18, 2024

This part of the compatibility may not be easy to handle, you can refer to my previous issue. #613

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants