Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue of RAG results plots rendering #2036

Merged
merged 1 commit into from
Sep 30, 2024

Conversation

kevinmessiaen
Copy link
Member

Description

Bokeh >= 3.6 cause issue with rendering plots when saving to html file

How to reproduce

Run this code

import os

import giskard
from giskard.llm.client.openai import OpenAIClient

os.environ["OPENAI_API_KEY"] = "sk-..."

giskard.llm.set_llm_api("openai")
oc = OpenAIClient(model="gpt-4-turbo-preview")
giskard.llm.set_default_client(oc)

import pandas as pd
from giskard.rag import KnowledgeBase, generate_testset

# Load your data and initialize the KnowledgeBase
df = pd.read_csv("knowledge_base.csv")


knowledge_base = KnowledgeBase.from_pandas(df, columns=["q", "a"])

# Generate a testset with 10 questions & answers for each question types (this will take a while)
testset = generate_testset(
    knowledge_base,
    num_questions=10,
    language='en',  # optional, we'll auto detect if not provided
    agent_description="A customer support chatbot for company X", # helps generating better questions
)

# Save the generated testset
testset.save("my_testset.jsonl")

# You can easily load it back
from giskard.rag import QATestset

loaded_testset = QATestset.load("my_testset.jsonl")

# Convert it to a pandas dataframe
df = loaded_testset.to_pandas()

#%%
from giskard.rag import evaluate

# Wrap your RAG model
def get_answer_fn(question: str, history=None) -> str:
    """A function representing your RAG agent."""
    # Format appropriately the history for your RAG agent
    messages = history if history else []
    messages.append({"role": "user", "content": question})

    # Get the answer
    answer = "I don't know"  # could be langchain, llama_index, etc.

    return answer


# Run the evaluation and get a report
report = evaluate(get_answer_fn, testset=testset, knowledge_base=knowledge_base)

report.to_html('test.html')

Open the test.html generated file to see that the plot is not rendering when bokeh>3.6

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

 Bokeh >= 3.6 cause issue with rendering plots when saving to html file
Copy link

sonarcloud bot commented Sep 27, 2024

Copy link
Member

@henchaves henchaves left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with bokeh==3.6.0 and bokeh==3.4.3. Indeed, the plot generated using the version 3.6.0 was broken.

LGTM!

@kevinmessiaen kevinmessiaen merged commit 74553db into main Sep 30, 2024
17 checks passed
@kevinmessiaen kevinmessiaen deleted the fix/rag-plot-rendering branch September 30, 2024 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants