Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrieval Agent not providing quality queries. #30

Open
lucasbarrosomk6 opened this issue Apr 12, 2024 · 0 comments
Open

Retrieval Agent not providing quality queries. #30

lucasbarrosomk6 opened this issue Apr 12, 2024 · 0 comments

Comments

@lucasbarrosomk6
Copy link

Hey guys, I picked up this project this morning and got everything set up (had to learn a bunch about supabase in order to get started) but I have a problem, My use case is to create an agent that will be given a task and use tools with access to different tables in supabase to gain a the information needed to answer the question.

First test:
uploaded a write up on "Brand Tracking" to the table:
image
As you can see, the data is filled with the term "Brand Tracking"

Then I asked it about Brand Tracking
image

it seemed to give a pretty lackluster query, is this a normal query for this tool?

Here is the related code for the tool and the agent.

` const chatModel = new ChatOpenAI({
modelName: "gpt-3.5-turbo-1106",
temperature: 0.9,
// IMPORTANT: Must "streaming: true" on OpenAI to enable final output streaming below.
streaming: true,
});

const productKnowledgeStore = new SupabaseVectorStore(
  new OpenAIEmbeddings(),
  {
    client,
    tableName: "productknowledge",
    queryName: "match_documents",
  },
);

const productKnowledgeRetriever = productKnowledgeStore.asRetriever();

the SQL
CREATE FUNCTION match_documents (

query_embedding VECTOR(1536),
match_threshold FLOAT
) RETURNS SETOF documents AS $$
BEGIN
RETURN QUERY
SELECT *
FROM documents
WHERE documents.embedding <#> query_embedding < -match_threshold
ORDER BY documents.embedding <#> query_embedding;
END;
$$ LANGUAGE plpgsql;

const prompt = ChatPromptTemplate.fromMessages([
  ["system", AGENT_SYSTEM_TEMPLATE],
  ...previousMessages.map((msg: { content: string }) => [
    "system",
    msg.content,
  ]), // Add previous context to the system message
  [
    "human",
    `Please formulate a detailed query aimed to provide the human with information on the following input: {input}`,
  ],
  new MessagesPlaceholder("agent_scratchpad"),
]);

const agent = await createToolCallingAgent({
  llm: chatModel,
  tools: [productKnowledge, companyKnowledge],
  prompt,
});

const agentExecutor = new AgentExecutor({
  agent,
  tools: [productKnowledge, companyKnowledge],
  // Set this if you want to receive all intermediate steps in the output of .invoke().
  returnIntermediateSteps,
});

const result = await agentExecutor.invoke({
input: currentMessageContent,
chat_history: previousMessages,
recursionLimit: 5,
});`

I am positive that something is wrong here, I feel like it may be the SQL query or the query text created by the agent. When I make talk to docs apps without supabase I get pretty great results, I feel like I need to tweak something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant