Metadata retrieval

[ZeyadAttia-Tek]

I am implementing a RAG agent using PGVector and the retrieval tool. I inserted the data in PGVector and want the AI model to retrieve it using metadata. aka. the model will search for a ticket/release name or id and that should be retrieved using metadata. But I cant find a way to filter using metadata. Any ideas?

[dosubot[bot]]

Hey @ZeyadAttia-Tek! 👋 I'm here to help you with any bugs, questions, or contributions while you wait for a human maintainer. Let's tackle this together!

To filter and retrieve data using metadata in a RAG agent implemented with PGVector and the retrieval tool, you can use the filter input parameter in the VectaraRagComponent. This parameter allows you to specify a filter string to narrow the search based on metadata attributes. Here's a basic setup:

from langflow.custom import Component
from langflow.io import MessageTextInput

class VectaraRagComponent(Component):
    # Other configurations...

    inputs = [
        # Other inputs...
        MessageTextInput(
            name="filter",
            display_name="Metadata Filters",
            value="",
            advanced=True,
            info="The filter string to narrow the search to according to metadata attributes.",
        ),
        # Other inputs...
    ]

    def generate_response(self) -> Message:
        # Other configurations...

        config = VectaraQueryConfig(
            lambda_val=self.lexical_interpolation,
            filter=self.filter,  # Use the filter here
            summary_config=summary_config,
            rerank_config=rerank_config,
        )
        rag = vectara.as_rag(config)
        response = rag.invoke(self.search_query, config={"callbacks": self.get_langchain_callbacks()})

        text_output = response["answer"]

        return Message(text=text_output)

Set the filter parameter with the appropriate metadata filter string to retrieve the desired data ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[ZeyadAttia-Tek]

@dosu

I dont want to use Vectara. Just plain python, pgvector, and the Agent component. I can also create custom components.

[dosubot[bot]]

To filter and retrieve data using metadata with plain Python, PGVector, and the Agent component in Langflow, you can create a custom component that interacts with PGVector. Here's a general approach:

1. Create a Custom Component: Define a custom component that connects to your PGVector instance and performs queries based on metadata.

2. Build the Vector Store: Use the build method to create a vector store from your PGVector instance. This method can handle inputs and create a vector store using the provided connection string and collection name.

3. Query with Metadata: Implement a method to query the vector store using metadata. You can use SQL queries or PGVector's built-in methods to filter results based on metadata attributes.

Here's a simplified example of how you might set up such a component:

from langflow.custom import Component
from langflow.schema import Data
from pgvector import PGVector

class MetadataFilterComponent(Component):
    display_name = "Metadata Filter"
    description = "Filters data using metadata in PGVector."

    def build(self, pg_server_url: str, collection_name: str):
        # Connect to PGVector
        self.vector_store = PGVector(connection_string=pg_server_url, collection_name=collection_name)

    def query_with_metadata(self, metadata_filter: str) -> Data:
        # Perform a query using metadata
        query = f"SELECT * FROM {self.vector_store.collection_name} WHERE metadata_column = '{metadata_filter}'"
        results = self.vector_store.execute_query(query)
        return Data(value=results)

# Example usage
component = MetadataFilterComponent()
component.build(pg_server_url="your_pg_server_url", collection_name="your_collection_name")
results = component.query_with_metadata(metadata_filter="your_metadata_value")

This example demonstrates how to set up a custom component to query PGVector using metadata. Adjust the query logic to match your specific metadata structure and retrieval needs.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}