fix: ID Mismatch Error in VectorDB During Evaluation #1033

[e7217]

description
Hello

I am suggesting some code changes to address issue #1033. The error occurs when an item in the vectordb is searched, but its ID does not match the ID of the raw_doc corpus. I think the retriever aims to retrieve the item with the highest score. To address this, I have added a key for the content. While this change may require additional storage capacity for the vectordb, it's similar to how Langchain uses a page_content key.

I have modified some code, but I have only referred to the documentation and have not run the code in practice, so there may be errors.

I appreciate your review. Thank you.

references

• chroma:
  • https://docs.trychroma.com/getting-started#5.-query-the-collection
  • https://docs.trychroma.com/reference/py-collection#query
• couchbase:
  • https://docs.couchbase.com/python-sdk/current/howtos/full-text-searching-with-sdk.html#search-options
• pinecode:
  • https://docs.pinecone.io/reference/api/2024-10/data-plane/query
  • https://github.com/langchain-ai/langchain/blob/8ec1c72e0355713fd20f8fc89cdf72d37aaf61f5/libs/community/langchain_community/vectorstores/pinecone.py#L101
• qdrant:
  • https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/vectorstores/qdrant.py#L1998
  • https://python-client.qdrant.tech/qdrant_client.http.models.models
  • https://qdrant.tech/documentation/concepts/search/#payload-and-vector-in-the-result
• weaviate:
  • https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/vectorstores/weaviate.py#L226
  • https://weaviate.io/developers/academy/py/starter_custom_vectors/object_searches/vector

[e7217]

This PR may not fully align with your intentions in autorag. I tried to consider as many cases as possible, but there may be aspects you have been concerned about that I am unaware of. I understand that it might not be approved, but I would appreciate any feedback you can provide. Thank you.

[vkehfdl1]

@e7217 Thank you for the PR! And apologize for the late review.
I will look through it, and will make some changes if needed. Thank you

[vkehfdl1]

@e7217 Actually we discussed about the structure that do not use corpus_df at all for the AutoRAG structure.
@bwook00 I need your opinion about it. I think we have to discuss about it.

Pros

1. There will be no such things that "no doc_id in vectorddb"
2. Managing the corpus will be more easier.
3. Someone only have vectorDB can start AutoRAG easily.

Cons

1. It will be difficult to synchronize different vector DBs. => Just use one DB? What about embedding model?
2. The inconsistent result of the one project when vector DB is changed. => Give up the precise experiments?
3. Managing the right retrieval_gt might be harder.