save and import embeddings for faster answer generation

[n400peanuts]

Hello,

Thank you for the fantastic work on PaperQA. I’ve been able to use it to ask questions by providing over 100 papers as input, and I’ve been using only local models via Ollama. Everything is working well, but I’d like to know how I can avoid reloading the same files and retraining an embedding model each time I have a new Question.

Is there a way to save the vector store and load it later, so it can be used by the LLM to generate answers? I couldn't find documentation about that, I found something re- caching, but it's unclear to me how to use it properly. Can you provide some help?

best wishes

My code so far:

from paperqa import Settings, ask

question = "Can you list all climate-sensitive pathways that affect rotavirus incidence?"



model = "ollama/llama3.1" # "ollama/llama3.2" 
embedding = "ollama/nomic-embed-text" # "nomic-embed-text"

local_llm_config = {
    "model_list": [
        {
            "model_name": model,
            "litellm_params": {
                "model": model,
                "api_base": "http://localhost:11434",
            },
            "answer": {
                "evidence_k": 40,
                "evidence_detailed_citations": True,
                "evidence_summary_length": "about 100 words",
                "answer_max_sources": 5,
                "answer_length": "about 300 words, but can be longer",
                "max_concurrent_requests": 4,
                "answer_filter_extra_background": False
            }
            
        }
    ]
}

answer = ask(
    question,
    settings=Settings(
        llm=model,
        llm_config=local_llm_config,
        summary_llm=model,
        summary_llm_config=local_llm_config,
        agent=AgentSettings(
            agent_llm=model, agent_llm_config=local_llm_config
        ),
        embedding=embedding,
        paper_directory="papers/"
    ),
)

[dosubot]

Hey @n400peanuts 👋, I'm here to help you with any issues or questions you have about PaperQA. Let's work together to solve any bugs or address any questions you have.

To save and import embeddings in PaperQA and avoid reloading files and retraining an embedding model each time you have a new question, you can pickle the Docs object. This will cache the embeddings, allowing you to load them later without re-embedding your documents. Here's how you can do it:

import pickle
from paperqa import Docs

# Assuming you have already added documents to your Docs object
docs = Docs()
# Add documents to docs...

# Save the Docs object with embeddings
with open("my_docs.pkl", "wb") as f:
    pickle.dump(docs, f)

# Later, load the Docs object with embeddings
with open("my_docs.pkl", "rb") as f:
    docs = pickle.load(f)

By pickling the Docs object, you ensure that the embeddings are cached and can be reused, thus avoiding the need to reload files and retrain the embedding model for each new question ^[1].

To continue talking to Dosu, mention @dosu.

[n400peanuts]

ok let's do a step back, @dosu. Show me:

1. How to add my documents to the Docs object and how this is used in to generate the embeddings

2. Show me how to use the loaded embeddings in the code to generate the answer

[dosubot]

To add your documents to the Docs object and generate embeddings, you can use the add, add_file, or add_url methods, depending on the source of your document. Here's how you can do it:

1. Add a document from a file:

from pathlib import Path
from paperqa import Docs

docs = Docs()
file_path = Path("path/to/your/document.pdf")
docs.add(file_path)

2. Add a document from a URL:

from paperqa import Docs

docs = Docs()
url = "http://example.com/document.pdf"
docs.add_url(url)

3. Add a document from a binary file object:

from paperqa import Docs

docs = Docs()
with open("path/to/your/document.pdf", "rb") as file:
    docs.add_file(file)

Once your documents are added, embeddings are generated for them. To use these embeddings to generate an answer, you can use the query method:

query = "Your question here"
session = docs.query(query)
answer = session.formatted_answer
print(answer)

This process retrieves relevant texts using the embeddings and generates an answer based on your question ^[1][2].

To continue talking to Dosu, mention @dosu.

[n400peanuts]

@dosu, Show me how to set to use the following embedding model when generating the embeddings stored into Docs

embedding = "ollama/nomic-embed-text" # "nomic-embed-text"