Generative AI: Retrieval Augmented Generation (RAG)

Guest lecture on Retrieval-Augmented Generation (RAG) for DS 3891 Generative AI

Intro

RAG was first introduced in Meta's 2020 paper titled "Retrieval Augmented Generation for Knowledge-Intensive Tasks" Since then, it has become a widely popular method to enhance generative AI reponses. This method relies on giving a corpus of information to a large language model (LLM), which then uses that information to generate its response. RAG is best for situations in which LLM users want fact-based answers or answers with sources from a specific collection.

Using this Repo

Use this repo to access a tutorial notebook on implementing RAG using the Python library Langchain.

Langchain Documentation

You can find the Langchain documentation used to create this tutorial below. The suggested order for the documentation is as follows:

1. Document Loading
2. Text Splitters
3. Vector Stores
4. Retrievers

Resources

• Langchain cookbook on RAG
• Langchain RAG Quickstart
• RAG Guide from DataStax
• Semantic Text Splitting
• Adding Citations