diff --git a/content/talks/2024-04-22-odsc-east-2024.md b/content/talks/2024-04-22-odsc-east-2024.md deleted file mode 100644 index a35c9e67..00000000 --- a/content/talks/2024-04-22-odsc-east-2024.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: "[UPCOMING] ODSC East 2024: RAG, the bad parts (and the good!)" -date: 2024-02-20 -author: "ZanSara" -tags: [LLM, NLP, Python, AI, RAG, "ODSC East", ODSC, Retrieval, Generation] -featuredImage: "/talks/2024-04-22-odsc-east-2024.png" -externalLink: https://odsc.com/speakers/rag-the-bad-parts-and-the-good-building-a-deeper-understanding-of-this-hot-llm-paradigms-weaknesses-strengths-and-limitations/ ---- diff --git a/content/talks/2024-04-25-odsc-east-rag.md b/content/talks/2024-04-25-odsc-east-rag.md new file mode 100644 index 00000000..f7f4c9af --- /dev/null +++ b/content/talks/2024-04-25-odsc-east-rag.md @@ -0,0 +1,24 @@ +--- +title: "ODSC East 2024: RAG, the bad parts (and the good!)" +date: 2024-04-25 +author: "ZanSara" +tags: [LLM, NLP, Python, AI, RAG, "ODSC East", ODSC, Retrieval, Generation, Evaluation, Haystack] +featuredImage: "/talks/2024-04-25-odsc-east-rag.png" +--- + +[Announcement](https://odsc.com/speakers/rag-the-bad-parts-and-the-good-building-a-deeper-understanding-of-this-hot-llm-paradigms-weaknesses-strengths-and-limitations/), [slides](https://drive.google.com/file/d/19EDFCqOiAo9Cvx5fxx6Wq1Z-EoMKwxbs/view?usp=sharing) + +--- + +In this talk at [ODSC East 2024](https://odsc.com/boston/) I talk about RAG: how it works, how it fails, and how to evaluate its performance objectively. I give an overview of some useful open-source tools for RAG evalution and how to use them with [Haystack](https://haystack.deepset.ai/?utm_campaign=odsc-east), and then give you some ideas on how to expand your RAG architecture further than a simple two-step process. + +Some resources mentioned in the talk: + +- Haystack: open-source LLM framework for RAG and beyond: [https://haystack.deepset.ai/](https://haystack.deepset.ai/?utm_campaign=odsc-east) +- Build and evaluate RAG with Haystack: [https://haystack.deepset.ai/tutorials/35_model_based_evaluation_of_rag_pipelines](https://haystack.deepset.ai/tutorials/35_model_based_evaluation_of_rag_pipelines/?utm_campaign=odsc-east) +- Evaluating LLMs with UpTrain: https://docs.uptrain.ai/getting-started/introduction +- Evaluating RAG end-to-end with RAGAS: https://docs.ragas.io/en/latest/ +- Semantic Answer Similarity (SAS) metric: https://docs.ragas.io/en/latest/concepts/metrics/semantic_similarity.html +- Answer Correctness metric: https://docs.ragas.io/en/latest/concepts/metrics/answer_correctness.html + +Plus, shout-out to a very interesting LLM evaluation library I discovered at ODSC: [continuous-eval](https://docs.relari.ai/v0.3). Worth checking out especially if SAS or answer correctness are too vague and high level for your domain. \ No newline at end of file diff --git a/static/talks/2024-04-25-odsc-east-rag.png b/static/talks/2024-04-25-odsc-east-rag.png new file mode 100644 index 00000000..cc036b56 Binary files /dev/null and b/static/talks/2024-04-25-odsc-east-rag.png differ