Skip to content

Commit

Permalink
Update fastRAG 2.0 information (#102)
Browse files Browse the repository at this point in the history 
* update fastrag integration

* update link

* small update
  • Loading branch information
@peteriz
peteriz authored Jan 12, 2024
1 parent 44356e4 commit 47d9792
Showing 1 changed file with 85 additions and 23 deletions.
108 changes: 85 additions & 23 deletions integrations/fastrag.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: integration
name: fastRAG
description: A research framework designed to facilitate the building of retrieval augmented generative pipelines.
description: fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines
authors:
- name: Intel Labs
socials:
Expand All @@ -13,46 +13,108 @@ report_issue: https://github.com/IntelLabs/fastRAG/issues
logo: /logos/intel-labs.png
---

fast**RAG** is a research framework designed to facilitate the building of retrieval augmented generative pipelines. Its main goal is to make retrieval augmented generation as efficient as possible through the use of state-of-the-art and efficient retrieval and generative models. The framework includes a variety of sparse and dense retrieval models, as well as different extractive and generative information processing models. fastRAG aims to provide researchers and developers with a comprehensive tool-set for exploring and advancing the field of retrieval augmented generation.
fast**RAG** is a research framework, that extends [Haystack](https://github.com/deepset-ai/haystack), with abilities to build ***efficient*** and ***optimized*** retrieval augmented generative pipelines (with emphasis on ***Intel hardware***), incorporating state-of-the-art LLMs and Information Retrieval modules.

It includes custom nodes such as:
- Image Generators
- Knoweldge Graph Creator
- Document Shapers
- Reader with FiD implementation
- Efficient document vector store (PLAID)
- Benchmarking scripts
## Key Features

## Installation
- **Optimized RAG**: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
- **Optimized for Intel Hardware**: Leverage [Intel extensions for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch), [🤗 Optimum Intel](https://github.com/huggingface/optimum-intel) and [🤗 Optimum-Habana](https://github.com/huggingface/optimum-habana) for *running as optimal as possible* on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
- **Customizable**: fastRAG is built using [Haystack](https://github.com/deepset-ai/haystack) and HuggingFace. All of fastRAG's components are 100% Haystack compatible.

## :rocket: Components

For a brief overview of the various unique components in fastRAG refer to the [Components Overview]([components.md](https://github.com/IntelLabs/fastRAG/blob/main/components.md)) page.

<div class="tg-wrap" align="center">
<table style="undefined;table-layout: fixed; width: 600px; text-align: center;">
<colgroup>
<!-- <col style="width: 229px"> -->
<!-- <col style="width: 238px"> -->
</colgroup>
<tbody>
<tr>
<td colspan="2"><strong><em>LLM Backends</em></td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#fastrag-running-llms-with-habana-gaudi-(dl1)-and-gaudi-2">Intel Gaudi Accelerators</a></td>
<td><em>Running LLMs on Gaudi 2</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#fastrag-running-llms-with-onnx-runtime">ONNX Runtime</a></td>
<td><em>Running LLMs with optimized ONNX-runtime</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#fastrag-running-rag-pipelines-with-llms-on-a-llama-cpp-backend">Llama-CPP</a></td>
<td><em>Running RAG Pipelines with LLMs on a Llama CPP backend</td>
</tr>
<tr>
<td colspan="2"><strong><em>Optimized Components</em></td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/scripts/optimizations/embedders/README.md">Embedders</a></td>
<td>Optimized int8 bi-encoders</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/scripts/optimizations/reranker_quantization/quantization.md">Rankers</a></td>
<td>Optimized/sparse cross-encoders</td>
</tr>
<tr>
<td colspan="2"><strong><em>RAG-efficient Components</em></td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#ColBERT-v2-with-PLAID-Engine">ColBERT</a></td>
<td>Token-based late interaction</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#Fusion-In-Decoder">Fusion-in-Decoder (FiD)</a></td>
<td>Generative multi-document encoder-decoder</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#REPLUG">REPLUG</a></td>
<td>Improved multi-document decoder</td>
</tr>
<tr>
<td><a href="https://github.com/IntelLabs/fastRAG/blob/main/components.md#ColBERT-v2-with-PLAID-Engine">PLAID</a></td>
<td>Incredibly efficient indexing engine</td>
</tr>
</tbody>
</table></div>

## :round_pushpin: Installation

Preliminary requirements:

- Python 3.8+
- PyTorch
- **Python** 3.8 or higher.
- **PyTorch** 2.0 or higher.

To set up the software, clone the project and run the following, preferably in a newly created virtual environment:


In a new virtual environment, run:

```bash
git clone https://github.com/IntelLabs/fastRAG.git
cd fastrag
pip install .
```

There are various dependencies, based on usage:
There are several dependencies to consider, depending on your specific usage:

```bash
# Additional engines/components
pip install .[faiss-cpu] # CPU-based Faiss
pip install .[faiss-gpu] # GPU-based Faiss
pip install .[qdrant] # Qdrant support
pip install libs/colbert # ColBERT/PLAID indexing engine
pip install .[image-generation] # Stable diffusion library
pip install .[knowledge_graph] # spacy and KG libraries

# REST API + UI
pip install .[intel] # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic] # Support for ElasticSearch store
pip install .[qdrant] # Support for Qdrant store
pip install libs/colbert # Indexing engine for ColBERT/PLAID
pip install .[faiss-cpu] # CPU-based Faiss library
pip install .[faiss-gpu] # GPU-based Faiss library
pip install .[knowledge_graph] # Libraries for working with spacy and KG

# User interface (for demos)
pip install .[ui]

# Benchmarking
pip install .[benchmark]

# Dev tools
# Development tools
pip install .[dev]
```

0 comments on commit 47d9792

Please sign in to comment.