-
Notifications
You must be signed in to change notification settings - Fork 39
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
24 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,57 +1,60 @@ | ||
# vlite | ||
A blazing fast, lightweight, and simple vector database made with numpy and llama.cpp in ~1k lines of code. | ||
|
||
[vlite Documentation](docs.md) | ||
A blazing fast, lightweight, and simple vector database made with numpy and llama.cpp in ~1k lines of code. | ||
|
||
 | ||
|
||
## Features | ||
|
||
## features | ||
- 🔥 *fastest* vector db retrieval with binary embeddings and int8 rescoring | ||
- 🏎️ accelerated embedding generation with llama.cpp | ||
- 🍪 OMOM (pronounced "om-nom") file format, a novel abstraction for storing user context similar to browser cookies | ||
- injest text, PDF, CSV, PPTX, and webpages | ||
- batteries included chunking, metadata filtering, PDF OCR support for extracting text from scanned PDFs | ||
- **over 77.95% faster than Chroma on indexing, and 422% faster on retrieval** | ||
- 🔥 *Fastest* vector db retrieval with binary embeddings | ||
- 🔋 Made for RAG -- with embedding generation baked in | ||
- 🍪 CTX (context) file format, a novel abstraction for storing user context similar to browser cookies | ||
- Ingest text, PDF, CSV, PPTX, and webpages | ||
- Chunking, metadata filtering, PDF OCR support for extracting text from scanned PDFs | ||
- **Over 77.95% faster than Chroma on indexing, and 422% faster on retrieval** | ||
|
||
## Installation | ||
|
||
## installation | ||
```bash | ||
pip install vlite | ||
``` | ||
|
||
### installation with PDF OCR Support | ||
### Installation with PDF OCR Support | ||
|
||
To enable PDF OCR support (with [surya](https://github.com/VikParuchuri/surya)), install the `vlite[ocr]` extra: | ||
|
||
```bash | ||
pip install vlite[ocr] | ||
``` | ||
|
||
## usage | ||
## Usage | ||
|
||
```python | ||
from vlite import VLite | ||
from vlite.utils import process_pdf | ||
|
||
vdb = VLite() | ||
|
||
vdb.add("hello world", metadata={"artist": "adele"} | ||
|
||
vdb.add("hello world", metadata={"artist": "adele"}) | ||
vdb.add(process_pdf("attention-is-all-you-need.pdf", use_ocr=True)) | ||
|
||
results = vdb.retrieve("how do transformers work?") | ||
|
||
print(results) | ||
``` | ||
|
||
## about | ||
## About | ||
|
||
vlite is a vector database built for agents, ChatGPT Plugins, and other AI apps that need a fast and simple database to store vectors. It was developed to support the billions of embeddings generated, indexed, and sorted with [ChatWith+ ChatGPT Plugins](https://plugins.sdan.io/), which run for millions of users. Most vector databases either repeatedly crashed on a daily basis or were too expensive for the high throughput required. | ||
|
||
vlite introduces the OMOM (pronounced "om-nom") file format, which acts like a browser cookie for user embeddings, providing efficient storage, retrieval of embeddings, composability, portability, and user context. | ||
vlite introduces the CTX file format, which acts like a browser cookie for user embeddings, providing efficient storage, retrieval of embeddings, composability, portability, and user context. | ||
|
||
Under the hood, vlite uses Pytorch's Metal Performance Shader(MPS) for accelerated embedding generation and defaults to binary embeddings for the fastest retrieval in memory vector databases in Python. It beats Chroma on all metrics retrieval/indexing, specifically 77.95% faster indexing speed compared to Chroma. | ||
|
||
Under the hood, vlite uses llama.cpp for accelerated embedding generation and defaults to binary embeddings and INT8 embedding rescoring for the fastest retrieval in memory vector databases. It beats Chroma on all metrics retrieval/indexing, specifically 77.95% faster indexing speed compared to Chroma. | ||
# | ||
 | ||
|
||
## License | ||
|
||
AGPL-3.0 License | ||
|
||
## Contributing | ||
Thanks to [Claude](https://claude.ai), [Ray](https://github.com/raydelvecchio), and [Howard](https://github.com/HowieG) for their contributions to vlite. If you'd like to contribute, please open an issue or a pull request. | ||
|
||
Thanks to [Claude](https://claude.ai) and [Ray](https://github.com/raydelvecchio) for their contributions to vlite. If you'd like to contribute, please open an issue or a pull request. |