Skip to content

Commit

Permalink
[Doc] Introduce LLM App in doc
Browse files Browse the repository at this point in the history
  • Loading branch information
SiriusNEO committed Jun 3, 2024
1 parent f757f85 commit 651da25
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,17 @@
Parrot is a distributed serving system for LLM-based Applications. The Parrot API w/ Semantic Variable is served by a centralized cluster manager called `ServeCore`, which manages many `Engine` instances. Each Parrot `Engine` runs a single LLM model and communicates with `ServeCore` by contextual Fill/Gen APIs. Note that each `Engine` is capable of providing language model services independently, therefore the system is horizontally scalable and many types of `Engine`s can be integrated into Parrot (e.g., vLLM, FasterTransformer, etc.).

![](assets/arch_paper_ver.png)
<div align="center">
<img src="assets/arch_paper_ver.png" width="500px" />
</div>

## LLM Applications

The powerful language understanding capability of large language models (LLMs) has enabled a new application paradigm, where one or multiple application entities, known as AI agents or co-pilots, communicate with LLMs via natural language, known as “prompts”, to accomplish a task collaboratively. Parrot is designed to serve these LLM-based applications efficiently by adding `Semantic Variable` in current OpenAI-style API, exposing richer application-level knowledge to backend systems and engines for better optimization.

<div align="center">
<img src="assets/app_examples.png" width="500px" />
</div>

## Install

Expand Down Expand Up @@ -54,6 +64,7 @@ If you find Parrot useful or relevant to your research, please cite our paper as
year = {2024},
address = {Santa Clara, CA},
publisher = {USENIX Association},
url = {https://www.usenix.org/conference/osdi24/presentation/lin-chaofan},
month = jul
}
```
Expand Down
Binary file added assets/app_examples.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/arch_paper_ver.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 651da25

Please sign in to comment.