Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc][WIP] Add official doc zh #36

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/installation.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# 安装

### 1. 依赖
| 依赖 | 支持版本 | 推荐版本 | 请注意 |
| ------------ | ------- | ----------- | ----------- |
| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | 安装vllm需要 |
| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | 安装vllm-ascend 和 torch-npu需要 |
| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | 安装vllm-ascend需要 |
| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | 安装torch-npu 和 vllm需要|

### 2. NPU环境准备

以下为安装推荐版本软件的快速说明:

#### 容器化安装
您可以直接使用[容器镜像](https://hub.docker.com/r/ascendai/cann),只需一行命令即可:

```bash
docker run \
--name vllm-ascend-env \
--device /dev/davinci1 \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-it quay.io/ascend/cann:8.0.rc3.beta1-910b-ubuntu22.04-py3.10 bash
```

无需手动安装 torch 和 torch_npu ,它们将作为 vllm-ascend 依赖项被自动安装。

#### 手动安装

您可以按照[昇腾安装指南](https://ascend.github.io/docs/sources/ascend/quick_install.html)中提供的说明配置环境。


### 3. 构建

#### 从源码构建Python包

```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
pip install -e .
```

#### 从源码构建容器镜像
```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
docker build -t vllm-ascend-dev-image -f ./Dockerfile .
```
2 changes: 1 addition & 1 deletion docs/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@
| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | Required for vllm-ascend |
| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | Required for torch-npu and vllm |

Find more about how to setup your environment in [here](docs/environment.md).
Find more about how to setup your environment in [here](installation.md).
17 changes: 17 additions & 0 deletions docs/quick_start.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# 快速开始

## 前提
### 支持的设备
- Atlas A2 训练系列 (Atlas 800T A2, Atlas 900 A2 PoD, Atlas 200T A2 Box16, Atlas 300T A2)
- Atlas 800I A2 推理系列 (Atlas 800I A2)

### Dependencies
| 依赖 | 支持版本 | 推荐版本 | 请注意 |
| ------------ | ------- | ----------- | ----------- |
| vLLM | main | main |安装vllm-ascend 必要
| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | 安装vllm必要 |
| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | 安装vllm-ascend 及 torch-npu必要 |
| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | 安装vllm-ascend必要|
| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | 安装torch-npu 和 vllm必要 |

点击[此处](./installation.zh.md)了解更多环境安装信息。
16 changes: 16 additions & 0 deletions docs/usage/index.zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# vLLM昇腾插件
vLLM Ascend plugin (vllm-ascend) is a community maintained hardware plugin for running vLLM on the Ascend NPU.
昇腾插件(vllm-ascend)是一个社区维护的硬件插件,用于在 NPU 上运行 vLLM。

此插件是 vLLM 社区中支持昇腾后端的推荐方式。它遵循[[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162)所述原则:通过解耦的方式提供了vLLM对Ascend NPU的支持。

使用 vLLM 昇腾插件,可以让类Transformer、混合专家(MOE)、嵌入、多模态等流行的大语言模型在 Ascend NPU 上无缝运行。

## 内容

- [快速开始](./quick_start.md)
- [安装](./installation.md)
- Usage
- [在昇腾运行vLLM](./usage/running_vllm_with_ascend.md)
- [特性支持](./usage/feature_support.md)
- [模型支持](./usage/supported_models.md)