diff --git a/docs/installation.zh.md b/docs/installation.zh.md new file mode 100644 index 0000000..cd49392 --- /dev/null +++ b/docs/installation.zh.md @@ -0,0 +1,55 @@ +# 安装 + +### 1. 依赖 +| 依赖 | 支持版本 | 推荐版本 | 请注意 | +| ------------ | ------- | ----------- | ----------- | +| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | 安装vllm需要 | +| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | 安装vllm-ascend 和 torch-npu需要 | +| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | 安装vllm-ascend需要 | +| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | 安装torch-npu 和 vllm需要| + +### 2. NPU环境准备 + +以下为安装推荐版本软件的快速说明: + +#### 容器化安装 +您可以直接使用[容器镜像](https://hub.docker.com/r/ascendai/cann),只需一行命令即可: + +```bash +docker run \ + --name vllm-ascend-env \ + --device /dev/davinci1 \ + --device /dev/davinci_manager \ + --device /dev/devmm_svm \ + --device /dev/hisi_hdc \ + -v /usr/local/dcmi:/usr/local/dcmi \ + -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ + -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ + -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ + -v /etc/ascend_install.info:/etc/ascend_install.info \ + -it quay.io/ascend/cann:8.0.rc3.beta1-910b-ubuntu22.04-py3.10 bash +``` + +无需手动安装 torch 和 torch_npu ,它们将作为 vllm-ascend 依赖项被自动安装。 + +#### 手动安装 + +您可以按照[昇腾安装指南](https://ascend.github.io/docs/sources/ascend/quick_install.html)中提供的说明配置环境。 + + +### 3. 构建 + +#### 从源码构建Python包 + +```bash +git clone https://github.com/vllm-project/vllm-ascend.git +cd vllm-ascend +pip install -e . +``` + +#### 从源码构建容器镜像 +```bash +git clone https://github.com/vllm-project/vllm-ascend.git +cd vllm-ascend +docker build -t vllm-ascend-dev-image -f ./Dockerfile . +``` diff --git a/docs/quick_start.md b/docs/quick_start.md index 548eb5a..ccf5ed5 100644 --- a/docs/quick_start.md +++ b/docs/quick_start.md @@ -14,4 +14,4 @@ | torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | Required for vllm-ascend | | torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | Required for torch-npu and vllm | -Find more about how to setup your environment in [here](docs/environment.md). \ No newline at end of file +Find more about how to setup your environment in [here](installation.md). \ No newline at end of file diff --git a/docs/quick_start.zh.md b/docs/quick_start.zh.md new file mode 100644 index 0000000..1bcc05e --- /dev/null +++ b/docs/quick_start.zh.md @@ -0,0 +1,17 @@ +# 快速开始 + +## 前提 +### 支持的设备 +- Atlas A2 训练系列 (Atlas 800T A2, Atlas 900 A2 PoD, Atlas 200T A2 Box16, Atlas 300T A2) +- Atlas 800I A2 推理系列 (Atlas 800I A2) + +### Dependencies +| 依赖 | 支持版本 | 推荐版本 | 请注意 | +| ------------ | ------- | ----------- | ----------- | +| vLLM | main | main |安装vllm-ascend 必要 +| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | 安装vllm必要 | +| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | 安装vllm-ascend 及 torch-npu必要 | +| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | 安装vllm-ascend必要| +| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | 安装torch-npu 和 vllm必要 | + +点击[此处](./installation.zh.md)了解更多环境安装信息。 \ No newline at end of file diff --git a/docs/usage/index.zh.md b/docs/usage/index.zh.md new file mode 100644 index 0000000..d0b02b3 --- /dev/null +++ b/docs/usage/index.zh.md @@ -0,0 +1,16 @@ +# vLLM昇腾插件 +vLLM Ascend plugin (vllm-ascend) is a community maintained hardware plugin for running vLLM on the Ascend NPU. +昇腾插件(vllm-ascend)是一个社区维护的硬件插件,用于在 NPU 上运行 vLLM。 + +此插件是 vLLM 社区中支持昇腾后端的推荐方式。它遵循[[RFC]: Hardware pluggable](https://github.com/vllm-project/vllm/issues/11162)所述原则:通过解耦的方式提供了vLLM对Ascend NPU的支持。 + +使用 vLLM 昇腾插件,可以让类Transformer、混合专家(MOE)、嵌入、多模态等流行的大语言模型在 Ascend NPU 上无缝运行。 + +## 内容 + +- [快速开始](./quick_start.md) +- [安装](./installation.md) +- Usage + - [在昇腾运行vLLM](./usage/running_vllm_with_ascend.md) + - [特性支持](./usage/feature_support.md) + - [模型支持](./usage/supported_models.md)