Skip to content
This repository has been archived by the owner on Aug 10, 2024. It is now read-only.

Commit

Permalink
Remove cli command patch
Browse files Browse the repository at this point in the history
  • Loading branch information
sasha0552 authored Jul 15, 2024
1 parent bce932b commit 509f98e
Show file tree
Hide file tree
Showing 3 changed files with 1 addition and 24 deletions.
1 change: 0 additions & 1 deletion .github/workflows/build-vllm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ jobs:
patches=(
"https://raw.githubusercontent.com/${{ github.repository }}/${{ github.sha }}/patches/vllm/0000-enable-support-for-pascal-gpus.patch"
"https://raw.githubusercontent.com/${{ github.repository }}/${{ github.sha }}/patches/vllm/1000-set-torch-cuda-arch-list.patch"
"https://raw.githubusercontent.com/${{ github.repository }}/${{ github.sha }}/patches/vllm/9000-add-vllm-command-that-launches-api-server.patch"
)
# Apply patches
Expand Down
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ CI scripts designed to build a Pascal-compatible version of vLLM and Triton.

*Note: this repository holds "nightly" builds of `vLLM`, which may have the same `vLLM` version between releases in this repository, but have different source code. Despite the fact that they are "nightly", they are generally stable.*

*Note: the `vllm` command is an alias for the `python3 -m vllm.entrypoints.openai.api_server` command.*

*Note: kernels for all GPUs except Pascal have been excluded to reduce build time and wheel size. You can still use the new GPUs using tensor parallelism with Ray (and using two instances of `vLLM`, one of which will use upstream `vLLM`). Complain in [issues](https://github.com/sasha0552/vllm-ci/issues) if it disrupts your workflow.*

To install the patched `vLLM` (the patched `triton` will be installed automatically):
Expand All @@ -24,7 +22,7 @@ source venv/bin/activate
pip3 install --extra-index-url https://sasha0552.github.io/vllm-ci/ vllm

# Launch vLLM
vllm --help
vllm serve --help
```

To update a patched `vLLM` between same `vLLM` release versions (e.g. `0.5.0` (commit `000000`) -> `0.5.0` (commit `ffffff`))
Expand Down
20 changes: 0 additions & 20 deletions patches/vllm/9000-add-vllm-command-that-launches-api-server.patch

This file was deleted.

0 comments on commit 509f98e

Please sign in to comment.