Skip to content

Commit

Permalink
ci: build cuda arm64
Browse files Browse the repository at this point in the history 
Signed-off-by: thxCode <[email protected]>
  • Loading branch information
@thxCode
thxCode committed Jan 22, 2025
1 parent b4cde5f commit 95a631a
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 21 deletions.
20 changes: 11 additions & 9 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -363,8 +363,8 @@ jobs:
fail-fast: false
matrix:
# see https://hub.docker.com/r/nvidia/cuda/tags?page=&page_size=&ordering=&name=devel.
# 12.4 ==> 12.4.0, CentOS 7.
# 11.8 ==> 11.8.0, CentOS 7
# 12.4 ==> 12.4.0, CentOS 7, RockyLinux 8.
# 11.8 ==> 11.8.0, CentOS 7, RockyLinux 8.
# build fat binary,
# see https://developer.nvidia.com/cuda-gpus.
include:
Expand All @@ -376,7 +376,15 @@ jobs:
version: '11.8'
distro_container_image: 'gpustack/devel-nvidia-cuda:11.8.0-centos7'
cuda_arch: '60-real;61-real;70-real;75-real;80-real;86-real;89-real;90-real'
runs-on: ubuntu-22.04
- arch: 'arm64'
version: '12.4'
distro_container_image: 'gpustack/devel-nvidia-cuda:12.4.0-rockylinux8'
cuda_arch: '60-real;61-real;70-real;75-real;80-real;86-real;89-real;90-real'
- arch: 'arm64'
version: '11.8'
distro_container_image: 'gpustack/devel-nvidia-cuda:11.8.0-rockylinux8'
cuda_arch: '60-real;61-real;70-real;75-real;80-real;86-real;89-real;90-real'
runs-on: ${{ matrix.arch == 'amd64' && 'ubuntu-22.04' || 'ubuntu-22.04-arm64' }}
steps:
- name: Maximize Docker Build Space
uses: gpustack/.github/.github/actions/maximize-docker-build-space@main
Expand All @@ -395,12 +403,6 @@ jobs:
key: cache-linux-cuda-${{ matrix.arch }}-${{ matrix.version }}
path: |
${{ github.workspace }}/.cache
- name: Setup QEMU
if: ${{ matrix.arch == 'arm64' }}
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v8.1.5
platforms: "arm64"
- name: Build
env:
CCACHE_DIR: "${{ github.workspace }}/.cache/ccache"
Expand Down
25 changes: 13 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,8 @@ LLaMA Box supports the following platforms.

| Backend | OS/Arch | Device Requirement |
|----------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **NVIDIA CUDA 12.4** | `linux/amd64`<br/> `windows/amd64` | Compute capability matches `6.0`, `6.1`, `7.0`, `7.5` ,`8.0`, `8.6`, `8.9` or `9.0`, see <br/>https://developer.nvidia.com/cuda-gpus. <br/>Driver version requires `>=525.60.13`(linux)/`>=528.33`(windows), see <br/>https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id4. |
| **NVIDIA CUDA 11.8** | `linux/amd64`<br/> `windows/amd64` | Compute capability matches `6.0`, `6.1`, `7.0`, `7.5` ,`8.0`, `8.6`, `8.9` or `9.0`, see <br/>https://developer.nvidia.com/cuda-gpus. <br/>Driver version requires `>=450.80.02`(linux)/`>=452.39`(windows), see <br/>https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id4. |
| **NVIDIA CUDA 12.4** | `linux/amd64`<br/> `linux/arm64`<br/> `windows/amd64` | Compute capability matches `6.0`, `6.1`, `7.0`, `7.5` ,`8.0`, `8.6`, `8.9` or `9.0`, see <br/>https://developer.nvidia.com/cuda-gpus. <br/>Driver version requires `>=525.60.13`(linux)/`>=528.33`(windows), see <br/>https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id4. |
| **NVIDIA CUDA 11.8** | `linux/amd64`<br/> `linux/arm64`<br/> `windows/amd64` | Compute capability matches `6.0`, `6.1`, `7.0`, `7.5` ,`8.0`, `8.6`, `8.9` or `9.0`, see <br/>https://developer.nvidia.com/cuda-gpus. <br/>Driver version requires `>=450.80.02`(linux)/`>=452.39`(windows), see <br/>https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id4. |
| **AMD ROCm/HIP 6.2** | `linux/amd64`<br/> `windows/amd64` | LLVM target matches `gfx906 (linux only)`, `gfx908 (linux only)`, `gfx90a (linux only)`, `gfx942 (linux only)`, `gfx1030`, `gfx1100`, `gfx1101` or `gfx1102`, see <br/>https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.2.4/reference/system-requirements.html, <br/> https://rocm.docs.amd.com/projects/install-on-windows/en/docs-6.2.4/reference/system-requirements.html. |
| **Intel oneAPI 2025.0** | `linux/amd64`<br/> `windows/amd64` | Support [Intel oneAPI](https://en.wikipedia.org/wiki/OneAPI_(compute_acceleration)), see <br/>https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-base-toolkit-system-requirements.html. |
| **Huawei Ascend CANN 8.0** | `linux/amd64`<br/> `linux/arm64` | `Ascend 910b`, `Ascend 310p`, see <br/>https://www.hiascend.com/document/detail/en/CANNCommunityEdition/600alphaX/softwareinstall/instg/atlasdeploy_03_0015.html. |
Expand All @@ -115,16 +115,17 @@ LLaMA Box supports the following platforms.

> [!NOTE]
>
> Since v0.0.60, the build of Linux releases are as follows:
>
> - **"NVIDIA CUDA 12.4/11.8"** and **"AMD ROCm/HIP 6.2"** releases are built on CentOS 7 (glibc 2.17),
> - **"Intel oneAPI 2025.0"** releases are built on Ubuntu 22.04 (glibc 2.34).
> - **"Huawei Ascend CANN 8.0"** releases are built on Ubuntu 20.04 (glibc 2.31) and OpenEuler 20.03 (glibc 2.28).
> - **"Hygon DTK 24.04"** releases are built on Ubuntu 20.04 (glibc 2.31).
> - **"Moore Threads MUSA rc3.1"** releases are built on Ubuntu 22.04 (glibc 2.34).
> - _"AVX2"_ releases are built on CentOS 7 (glibc 2.17).
> - _"Advanced SIMD (NEON)"_ releases are built on Ubuntu 18.04 (glibc 2.27).
> - _"AVX512"_ releases are built on RockyLinux 8.9 (glibc 2.28).
> - **"NVIDIA CUDA 12.4/11.8"** amd64 releases are built on CentOS 7 (glibc 2.17), and arm64 releases are built on
RockyLinux 8.9 (glibc 2.28).
> - **"AMD ROCm/HIP 6.2"** amd64 releases are built on CentOS 7 (glibc 2.17).
> - **"Intel oneAPI 2025.0"** amd64 releases are built on Ubuntu 22.04 (glibc 2.34).
> - **"Huawei Ascend CANN 8.0"** amd64/arm64 releases are built on Ubuntu 20.04 (glibc 2.31) and OpenEuler 20.03 (glibc
2.28).
> - **"Hygon DTK 24.04"** amd64 releases are built on Ubuntu 20.04 (glibc 2.31).
> - **"Moore Threads MUSA rc3.1"** amd64 releases are built on Ubuntu 22.04 (glibc 2.34).
> - _"AVX2"_ amd64 releases are built on CentOS 7 (glibc 2.17).
> - _"Advanced SIMD (NEON)"_ amd64 releases are built on Ubuntu 18.04 (glibc 2.27).
> - _"AVX512"_ arm64 releases are built on RockyLinux 8.9 (glibc 2.28).

## Examples

Expand Down

0 comments on commit 95a631a

Please sign in to comment.