Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial version of aarch64 container with Vulkan #270

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Initial version of aarch64 container with Vulkan #270

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions container-images/aarch64/Containerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
FROM registry.access.redhat.com/ubi9/ubi:9.4-1214.1726694543

# renovate: datasource=github-releases depName=huggingface/huggingface_hub extractVersion=^v(?<version>.*)
ARG HUGGINGFACE_HUB_VERSION=0.25.2
# renovate: datasource=github-releases depName=containers/omlmd extractVersion=^v(?<version>.*)
ARG OMLMD_VERSION=0.1.5
# renovate: datasource=github-releases depName=tqdm/tqdm extractVersion=^v(?<version>.*)
ARG TQDM_VERSION=4.66.5
ARG LLAMA_CPP_SHA=70392f1f81470607ba3afef04aa56c9f65587664
# renovate: datasource=git-refs depName=ggerganov/whisper.cpp packageName=https://github.com/ggerganov/whisper.cpp gitRef=master versioning=loose type=digest
ARG WHISPER_CPP_SHA=5caa19240d55bfd6ee316d50fbad32c6e9c39528

# vulkan-headers vulkan-loader-devel vulkan-tools glslc glslang python3-pip mesa-libOpenCL-$MESA_VER.aarch64
RUN dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm && \
crb enable && \
dnf install -y epel-release && \
dnf --enablerepo=ubi-9-appstream-rpms install -y git procps-ng vim \
dnf-plugins-core python3-dnf-plugin-versionlock cmake gcc-c++ \
python3-pip python3-argcomplete && \
dnf clean all && \
rm -rf /var/cache/*dnf*

# Install the patched mesa-krunkit drivers
RUN dnf config-manager --add-repo https://copr.fedorainfracloud.org/coprs/slp/mesa-krunkit/repo/epel-9/slp-mesa-krunkit-epel-9.repo && \
dnf -y install mesa-vulkan-drivers vulkan-loader-devel vulkan-headers vulkan-tools vulkan-loader glslc glslang && \
dnf clean all

RUN /usr/bin/python3 --version
RUN pip install "huggingface_hub[cli]==${HUGGINGFACE_HUB_VERSION}"
RUN pip install "omlmd==${OMLMD_VERSION}"
RUN pip install "tqdm==${TQDM_VERSION}"

ENV GGML_CCACHE=0

RUN git clone https://github.com/ggerganov/llama.cpp && \
cd llama.cpp && \
git reset --hard ${LLAMA_CPP_SHA} && \
#cmake -B build -DCMAKE_INSTALL_PREFIX:PATH=/usr -DGGML_CCACHE=0 && \
Copy link
Collaborator

@ericcurtin ericcurtin Oct 10, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well remove this commented out line

cmake -B build -DCMAKE_INSTALL_PREFIX:PATH=/usr -DGGML_CCACHE=0 -DGGML_VULKAN=ON && \
cmake --build build --config Release -j $(nproc) && \
cmake --install build && \
cd / && \
rm -rf llama.cpp

RUN git clone https://github.com/ggerganov/whisper.cpp.git && \
cd whisper.cpp && \
git reset --hard ${WHISPER_CPP_SHA} && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to build whisper.cpp with GGML_VULKAN also?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried by just setting the env variable GGML_VULKAN=ON which failed because ggml-vulkan-shaders.hpp were missing.
Seems like additional steps like running ggml_vk_generate_shaders.py before are missing.

make -j $(nproc) && \
mv main /usr/bin/whisper-main && \
mv server /usr/bin/whisper-server && \
cd / && \
rm -rf whisper.cpp
Loading