Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remaining priority ubuntu PR jobs #571

Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ad96465
Updates for 3 remaining priority PR jobs
alan-forbes-cp Oct 24, 2024
1926414
Remove "cl.exe" extra_flags compiler spec.
alan-forbes-cp Nov 4, 2024
7b81187
Remove "cross_compiling:" spec.
alan-forbes-cp Nov 4, 2024
c5b26d1
Refactor build setup for x-compile following review
alan-forbes-cp Nov 6, 2024
bd86c7b
More x-compile refactoring - add supported runner_arch & cross_arch
alan-forbes-cp Nov 6, 2024
c1cf2b2
Restore add-architecture; remove vulkan-sdk
alan-forbes-cp Nov 6, 2024
45aa303
Restore vulkan; echo key data to console
alan-forbes-cp Nov 6, 2024
916c760
Restore config for aarch64 x-compile
alan-forbes-cp Nov 6, 2024
c197398
Standardize bash "if" syntax
alan-forbes-cp Nov 6, 2024
bcdde4f
Remove dpkg architecture commands; no vulkan sdk for ARM cross.
alan-forbes-cp Nov 7, 2024
0c2310e
Restore dpkg for i386; make aarch64 cross CL only
alan-forbes-cp Nov 7, 2024
72757bf
Restore dpkg arm64
alan-forbes-cp Nov 7, 2024
166889f
add logging: echo bash commands
alan-forbes-cp Nov 7, 2024
3ff66aa
Suppress vulkan packaging updates for x-compile
alan-forbes-cp Nov 7, 2024
abfb073
Re-add the sed command for aarch64; exclude non-aarch64 jobs
alan-forbes-cp Nov 7, 2024
60c7e54
Refactor for aarch64
alan-forbes-cp Nov 7, 2024
464ee3b
Re-set api for aarch64 x-compile
alan-forbes-cp Nov 7, 2024
dec4686
Remove armhf installs from aarch64
alan-forbes-cp Nov 7, 2024
e5cab4b
Restore all jobs; tidy comments
alan-forbes-cp Nov 7, 2024
ac3b899
Remove bash logging and obsolete dpkg commands
alan-forbes-cp Nov 7, 2024
f50b8d9
Fix 32-bit arm string usage
alan-forbes-cp Nov 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions .github/actions/do_build_ock/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,12 +103,9 @@ inputs:
shell_to_use:
description: "which shell to use (default bash)"
default: bash
c_compiler:
description: "C compiler (default gcc)"
default: "gcc"
cxx_compiler:
description: "C++ compiler (default g++)"
default: "g++"
toolchain_file:
description: "Path to toolchain file"
default: ""

runs:
# We don't want a new docker just a list of steps, so mark as composite
Expand Down Expand Up @@ -164,8 +161,7 @@ runs:
-DCA_CL_DISABLE_UNITCL_VECZ_CHECKS=${{ inputs.disable_unitcl_vecz_checks }}
-DCA_GTEST_LAUNCHER="${{ inputs.gtest_launcher }}"
-DCA_BUILD_32_BITS=${{ inputs.build_32_bit }}
-DCMAKE_C_COMPILER=${{ inputs.c_compiler }}
-DCMAKE_CXX_COMPILER=${{ inputs.cxx_compiler }}
-DCMAKE_TOOLCHAIN_FILE=${{ inputs.toolchain_file }}
${{ inputs.extra_flags }}
.
- name: build_ock
Expand Down
50 changes: 43 additions & 7 deletions .github/actions/setup_build/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@ inputs:
save:
description: 'Save the build cache at the end - not for PR testing'
default: false
arch:
description: 'Architecture'
default: x86_64
cross_arch:
description: 'Cross compilation architecture from: x86, x86_64, armhf, aarch64, riscv64'
alan-forbes-cp marked this conversation as resolved.
Show resolved Hide resolved
default: ""
# Note: runner architectures from: x86_64 (aka runner.arch: X64), aarch64 (aka runner.arch: ARM64)

runs:
# We don't want a new docker just a list of steps, so mark as composite
Expand All @@ -32,16 +33,31 @@ runs:
if: ${{ inputs.os == 'ubuntu' }}
shell: bash
run: |
pip install lit clang-format==19.1.0 virtualenv
if [ "${{ inputs.arch }}" = "x86" ]; then sudo dpkg --add-architecture i386 ; fi
if [ "${{ inputs.cross_arch }}" = "x86" ]; then sudo dpkg --add-architecture i386 ; fi
wget -qO - https://packages.lunarg.com/lunarg-signing-key-pub.asc | sudo apt-key add -
if [ "${{ inputs.ubuntu_version }}" = "20.04" ]; then sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-1.3.243-focal.list https://packages.lunarg.com/vulkan/1.3.243/lunarg-vulkan-1.3.243-focal.list; fi
if [ "${{ inputs.ubuntu_version }}" = "22.04" ]; then sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-1.3.243-jammy.list https://packages.lunarg.com/vulkan/1.3.243/lunarg-vulkan-1.3.243-jammy.list; fi
if [ "${{ inputs.ubuntu_version }}" = "24.04" ]; then sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-noble.list https://packages.lunarg.com/vulkan/lunarg-vulkan-noble.list; fi
sudo apt-get update
pip install lit clang-format==19.1.0 virtualenv
sudo apt-get install --yes doxygen
sudo apt-get install --yes vulkan-sdk
if [ "${{ inputs.arch }}" = "x86" ]; then sudo apt-get install --yes gcc-multilib g++-multilib libc6-dev:i386 lib32tinfo-dev ; fi
if [ "${{ inputs.cross_arch }}" = "x86" ]; then \
sudo apt-get install --yes gcc-multilib g++-multilib libc6-dev:i386 lib32tinfo-dev; \
fi
if [ "${{ inputs.cross_arch }}" = "armhf" ]; then \
sudo apt-get install --yes gcc-11-arm-linux-gnueabihf g++-11-arm-linux-gnueabihf; \
fi
if [ "${{ inputs.cross_arch }}" = "aarch64" ]; then \
sudo apt-get install --yes gcc-11-aarch64-linux-gnu g++-11-aarch64-linux-gnu; \
fi
if [ "${{ inputs.cross_arch }}" = "riscv64" ]; then \
sudo apt-get install --yes gcc-11-riscv64-linux-gnu g++-11-riscv64-linux-gnu; \
fi
if [ "${{ inputs.cross_arch }}" = "armhf" -o "${{ inputs.cross_arch }}" = "aarch64" -o "${{ inputs.cross_arch }}" = "riscv64" ]; then \
alan-forbes-cp marked this conversation as resolved.
Show resolved Hide resolved
# Install QEMU for testing cross compilation.
sudo apt-get install --yes qemu-user; \
fi

- name: Install windows prerequisites
if: ${{ inputs.os == 'windows' }}
Expand All @@ -57,11 +73,31 @@ runs:
- name: Install Ninja
uses: llvm/actions/install-ninja@main

- name: set llvm key
id: set_llvm_key
shell: bash
run: |
KEY_VERSION="${{ inputs.ubuntu_version }}"
if [ "${{ inputs.os }}" = "windows" ]; then \
KEY_VERSION="${{ inputs.windows_version }}" ; \
fi
echo "key_version=$KEY_VERSION" >> "$GITHUB_OUTPUT"
KEY_ARCH="x86_64"
if [ "${{ runner.arch }}" = "ARM64" ]; then \
KEY_ARCH="aarch64" ; \
fi
if [ "${{ inputs.cross_arch }}" != "" ]; then \
KEY_ARCH="${{ inputs.cross_arch }}" ; \
fi
echo "key_arch=$KEY_ARCH" >> "$GITHUB_OUTPUT"
echo "key_version=$KEY_VERSION"
echo "key_arch=$KEY_ARCH"

- name: load llvm
uses: actions/cache/restore@v4
with:
path: llvm_install/**
key: llvm-${{ inputs.os }}-${{ inputs.os == 'ubuntu' && inputs.ubuntu_version || inputs.windows_version }}-${{ inputs.arch }}-v${{ inputs.llvm_version }}-${{ inputs.llvm_build_type }}
key: llvm-${{ inputs.os }}-${{ steps.set_llvm_key.outputs.key_version }}-${{ steps.set_llvm_key.outputs.key_arch }}-v${{ inputs.llvm_version }}-${{ inputs.llvm_build_type }}
fail-on-cache-miss: true

# note the PR testing usage should set 'save' to false, to avoid PR testing creating new caches on a branch
Expand Down
108 changes: 97 additions & 11 deletions .github/workflows/run_pr_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
with:
llvm_version: 18
llvm_build_type: RelAssert
save: ${{ inputs.update_cache }}
save: ${{ inputs.update_cache }}

# These need to match the configurations of build_pr_cache to use the cache effectively
- name: build host x86_64 online release
Expand Down Expand Up @@ -138,6 +138,7 @@ jobs:
with:
llvm_version: 19
llvm_build_type: RelAssert
save: ${{ inputs.update_cache }}
os: windows

- name: build ock x86_64 relassert
Expand All @@ -149,8 +150,6 @@ jobs:
shell_to_use: pwsh
gtest_launcher: "python3;-u;${{ github.workspace }}/scripts/gtest-terse-runner.py"
debug_support: ON
c_compiler: 'cl.exe'
cxx_compiler: 'cl.exe'

- name: build ock x86_64 offline
uses: ./.github/actions/do_build_ock
Expand All @@ -163,8 +162,6 @@ jobs:
debug_support: ON
install_dir: ${{ github.workspace }}/install_offline
build_dir: ${{ github.workspace }}/build_offline
c_compiler: 'cl.exe'
cxx_compiler: 'cl.exe'

# Based on: mr-ubuntu-gcc-x86_64-riscv-fp16-cl3.0-unitcl_vecz
run_ubuntu_gcc_x86_64_riscv_fp16_cl3_0_unitcl_vecz:
Expand All @@ -178,7 +175,7 @@ jobs:
with:
llvm_version: '18'
llvm_build_type: RelAssert
os: ubuntu
save: ${{ inputs.update_cache }}
- run: echo WORKSPACE is $GITHUB_WORKSPACE && echo PWD is `pwd` && ls -al
- name: build ock
uses: ./.github/actions/do_build_ock
Expand Down Expand Up @@ -207,14 +204,14 @@ jobs:
with:
llvm_version: '19'
llvm_build_type: RelAssert
arch: x86
save: ${{ inputs.update_cache }}
cross_arch: x86
- run: echo WORKSPACE is $GITHUB_WORKSPACE && echo PWD is `pwd` && ls -al
- name: build ock x86 relassert
uses: ./.github/actions/do_build_ock
with:
build_32_bit: ON
c_compiler: $GITHUB_WORKSPACE/llvm_install/bin/clang
cxx_compiler: $GITHUB_WORKSPACE/llvm_install/bin/clang++
extra_flags: -DCMAKE_C_COMPILER=$GITHUB_WORKSPACE/llvm_install/bin/clang -DCMAKE_CXX_COMPILER=$GITHUB_WORKSPACE/llvm_install/bin/clang++
build_targets: check-ock
enable_api: ""
builtin_kernel: ON
Expand All @@ -224,8 +221,7 @@ jobs:
uses: ./.github/actions/do_build_ock
with:
build_32_bit: ON
c_compiler: $GITHUB_WORKSPACE/llvm_install/bin/clang
cxx_compiler: $GITHUB_WORKSPACE/llvm_install/bin/clang++
extra_flags: -DCMAKE_C_COMPILER=$GITHUB_WORKSPACE/llvm_install/bin/clang -DCMAKE_CXX_COMPILER=$GITHUB_WORKSPACE/llvm_install/bin/clang++
build_targets: check-ock
runtime_compiler_enabled: OFF
assemble_spirv_ll_lit_test_offline: ON
Expand All @@ -234,3 +230,93 @@ jobs:
debug_support: ON
install_dir: $GITHUB_WORKSPACE/install_offline
build_dir: $GITHUB_WORKSPACE/build_offline

# Based on: mr-ubuntu-gcc-x86_64-riscv-fp16-cl3-0
run-ubuntu-gcc-x86_64-riscv-fp16-cl3-0:
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- name: Checkout repo
uses: actions/[email protected]
- name: setup-ubuntu
uses: ./.github/actions/setup_build
with:
llvm_version: '19'
llvm_build_type: RelAssert
alan-forbes-cp marked this conversation as resolved.
Show resolved Hide resolved
save: ${{ inputs.update_cache }}
- run: echo WORKSPACE is $GITHUB_WORKSPACE && echo PWD is `pwd` && ls -al
- name: build ock
uses: ./.github/actions/do_build_ock
with:
build_targets: check-ock
mux_targets_enable: riscv
mux_compilers_enable: riscv
riscv_enabled: ON
disable_unitcl_vecz_checks: ON
enable_rvv_scalable_vecz_check: ON
enable_rvv_scalable_vp_vecz_check: ON
host_enable_builtins: OFF
use_linker: gold
hal_description: RV64GCV_Zfh
hal_refsi_soc: G1
hal_refsi_thread_mode: WG
debug_support: ON

# Based on: mr-ubuntu-gcc-x86-llvm-latest-x86_64-images-cl3-0-release:
run-ubuntu-gcc-x86-llvm-latest-x86_64-images-cl3-0-release:
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- name: Checkout repo
uses: actions/[email protected]
- name: setup-ubuntu
uses: ./.github/actions/setup_build
with:
llvm_version: '19'
llvm_build_type: Release
alan-forbes-cp marked this conversation as resolved.
Show resolved Hide resolved
save: ${{ inputs.update_cache }}
- run: echo WORKSPACE is $GITHUB_WORKSPACE && echo PWD is `pwd` && ls -al
- name: build ock
uses: ./.github/actions/do_build_ock
with:
build_type: Release
build_targets: check-ock
host_image: ON
use_linker: gold
enable_api: ""
builtin_kernel: ON

# Based on: mr-ubuntu-gcc-aarch64-llvm-previous-cl3-0-fp16:
run-ubuntu-gcc-aarch64-llvm-latest-cl3-0-fp16:
runs-on: ubuntu-22.04
timeout-minutes: 90 # aarch64 needs longer timeout
steps:
- name: Checkout repo
uses: actions/[email protected]
- name: load native llvm
uses: actions/cache/restore@v4
with:
path: llvm_install/**
key: llvm-ubuntu-22.04-x86_64-v19-RelAssert
fail-on-cache-miss: true
- run: mv llvm_install llvm_install_native
- name: setup-ubuntu
uses: ./.github/actions/setup_build
with:
llvm_version: '19'
llvm_build_type: RelAssert
save: ${{ inputs.update_cache }}
cross_arch: aarch64
- run: echo WORKSPACE is $GITHUB_WORKSPACE && echo PWD is `pwd` && ls -al
- name: build ock
uses: ./.github/actions/do_build_ock
with:
build_targets: check-ock-cross
host_fp16: ON
use_linker: gold
debug_support: ON
builtin_kernel: ON
enable_api: ""
toolchain_file: "scripts/../platform/arm-linux/aarch64-toolchain.cmake"
extra_flags: -DCA_BUILTINS_TOOLS_DIR=${{ github.workspace }}/llvm_install_native/bin