ci: add github workflow to test the sched-ext kernel

Add a github action to test the sched-ext kernel with all the shipped
schedulers.

The test uses a similar approach to the scx workflow [1], using
virtme-ng to run each scheduler inside a sched-ext enabled kernel for a
certain amount of time (30 sec) and checking for potential stall, oops
or bug conditions.

In this case we can use `virtme-ng --build` to build a kernel with bare
minimum support to run inside virtme-ng itself, instead of generating a
fully featured kernel, to expedite the testing process.

The mandatory .config options required by sched-ext are stored in
`.github/workflows/sched-ext.config` and they are passed to virtme-ng
via the `--config` option.

The test itself is defined in `.github/workflows/run-schedulers`: the
script looks for all the binaries in `tools/sched_ext/build/bin` and
runs each one in a separate virtme-ng instance, to ensure that each run
does not impact the others.

[1] https://github.com/sched-ext/scx/blob/main/.github/workflows/build-scheds.yml

Signed-off-by: Andrea Righi <[email protected]>

.github/workflows/run-schedulers

@@ -0,0 +1,51 @@
+#!/bin/bash
+#
+# Run sched-ext scheduler for TIMEOUT seconds inside virtme-ng and catch
+# potential errors, then unload the scheduler and return the exit status.
+
+# Maximum time for each scheduler run.
+TEST_TIMEOUT=30
+
+# Maximum timeout for the guest used for each scheduler run (this is used to
+# hard-shutdown the guest in case of system hangs).
+GUEST_TIMEOUT=60
+
+# Check if virtme-ng is available.
+if [ ! -x `which vng` ]; then
+    echo "vng not found, please install virtme-ng to enable testing"
+    exit 1
+fi
+
+# Test all the available schedulers.
+#
+# NOTE: virtme-ng automatically runs the kernel from the current working
+# directory by default.
+#
+# Each scheduler will be tested in a separate instance booted from scratch, to
+# ensure that each run does not impact the others.
+#
+# TODO: exclude scx_layered for now, because it requires a special config
+# file, otherwise its test would fail with "Error: No layer spec".
+#
+# Maybe in the future change scx_layered to run with a default layer spec, just
+# for testing it.
+#
+for sched in $(find tools/sched_ext/build/bin -type f -executable | grep -v scx_layered); do
+    rm -f /tmp/output
+    (timeout --foreground --preserve-status ${GUEST_TIMEOUT} \
+        vng --force-9p --disable-microvm --verbose -- \
+            "timeout --foreground --preserve-status ${TEST_TIMEOUT} ${sched}" \
+                2>&1 </dev/null || true) | tee /tmp/output
+    sed -n -e '/\bBUG:/q1' \
+           -e '/\bWARNING:/q1' \
+           -e '/\berror\b/Iq1' \
+           -e '/\bstall/Iq1' \
+           -e '/\btimeout\b/Iq1' /tmp/output
+    res=$?
+    if [ ${res} -ne 0 ]; then
+        echo "FAIL: ${sched}"
+        exit 1
+    else
+        echo "OK: ${sched}"
+    fi
+done

.github/workflows/sched-ext.config

@@ -0,0 +1,34 @@
+# sched-ext mandatory options
+#
+CONFIG_BPF=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_BPF_JIT=y
+CONFIG_DEBUG_INFO_BTF=y
+CONFIG_BPF_JIT_ALWAYS_ON=y
+CONFIG_BPF_JIT_DEFAULT_ON=y
+CONFIG_SCHED_CLASS_EXT=y
+
+# Enable scheduling debugging
+#
+CONFIG_SCHED_DEBUG=y
+
+# Enable extra scheduling features (for a better code coverage while testing
+# the schedulers)
+#
+CONFIG_SCHED_AUTOGROUP=y
+CONFIG_SCHED_CORE=y
+
+# Enable fully preemptible kernel for a better test coverage of the schedulers
+#
+# CONFIG_PREEMPT_NONE is not set
+# CONFIG_PREEMPT_VOLUNTARY is not set
+CONFIG_PREEMPT=y
+CONFIG_PREEMPT_COUNT=y
+CONFIG_PREEMPTION=y
+CONFIG_PREEMPT_DYNAMIC=y
+CONFIG_PREEMPT_RCU=y
+
+# Additional debugging information (useful to catch potential locking issues)
+#
+CONFIG_DEBUG_LOCKDEP=y
+CONFIG_DEBUG_ATOMIC_SLEEP=y

.github/workflows/test-kernel.yml

@@ -0,0 +1,47 @@
+name: test-kernel
+run-name: ${{ github.actor }} PR run
+on: [pull_request, push]
+jobs:
+  test-schedulers:
+    runs-on: ubuntu-22.04
+    steps:
+      ### OTHER REPOS ####
+
+      # Hard turn-off interactive mode
+      - run: echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections
+
+      # Refresh packages list
+      - run: sudo apt update
+
+      ### DOWNLOAD AND INSTALL DEPENDENCIES ###
+
+      # Download dependencies packaged by Ubuntu
+      - run: sudo apt -y install gcc make git coreutils cmake elfutils libelf-dev libunwind-dev libzstd-dev linux-headers-generic linux-tools-common linux-tools-generic ninja-build python3-pip python3-requests qemu-kvm udev iproute2 busybox-static libvirt-clients kbd kmod file rsync zstd pahole flex bison cpio libcap-dev libelf-dev python3-dev cargo rustc
+
+      # clang 17
+      # Use a custom llvm.sh script which includes the -y flag for
+      # add-apt-repository. Otherwise, the CI job will hang. If and when
+      # https://github.com/opencollab/llvm-jenkins.debian.net/pull/26 is
+      # merged, we can go back to using https://apt.llvm.org/llvm.sh.
+      - run: wget https://raw.githubusercontent.com/Decave/llvm-jenkins.debian.net/fix_llvmsh/llvm.sh
+      - run: chmod +x llvm.sh
+      - run: sudo ./llvm.sh all
+      - run: sudo ln -sf /usr/bin/clang-17 /usr/bin/clang
+      - run: sudo ln -sf /usr/bin/llvm-strip-17 /usr/bin/llvm-strip
+
+      # Checkout repository
+      - uses: actions/checkout@v4
+
+      # Install virtme-ng
+      - run: pip install virtme-ng
+
+      ### END DEPENDENCIES ###
+
+      # Build a minimal kernel (with sched-ext enabled) using virtme-ng
+      - run: vng -v --build --config .github/workflows/sched-ext.config
+
+      # Build the in-kernel schedulers
+      - run: cd tools/sched_ext && make
+
+      # Test the schedulers inside the recompile kernel
+      - run: .github/workflows/run-schedulers

.gitignore

@@ -169,3 +169,6 @@ sphinx_*/
 
 # Rust analyzer configuration
 /rust-project.json
+
+# Include ".github" directory
+!.github/