Provide CI testing support (sandialabs#8)

This provides a starting point for converting our GitLab CI testing
infrastructure to the GitHub Actions workflow.

### Workflow Upgrades

To avoid duplication in the workflow scripts, I've created a new set of
[composite
actions](https://docs.github.com/en/actions/sharing-automations/creating-actions/creating-a-composite-action)
to install minimega, discovery, FIREWHEEL, and tox. We should hopefully
be able to use these composite actions to dramatically simplify some of
the CI scripts we had before. I've also upgraded the versions of
referenced reusable versions where possible.

### Test Enhancements

The unit tests have been upgraded to provide increased utility and work
better through the FIREWHEEL interface:
- Running the `firewheel test unit` helper now returns the appropriate
exit code returned by pytest, which was previously unintentionally
suppressed.
- A new marker includes or excludes tests which depend on model
components.
- In service of that functionality, the tests now also make use of
[pytest's built-in `-m`
option](https://docs.pytest.org/en/stable/example/markers.html#marking-test-functions-and-selecting-them-for-a-run)
to select marked tests for inclusion/exclusion rather than adding custom
options.

### Bug Fixes

This PR also fixes a few bugs in the current implementation:
- This fixes a testing error identified by testing the FIREWHEEL
deployment in an fresh installation (a hardcoded path in the
`test_cli_completion.py` script).
- Arguments passed to the helpers were not parsed correctly, ignoring
quoted inputs and just splitting on spaces. This changes the helpers to
use `shlex` for proper parsing.

.github/actions/prepare-discovery/action.yml

@@ -0,0 +1,15 @@
+# A composite action to install discovery
+
+name: Prepare discovery
+
+description: Install discovery
+
+runs:
+  using: composite
+  steps:
+    - name: Install discovery
+      run: |
+        wget https://github.com/mitchnegus/minimega-discovery/releases/download/firewheel-debian_faed761/discovery.deb
+        sudo dpkg -i discovery.deb
+        sudo chown -R $USER:minimega /opt/discovery
+      shell: bash

.github/actions/prepare-firewheel/action.yml

@@ -0,0 +1,33 @@
+# A composite action to install, configure, and initialize FIREWHEEL.
+# This action assumes that minimega and discovery have already been installed.
+
+name: Prepare FIREWHEEL
+
+description: Install, configure, and initialize FIREWHEEL
+
+runs:
+  using: composite
+  steps:
+    - name: Install FIREWHEEL
+      run: |
+        pip install --upgrade pip
+        pip install .
+        sudo ln -s $(which firewheel) /usr/local/bin/firewheel
+        ssh-keygen -t rsa -f "$HOME/.ssh/id_rsa" -N ""
+        ssh-keyscan -t rsa $(hostname) >> $HOME/.ssh/known_hosts
+        cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+      shell: bash
+    - name: Configure FIREWHEEL
+      run: |
+        firewheel config set -s cluster.compute $(hostname)
+        firewheel config set -s cluster.control $(hostname)
+        firewheel config set -s grpc.hostname $GRPC_HOSTNAME
+        firewheel config set -s minimega.experiment_interface $EXPERIMENT_INTERFACE
+        firewheel config set -s logging.root_dir $LOG_DIR
+      shell: bash
+    - name: Initialize FIREWHEEL
+      run: |
+        firewheel init
+        firewheel sync  # will produce `chgrp` errors (but permissions are sufficient)
+        firewheel start
+      shell: bash

.github/actions/prepare-minimega/action.yml

@@ -0,0 +1,29 @@
+# A composite action to install and initialize minimega
+
+name: Prepare minimega
+
+description: Install and initialize minimega
+
+runs:
+  using: composite
+  steps:
+    - name: Install minimega
+      run: |
+        wget https://github.com/sandia-minimega/minimega/releases/download/2.9/minimega-2.9.deb
+        sudo dpkg -i minimega-2.9.deb
+        sudo chown -R $USER:minimega $MM_INSTALL_DIR
+        sudo ln -s $MM_INSTALL_DIR/bin/minimega /usr/local/bin/minimega
+        sudo ln -s $MM_INSTALL_DIR/bin/minimega /usr/local/bin/mm
+      shell: bash
+    - name: Initialize minimega
+      run: |
+        echo -n "" | sudo $MM_INSTALL_DIR/misc/daemon/minimega.init install
+        sudo mkdir -p $(dirname $MINIMEGA_CONFIG)
+        sudo sed -i "s|MINIMEGA_DIR=\"/opt/minimega/\"|MINIMEGA_DIR=\"$MM_INSTALL_DIR/\"|g" $MINIMEGA_CONFIG
+        sudo sed -i "s|MM_RUN_PATH=\"/tmp/minimega\"|MM_RUN_PATH=\"$MM_BASE/\"|g" $MINIMEGA_CONFIG
+        sudo sed -i "s|MM_MESH_DEGREE=0|MM_MESH_DEGREE=1|g" $MINIMEGA_CONFIG
+        sudo sed -i "s|MM_LOG_LEVEL=\"error\"|MM_LOG_LEVEL=\"debug\"|g" $MINIMEGA_CONFIG
+        sudo sed -i "s|MM_LOG_FILE=\"/tmp/minimega.log\"|MM_LOG_FILE=\"$LOG_DIR/minimega.log\"|g" $MINIMEGA_CONFIG
+        sudo systemctl restart minimega
+        sudo chown -R $USER:minimega $MM_BASE $LOG_DIR
+      shell: bash

.github/actions/prepare-tox/action.yml

@@ -0,0 +1,15 @@
+# A composite action to prepare tox-based workflows
+
+name: Prepare tox
+
+description: Prepare tox-based workflows
+
+runs:
+  using: composite
+  steps:
+    - name: Ensure upgraded pip
+      run: pip install --upgrade pip
+      shell: bash
+    - name: Install tox
+      run: pip install tox
+      shell: bash

.github/workflows/documentation.yml

@@ -1,5 +1,4 @@
-# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
-# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+# This workflow will build and deploy the FIREWHEEL documentation
 
 name: Documentation
 
@@ -34,13 +33,11 @@ jobs:
     - name: Setup Pages
       uses: actions/configure-pages@v5
     - name: Set up Python
-      uses: actions/setup-python@v3
+      uses: actions/setup-python@v5
       with:
         python-version: '3.x'
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install tox
+    - name: Prepare to use tox-based environments
+      uses: ./.github/actions/prepare-tox
     - name: Build Documentation
       run: |
         tox -e dependencies,docs
@@ -60,4 +57,4 @@ jobs:
     steps:
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v4
+        uses: actions/deploy-pages@v4

.github/workflows/linting.yml

@@ -1,4 +1,4 @@
-# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
+# This workflow will install Python dependencies and lint with a variety of Python versions
 # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
 
 name: Linting
@@ -23,13 +23,11 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v3
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install tox
+    - name: Prepare to use tox-based environments
+      uses: ./.github/actions/prepare-tox
     - name: Lint code
       run: |
         tox -e lint

.github/workflows/testing.yml

@@ -0,0 +1,49 @@
+# This workflow will install Python dependencies and run tests with a variety of Python versions
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: Testing
+
+on:
+  push:
+    branches: [ "*" ]
+  pull_request:
+    branches: [ "main" ]
+
+env:
+  LOG_DIR: /var/log/firewheel
+  MINIMEGA_CONFIG: /etc/minimega/minimega.conf
+  # Set the FIREWHEEL environment variables
+  EXPERIMENT_INTERFACE: lo
+  MM_BASE: /tmp/minimega
+  MM_INSTALL_DIR: /opt/minimega
+  GRPC_HOSTNAME: localhost
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y tar net-tools procps uml-utilities \
+                                openvswitch-switch qemu-kvm qemu-utils dnsmasq \
+                                ntfs-3g iproute2 libpcap-dev
+    - name: Prepare minimega
+      uses: ./.github/actions/prepare-minimega
+    - name: Prepare discovery
+      uses: ./.github/actions/prepare-discovery
+    - name: Prepare FIREWHEEL
+      uses: ./.github/actions/prepare-firewheel
+    - name: Run unit tests
+      run: |
+        firewheel test unit -m 'not long and not mcs' \
+                            --cov --cov-report=term --cov-fail-under=60

pyproject.toml

@@ -55,28 +55,30 @@ classifiers = [
 ]
 dependencies = [
     "minimega==2.9",
+    "ClusterShell<=1.9.2",
     "colorama<=0.4.6",
+    "coverage<=7.6.10",
     "grpcio>=1.49.0,<=1.67.0",
     "grpcio-tools>=1.49.0,<=1.69.0",
+    "importlib_metadata>=3.6,<=8.5.0",
     "Jinja2>=3.1.2,<=3.1.5",
     "netaddr<=1.3.0,>=0.7.0",
     "networkx>=2.3,<=3.4.2",
     "protobuf>=5.0.0,<=5.29.3",
-    "ClusterShell<=1.9.2",
     "pytest<=8.3.4",
+    "pytest-cov<=6.0.0",
     "python-dotenv<=1.0.1",
     "PyYAML<=6.0.2",
     "qemu.qmp==0.0.3",
-    "rich>=13.6.0,<13.10",
     "requests>=2.22.0,<=2.32.3",
-    "importlib_metadata>=3.6,<=8.5.0",
+    "rich>=13.6.0,<13.10",
 ]
 
 [project.optional-dependencies]
-test = [
-    "tox<=4.23.2",
-    "coverage<=7.6.10",
-    "pytest-cov<=6.0.0",
+mcs = [
+    "firewheel-repo-base",
+    "firewheel-repo-linux",
+    "firewheel-repo-vyos",
 ]
 format = [
     "ruff==0.9.2",      # Linting/formatting
@@ -95,8 +97,9 @@ docs = [
     "sphinx-design",
 ]
 dev = [
-    "firewheel[test,format,docs]",
+    "firewheel[mcs,format,docs]",
     "pre-commit",
+    "tox~=4.0",
 ]
 
 [project.urls]  # Optional

src/firewheel/cli/firewheel_cli.py

@@ -3,6 +3,7 @@
 import os
 import cmd
 import sys
+import shlex
 import logging
 import textwrap
 from math import floor
@@ -1199,7 +1200,7 @@ def complete_help(self, text, line, _begidx, _endidx):
 def main():  # pragma: no cover
     """Provide an entry point to the FIREWHEEL CLI."""
     if len(sys.argv) > 1:
-        argstr = " ".join(sys.argv[1:])
+        argstr = shlex.join(sys.argv[1:])
         try:
             sys.exit(FirewheelCLI().onecmd(argstr))
         except KeyboardInterrupt:

src/firewheel/cli/helpers/test/unit

@@ -21,5 +21,5 @@ from firewheel import FIREWHEEL_PACKAGE_DIR
 
 if __name__ == "__main__":
     test_dir = FIREWHEEL_PACKAGE_DIR / "tests" / "unit"
-    pytest.main([str(test_dir), *sys.argv[1:]])
+    sys.exit(pytest.main([*sys.argv[1:], str(test_dir)]))
 DONE

src/firewheel/cli/init_firewheel.py

@@ -118,16 +118,16 @@ def _check_minimega_socket(self):
         Returns:
             bool: False if minimega is not running, True otherwise.
         """
-        status = False
         try:
             minimegaAPI()
-            status = True
-            return True
         except (RuntimeError, TimeoutError):
-            return False
+            status = False
+        else:
+            status = True
         finally:
             success_str = self._get_success_str(status)
             print(f"Checking minimega service status: {success_str}")
+        return status
 
     def _get_minimega_install_dir(self):
         # We should check that the minimega bin is in the expected location.

src/firewheel/cli/utils.py

@@ -1,5 +1,6 @@
 import os
 import sys
+import shlex
 from pathlib import Path
 
 from rich.table import Table
@@ -78,7 +79,7 @@ def parse_to_helper(args, helpers_dict):
         InvalidHelperTypeError: If a Helper group was specified rather than a Helper.
     """
     # Check for the Helper name in the Helpers dict.
-    args = args.split()
+    args = shlex.split(args)
     if args[0] not in helpers_dict:
         raise HelperNotFoundError("Unable to find Helper")
     # Check the type of the Helper entry.

src/firewheel/tests/conftest.py

@@ -10,33 +10,9 @@
     <https://docs.pytest.org/en/7.1.x/example/simple.html#control-skipping-of-tests-according-to-command-line-option>`.
 """
 
-from typing import List
-
 import pytest
 
 
-def pytest_addoption(parser: pytest.Parser) -> None:
-    """
-    Register argparse-style options for running tests.
-
-    Register argparse-style options and ini-style config values,
-    called once at the beginning of a test run. This is an
-    `initialization hook
-    <https://docs.pytest.org/en/7.4.x/reference/reference.html#pytest.hookspec.pytest_addoption>`
-    provided by :py:mod:`pytest`.
-
-    Args:
-        parser (pytest.Parser): The parser that will received added
-            options.
-    """
-    parser.addoption(
-        "--quick",
-        action="store_true",
-        default=False,
-        help="exclude tests marked as long",
-    )
-
-
 def pytest_configure(config: pytest.Config) -> None:
     """
     Enable this conftest file to perform initial configuration.
@@ -50,39 +26,16 @@ def pytest_configure(config: pytest.Config) -> None:
     they are imported.
 
     This specific hook adds custom markers to the test suite, such as a
-    ``long`` marker to indicate long-running tests.
-
-    Args:
-        config (pytest.Config): The pytest config object.
-    """
-    # As a heuristic, mark tests that take more than 10 seconds as "long"
-    config.addinivalue_line("markers", "long: mark test as long running")
-
-
-def pytest_collection_modifyitems(
-    session: pytest.Session,  # noqa: ARG001
-    config: pytest.Config,
-    items: List[pytest.Item],
-) -> None:
-    """
-    Modify the set of tests/items collected by :py:mod:`pytest`.
-
-    This is a `collection hook
-    <https://docs.pytest.org/en/7.4.x/reference/reference.html#pytest.hookspec.pytest_collection_modifyitems>`
-    provided by :py:mod:`pytest` to modify the set of tests or items
-    collected by the test runner. The hook is called after collection
-    has been performed and it may filter or re-order the items in-place.
+    ``long`` marker to indicate long-running tests (more than 10 seconds
+    duration) and the ``mcs`` marker to indicate tests that require
+    model components beyond the base FIREWHEEL package.
 
     Args:
-        session (pytest.Session): The pytest session object.
         config (pytest.Config): The pytest config object.
-        items (list): A list of item objects
     """
-    # Use the custom `--quick` option with the `long` marker to skip long running tests
-    if config.getoption("--quick"):
-        skip_long = pytest.mark.skip(
-            reason="--quick option excludes long running tests"
-        )
-        for item in items:
-            if "long" in item.keywords:
-                item.add_marker(skip_long)
+    markers = [
+        "long: mark test as long running",
+        "mcs: mark test as dependent on model components",
+    ]
+    for marker in markers:
+        config.addinivalue_line("markers", marker)