Skip to content

Commit

Permalink
Parallelize CI
Browse files Browse the repository at this point in the history
  • Loading branch information
cjdsellers committed Aug 20, 2022
1 parent c06520b commit 5316914
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 25 deletions.
20 changes: 3 additions & 17 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,28 +93,14 @@ jobs:
# Run tests (Linux)
- name: Run tests (Linux)
if: runner.os == 'Linux'
run: nox -rs coverage

# Upload coverage report to codecov (Linux)
- name: Upload coverage report (Linux)
if: runner.os == 'Linux'
uses: codecov/codecov-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
verbose: true
run: nox -rs tests_all

# Run tests (macOS)
- name: Run tests (macOS)
if: runner.os == 'macOS'
run: nox -rs tests_with_integration
run: nox -rs tests_all

# Run tests (Windows) without parallel build (avoids linker errors)
- name: Run tests (Windows)
if: runner.os == 'Windows'
run: nox -rs tests_with_integration -- no-parallel

# Run performance tests (Linux)
- name: Run performance tests (Linux)
if: runner.os == 'Linux'
run: nox -rs performance_tests
run: nox -rs tests_all -- no-parallel
88 changes: 88 additions & 0 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: coverage

# Run code coverage analysis for NautilusTrader

on:
push:
branches: [ develop ]

jobs:
build:
strategy:
fail-fast: false
matrix:
arch: [ x86 ]
os: [ ubuntu-latest ]
python-version: [ "3.10" ]
name: build - Python ${{ matrix.python-version }} (${{ matrix.arch }} ${{ matrix.os }})
runs-on: ${{ matrix.os }}

steps:
- name: Checkout repository
uses: actions/checkout@v2

# Rust setup
- name: Set up Rust environment
uses: hecrj/setup-rust-action@v1
with:
rust-version: stable
components: clippy

# Python setup
- name: Set up Python environment
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

# Install build dependencies
- name: Install build dependencies
run: |
python -m pip install --upgrade pip setuptools wheel poetry pre-commit nox
# Setup cached pre-commit
- name: Setup cached pre-commit
id: cache-pre-commit
uses: actions/cache@v2
with:
path: ~/.cache/pre-commit
key: ${{ runner.os }}-${{ matrix.python-version }}-pre-commit-${{ hashFiles('.github/workflows/*.yml', '.pre-commit-config.yaml') }}
restore-keys: |
${{ runner.os }}-${{ matrix.python-version }}-pre-commit-
# Run pre-commit
- name: Run pre-commit
run: pre-commit run --all-files

# Setup Poetry caching
- name: Get Poetry cache dir
id: cache-poetry
run: echo "::set-output name=dir::$(poetry config cache-dir)"

- name: Poetry/Nox cache
uses: actions/cache@v2
with:
path: ${{ steps.cache-poetry.outputs.dir }}
key: ${{ runner.os }}-${{ matrix.python-version }}-poetry-${{ hashFiles('.github/workflows/*.yml', '**/poetry.lock') }}
restore-keys: |
${{ runner.os }}-${{ matrix.python-version }}-poetry-
# Install and run database dependencies
- name: Install Redis (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get install redis-server
redis-server --daemonize yes
# Run tests (Linux)
- name: Run tests (Linux)
if: runner.os == 'Linux'
run: nox -rs coverage

# Upload coverage report to codecov (Linux)
- name: Upload coverage report (Linux)
if: runner.os == 'Linux'
uses: codecov/codecov-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
verbose: true
2 changes: 1 addition & 1 deletion .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:

# Run tests
- name: Run tests
run: nox -rs tests_with_integration
run: nox -rs tests_all

docker:
needs: [ build ]
Expand Down
10 changes: 7 additions & 3 deletions docs/developer_guide/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,15 @@ To run unit tests with nox:

If you have a redis-server up you can run integration tests with nox:

nox -s integration_tests
nox -s tests_integration

Or both unit and integration tests:
Or run the performance tests:

nox -s tests_with_integration
nox -s tests_performance

Or run the entire test suite:

nox -s tests_all

## Mocks
Unit tests will often include other components acting as mocks. The intent of this is to simplify
Expand Down
8 changes: 4 additions & 4 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,21 @@ def tests(session: Session) -> None:


@nox.session
def tests_with_integration(session: Session) -> None:
def tests_all(session: Session) -> None:
"""Run the test suite including integration tests."""
_setup_poetry(session, "--extras", ALL_EXTRAS, env={"PYTHONDEVMODE": "1"})
_run_pytest(session, "--ignore=tests/performance_tests/")
_run_pytest(session)


@nox.session
def integration_tests(session: Session) -> None:
def tests_integration(session: Session) -> None:
"""Run the integration test suite."""
_setup_poetry(session, "--extras", ALL_EXTRAS)
_run_pytest(session, "tests/integration_tests/")


@nox.session
def performance_tests(session: Session) -> None:
def tests_performance(session: Session) -> None:
"""Run the performance test suite."""
_setup_poetry(session, "--extras", ALL_EXTRAS)
_run_pytest(
Expand Down

0 comments on commit 5316914

Please sign in to comment.