Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test reporting to CI #112

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+87 −40
Open

Add test reporting to CI #112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
11ca998
Add test reporting to CI
rsanchez15 Dec 3, 2024
2c8b472
Move to a temporary CI branch
rsanchez15 Dec 3, 2024
4ea340c
Assign id to test steps
rsanchez15 Dec 3, 2024
2753036
Remove flaky tests
rsanchez15 Dec 3, 2024
fb55510
Temporary change nightly branch for testing
rsanchez15 Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/mirror.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
steps:
- name: Mirror action step
id: mirror
uses: eProsima/eProsima-CI/external/mirror-branch-action@v0
uses: eProsima/eProsima-CI/external/mirror-branch-action@feature/test-reporting
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
source: 'main'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly-windows-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ jobs:
with:
custom_version_build: 'v3'
dependencies_artifact_postfix: '_nightly'
ref: 'main'
ref: 'ci/test-reporting'
secrets: inherit
108 changes: 72 additions & 36 deletions .github/workflows/reusable-ubuntu-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ jobs:
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
with:
path: src
ref: ${{ inputs.ref }}
Expand All @@ -112,14 +112,24 @@ jobs:
secret_token: ${{ secrets.GITHUB_TOKEN }}

- name: Compile and run tests
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@v0
id: compile_and_test
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@feature/test-reporting
with:
packages_names: ${{ env.code_packages_names }}
cmake_args: -DBUILD_TESTS=ON -DASAN_BUILD=ON
ctest_args: --label-exclude "xfail|xasan"
workspace_dependencies: './install'
cmake_build_type: Debug
test_report_artifact: test_report${{ inputs.dependencies_artifact_postfix }}_${{ inputs.custom_version_build }}
test_report_artifact: test_report_asan${{ inputs.dependencies_artifact_postfix }}_${{ inputs.custom_version_build }}

- name: Test Report
uses: eProsima/eProsima-CI/external/test-reporter@feature/test-reporting
if: success() || failure()
with:
name: "Report: ASAN "
path: "${{ steps.compile_and_test.outputs.ctest_results_path }}*.xml"
working-directory: 'src'
list-tests: 'failed'

#####################################################################
# TSAN
Expand All @@ -129,7 +139,7 @@ jobs:
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
with:
path: src
ref: ${{ inputs.ref }}
Expand All @@ -144,7 +154,8 @@ jobs:
secret_token: ${{ secrets.GITHUB_TOKEN }}

- name: Compile and run tests
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@v0
id: compile_and_test
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@feature/test-reporting
env:
# GCC 11.3 (Ubuntu Jammy default) produces several false positives regarding timed synchronization protocols
# These issues were fixed in GCC 12 so we upgrade to that version.
Expand All @@ -158,6 +169,15 @@ jobs:
cmake_build_type: Debug
test_report_artifact: test_report_tsan${{ inputs.dependencies_artifact_postfix }}_${{ inputs.custom_version_build }}

- name: Test Report
uses: eProsima/eProsima-CI/external/test-reporter@feature/test-reporting
if: success() || failure()
with:
name: "Report: TSAN "
path: "${{ steps.compile_and_test.outputs.ctest_results_path }}*.xml"
working-directory: 'src'
list-tests: 'failed'


#####################################################################
# CLANG
Expand All @@ -167,7 +187,7 @@ jobs:
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
with:
path: src
ref: ${{ inputs.ref }}
Expand All @@ -182,7 +202,7 @@ jobs:
secret_token: ${{ secrets.GITHUB_TOKEN }}

- name: Compile and run tests
uses: eProsima/eProsima-CI/multiplatform/clang_build_test@v0
uses: eProsima/eProsima-CI/multiplatform/clang_build_test@feature/test-reporting
with:
packages_names: ${{ env.code_packages_names }}
workspace_dependencies: './install'
Expand All @@ -198,7 +218,7 @@ jobs:
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
with:
path: src
ref: ${{ inputs.ref }}
Expand All @@ -213,7 +233,7 @@ jobs:
secret_token: ${{ secrets.GITHUB_TOKEN }}

- name: Compile and run tests
uses: eProsima/eProsima-CI/ubuntu/coverage_build_test_upload@v0
uses: eProsima/eProsima-CI/ubuntu/coverage_build_test_upload@feature/test-reporting
with:
packages_names: ${{ env.code_packages_names }}
workspace_dependencies: './install'
Expand All @@ -225,31 +245,47 @@ jobs:
#####################################################################
# FLAKY

flaky:
runs-on: ubuntu-22.04
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
with:
path: src
ref: ${{ inputs.ref }}

- name: Download dependencies and install requirements
uses: ./src/.github/actions/project_dependencies
with:
os: ubuntu-22.04
cmake_build_type: Release
custom_version_build: ${{ inputs.custom_version_build }}
dependencies_artifact_postfix: ${{ inputs.dependencies_artifact_postfix }}
secret_token: ${{ secrets.GITHUB_TOKEN }}
# Uncomment when flaky tests are available

# flaky:
# runs-on: ubuntu-22.04
# steps:

# - name: Sync repository
# uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
# with:
# path: src
# ref: ${{ inputs.ref }}

# - name: Download dependencies and install requirements
# uses: ./src/.github/actions/project_dependencies
# with:
# os: ubuntu-22.04
# cmake_build_type: Release
# custom_version_build: ${{ inputs.custom_version_build }}
# dependencies_artifact_postfix: ${{ inputs.dependencies_artifact_postfix }}
# secret_token: ${{ secrets.GITHUB_TOKEN }}

# - name: Compile and run tests
# id: compile_and_test
# uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@feature/test-reporting
# with:
# packages_names: ${{ env.code_packages_names }}
# workspace_dependencies: './install'
# ctest_args: --label-regex "xfail"
# cmake_build_type: Release
# test_report_artifact: test_report_flaky${{ inputs.dependencies_artifact_postfix }}_${{ inputs.custom_version_build }}

# - name: Test Report
# uses: eProsima/eProsima-CI/external/test-reporter@feature/test-reporting
# if: success() || failure()
# with:
# name: "Report: Flaky "
# path: "${{ steps.compile_and_test.outputs.ctest_results_path }}*.xml"
# working-directory: 'src'
# fail-on-empty: 'false'
# list-tests: 'failed'

- name: Compile and run tests
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@main
with:
packages_names: ${{ env.code_packages_names }}
workspace_dependencies: './install'
ctest_args: --label-regex "xfail"


#####################################################################
Expand All @@ -259,7 +295,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- name: Build and test documentation
uses: eProsima/eProsima-CI/ubuntu/sphinx_docs@main
uses: eProsima/eProsima-CI/ubuntu/sphinx_docs@feature/test-reporting
with:
checkout_path: "src/fastddsspy"
path_to_requirements: "src/fastddsspy/docs/requirements.txt"
Expand All @@ -275,7 +311,7 @@ jobs:
steps:

- name: Uncrustify
uses: eProsima/eProsima-CI/ubuntu/uncrustify@v0
uses: eProsima/eProsima-CI/ubuntu/uncrustify@feature/test-reporting


#####################################################################
Expand All @@ -286,4 +322,4 @@ jobs:
steps:

- name: Python Linter
uses: eProsima/eProsima-CI/ubuntu/python_linter@v0
uses: eProsima/eProsima-CI/ubuntu/python_linter@feature/test-reporting
15 changes: 13 additions & 2 deletions .github/workflows/reusable-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
steps:

- name: Sync repository
uses: eProsima/eProsima-CI/external/checkout@v0
uses: eProsima/eProsima-CI/external/checkout@feature/test-reporting
with:
path: src
ref: ${{ inputs.ref }}
Expand All @@ -103,7 +103,8 @@ jobs:
secret_token: ${{ secrets.GITHUB_TOKEN }}

- name: Compile and run tests
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@v0
id: compile_and_test
uses: eProsima/eProsima-CI/multiplatform/colcon_build_test@feature/test-reporting
with:
packages_names: ${{ env.code_packages_names }}
cmake_args: -DBUILD_TESTS=ON -DCMAKE_BUILD_TYPE=${{ matrix.cmake_build_type }}
Expand All @@ -112,4 +113,14 @@ jobs:
colcon_meta_file: ./src/.github/workflows/configurations/${{ runner.os }}/colcon.meta
test_report_artifact: test_report${{ inputs.dependencies_artifact_postfix }}_${{ inputs.custom_version_build }}_${{ inputs.os }}_${{ matrix.cmake_build_type }}

- name: Test Report
uses: eProsima/eProsima-CI/external/test-reporter@feature/test-reporting
if: success() || failure()
with:
name: "Report: ${{ matrix.os }} | ${{ matrix.cmake_build_type }} "
path: "${{ steps.compile_and_test.outputs.ctest_results_path }}*.xml"
working-directory: 'src'
path-replace-backslashes: 'true'
list-tests: 'failed'


Loading