Pull Request Testing Interface

Pull Request Testing

Changes to the Trilinos develop branch must be made through Pull Requests. Prior to applying changes to the develop branch, the Trilinos Project supports a pull request testing feature. Pull request testing checks a small number of builds to support basic stability and provide developers with confidence that their changes are not breaking key configurations. Pull request testing is not comprehensive, and is not meant to guarantee broad portability. Also, pull request testing is only supported for pull requests submitted against the develop branch of Trilinos. Pull requests to all other branches are ignored.

The AutoTester tool that drives pull request testing for Trilinos is scheduled to run every 20 minutes, and does so unless another pull request being tested blocks it from doing so, in which case, the tester will run again after the current test run has completed. The tool has limited ability to run in parallel. Prior to being tested, a pull request must pass pre-test inspection. This inspection can be satisfied by either

• the pull request submitter being a member of the Trilinos GitHub organization

or

• a member of the Trilinos GitHub Developers team applying the AT: PRE-TEST INSPECTED label

There is also a pre-merge inspection. This inspection can only be satisfied through a member of the Trilinos GitHub organization approving the pull request (with no other members requesting changes), or through special exception.

The AutoTester schedules Jenkins jobs to confirm the submitted code will configure, build, and test using broad but fairly vanilla configurations and implements the Pull Request AutoTester inspection. The current set of builds performed on each pull request includes:

Pull Request Build	Todays CDash Results	Configuration and Environment settings	C++ Standard	POC(s) Software	POC(s) Hardware	GitHub/Jira issues	Comments
rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++17	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror, Tpetra_INST_Serial ON
rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no_rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror, Fortran OFF
rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no_rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror
rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no_rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Tpetra_INST_INT_INT ON, SuperLU_Dist ON
rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_cuda-10.1.105-gnu-7.2.0-spmpi-rolling_release_static_Volta70_Power9_no-asan_complex_fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		This build is temporarily disabled
rhel7_cuda-10.1.105-gnu-7.2.0-spmpi-rolling_release_static_Volta70_Power9_no-asan_complex_fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_pr-framework	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework

Dev2Master Build	Todays CDash Results	Configuration and Environment settings	C++ Standard	POC(s) Software	POC(s) Hardware	GitHub/Jira issues	Comments
rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++17	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror, Tpetra_INST_Serial ON
rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror, Fortran OFF
rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Complex double ON, -Werror
rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework		Tpetra_INST_INT_INT ON, SuperLU_Dist ON
rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_cuda-10.1.105-gnu-7.2.0-spmpi-rolling_release_static_Volta70_Power9_no-asan_complex_fpic_mpi_pt_no-rdc_no-uvm_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_cuda-10.1.105-gnu-7.2.0-spmpi-rolling_release_static_Volta70_Power9_no-asan_complex_fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-intel-19.0.5-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-clang-7.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-clang-9.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework
ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables	CDash	Config, Env	C++14	@trilinos/framework	@trilinos/framework

Note that these are meant to assure that the basic work of other developers can continue, not to replace the fuller SQA cycle represented by the nightly dashboard and release process.

When the three status tests listed above (pre-test inspection, AutoTester inspection, and pre-merge inspection) all pass, and GitHub indicates that the merge can be done without conflict, the merge button will turn green as an indicator the PR is ready to be merged. Note that having the AT: WIP label set will stop the AutoTester from proceeding to this check.

Understanding and controlling what the AutoTester does

• Turn off testing - Add the AT: WIP (Work In Progress) label to the PR or mark the PR as a Draft. No testing will occur until both the AT: WIP label and Draft marking are removed.

• Retest - Add the AT: RETEST label to the PR. The AutoTester will remove this label when it schedules the next build and test.

• Automerge when testing completes - Add the AT: AUTOMERGE label to the PR. Once all required checks have passed, the AutoTester will merge the PR without further user intervention.

• Prevent stale PRs from being merged - If testing passed, but it has been at least 6 days since the PR was tested, the AutoTester will apply the AT: STALE label to the PR. This will prevent the PR from being merged until the PR is tested again. Apply the AT: RETEST to request that a stale PR be retested.

Finding the errors from a Pull Request Build

The Jenkins builds from the AutoTester report to the CDash page under the Pull Request Group. These builds have names of the form PR-<pr-id>-test-Trilinos_pullrequest_<build-name>-<build-num>, where <pr-id> and <build-num> are shown in the PR comments created by the AutoTester when testing is started and when testing is completed.

Note that the CDash entry you are looking in the Pull Request Group for may not be on the current date. A full query can be constructed using the PR number and all dates before today similar to this query - just replace the 99999 (the <pr-id>) with your PR number.

Steps to reproduce a pull request build

See https://github.com/trilinos/Trilinos/wiki/Reproducing-PR-Testing-Errors.