Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up CTest #5068

Merged
merged 183 commits into from
Sep 5, 2024
Merged

Set up CTest #5068

merged 183 commits into from
Sep 5, 2024

Conversation

EZoni
Copy link
Member

@EZoni EZoni commented Jul 20, 2024

Work-in-progress PR to set up CTest. Apply the same logic as in ImpactX and adjust/advance for WarpX.

  • Example of current input files organization:

    • inputs_base_2d
      (base input file for shared input parameters named as inputs_base_<1d;2d;3d;rz>)
    • inputs_test_2d_langmuir_multi_nodal,
      inputs_test_2d_langmuir_multi_psatd,
      ...
      (test input files for test-specific input parameters named as inputs_test_<1d;2d;3d;rz>_..., read base input file through the FILE parameter)
    • test names must start with test_<1d;2d;3d;rz>_... to guarantee consistency and make maintenance easier
  • New Azure CI matrix elements (list coming soon).

  • Run all MPI tests with 2 MPI processes and 1 OpenMP thread. Run single-process tests without MPI.

To-do:

  • WarpX_TEST_CLEANUP to remove test directories
  • WarpX_TEST_FPETRAP for runtime fpe-trapping
  • Resolve FIXME comments or redirect to new issues
  • Avoid copying inputs files ParmParse: Prefix to FILE AMReX-Codes/amrex#4126
  • Enforce new naming convention through CI style check
  • Remove obsolete CI style check scripts
  • Remove obsolete regression testing scripts: CTest follow-up: remove obsolete files/scripts #5220
  • Update docs
  • Remove files after each test: new build flag WarpX_CI
  • Make sure we don't miss any build variant:
  • Add missing checksums (new issue CTest follow-up: add missing checksum files #5206):
    • dirichlet_bc
    • dirichlet_bc_picmi
    • field_probe
    • embedded_boundary_picmi
    • particle_absorption
    • particle_reflection_picmi
    • id_cpu_read_picmi
    • runtime_components_picmi
    • laser_acceleration_rz_opmd
  • Support restart tests
    • Separate input file based on original input file (read through the FILE parameter)
    • Call default regression analysis against original checksum file, with small tolerance
    • Support restart from PICMI (can we set FILE through PICMI?)
    • Update analysis_default_restart.py
  • Support custom run commands (e.g., custom PICMI scripts using lists vs strings)
  • Add stricter dependency between .analysis test and .run test:
    run .analysis test only if .run test passed (now runs if .run test ran)
  • Update test README files (e.g., Ohm solver tests)
  • Skip pyAMReX tests directly from CMakeLists.txt

@EZoni EZoni added enhancement New feature or request component: tests Tests and CI labels Jul 20, 2024
@ax3l ax3l self-requested a review July 22, 2024 17:23
@ax3l ax3l self-assigned this Jul 22, 2024
@EZoni EZoni force-pushed the EZoni_ctest branch 4 times, most recently from 15146f7 to a382488 Compare July 22, 2024 23:25
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
Regression/WarpX-tests.ini Outdated Show resolved Hide resolved
@EZoni EZoni force-pushed the EZoni_ctest branch 4 times, most recently from 09ca93c to a81a95b Compare July 26, 2024 00:23
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
WeiqunZhang pushed a commit to AMReX-Codes/amrex that referenced this pull request Sep 5, 2024
## Summary

For CI/CD workflows and out-of-source tests we often want to include
dependent inputs files via `FILE = <filename>`. For development, we
prefer to run in temporary run directories but want to avoid having to
copy over the latest inputs file from a source directory (mostly to
avoid confusion between source and copy and to enable rapid development
cycles).

Now, the environment variable `AMREX_INPUTS_FILE_PREFIX` can be set to
prefix every `FILE = <filename>` with a custom path. We will use this in
the CTests integration of WarpX.

## Additional background

CC @EZoni
ECP-WarpX/WarpX#5068
CMakeLists.txt Show resolved Hide resolved
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, fantastic work! 🚀 ✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: tests Tests and CI enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants