Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GeoMechanicsApplication] Investigate the coverage of the "small" and "nightly" test suites #12993

Open
rfaasse opened this issue Jan 9, 2025 · 2 comments · May be fixed by #13084
Open

[GeoMechanicsApplication] Investigate the coverage of the "small" and "nightly" test suites #12993

rfaasse opened this issue Jan 9, 2025 · 2 comments · May be fixed by #13084
Assignees

Comments

@rfaasse
Copy link
Contributor

rfaasse commented Jan 9, 2025

In order to decide which test suite we'd like to run on GitHub, we need to know about the coverage of the "small" and "nightly" test suites.

@rfaasse rfaasse converted this from a draft issue Jan 9, 2025
@rfaasse rfaasse self-assigned this Jan 10, 2025
@rfaasse rfaasse moved this from 📋 Sprint Backlog to 👷 In Progress in Kratos Product Backlog Jan 10, 2025
@mcgicjn2 mcgicjn2 moved this from 👷 In Progress to 👀 In Review in Kratos Product Backlog Jan 10, 2025
@mcgicjn2 mcgicjn2 moved this from 👀 In Review to 👷 In Progress in Kratos Product Backlog Jan 10, 2025
@rfaasse
Copy link
Contributor Author

rfaasse commented Jan 27, 2025

The small + nightly suites have the following numbers for coverage (the htmls are found in the teams channel for further inspection):

We see there is quite a difference in coverage between the two suites. Of course there is a balance between time it takes to run a suite and the coverage we're able to achieve.

Small suite
Image

Small + nightly suite
Image

Using this data, it is possible to drill into the parts which have a significant difference in coverage and see if it's possible to increase the coverage of the small suite by moving around some test cases, while not increasing the run-time too much.

@mcgicjn2, this was an issue I picked up as something I wanted to do during the 'Christmas Break' sprint. I would propose to close this issue (as I think I worked somewhere around 3-5 days on this already) and create some follow-ups to pick up work like:

  • Improve the coverage of the small suite
  • Create a kp command to locally be able to produce CC for test suites/UTs
  • Run coverage for several different test suites in TeamCity (so we don't only run coverage for unit tests
  • Investigate which test cover what functionality specifically (maybe part of revisiting the test cases)?

Etc. For now it has been useful to look at this, we were able to move some tests from nightly to validation (see #12990).

@rfaasse rfaasse moved this from 👷 In Progress to 👀 In Review in Kratos Product Backlog Jan 27, 2025
@mcgicjn2
Copy link
Contributor

May I ask what should be done with the branch that is open ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants