Organizing tests with TestKit

[dant3]

Hello! Thank you for a great product!
I'm writing custom reporting extension, and created some tests using TestKit.
I've created a sample test which have all kinds of outcomes, be it aborted tests, skipped tests and failed tests. I apply my reporting extension to it and run the thing using TestKit, and I'm able to verify that it works correctly.
In other words, I have my ExtensionTest which runs TestKit, where TestKit executes ExtensionTest.TestCase test class.
Stupid problem I'm facing is that whenever I do 'gradle test' in my project, junit picks up and runs not only my ExtensionTest with TestKit, but also sample ExtensionTest.TestCase, which is designed to run under my TestKit test. Since there are tests which will always fail, it fails the whole testrun. Is there any way to do so that I'm able to use this TestCase under TestKit, but it won't be included without the TestKit?

[sbrannen]

Yes, you have a few options.

• Introduce a naming convention for your test classes that does not include your "test cases" that you execute via the EngineTestKit.
• Introduce a tag or tags for such "test cases", annotate those test cases with @Tag("my-tag"), and exclude the "my-tag" tag from your build.

In the JUnit 5 and Spring Framework builds, we primarily do the former.

Specifically, all of our "real" test classes match the pattern *Tests, and all of our "test cases" match the pattern *TestCase. We therefore only run the *Tests from our build.

https://github.com/junit-team/junit5/blob/97d9c2ec57e24736788df8f732de9456d3edaacb/gradle/plugins/src/main/kotlin/junitbuild.testing-conventions.gradle.kts#L15

We actually include test classes matching the patterns *Test and *Tests.