Micronaut 4 Gradle project test execution broken

[dbalek]

Generate a sample Micronaut project using Micronaut Launch service with Micronaut 4.1.1, Java 17, Gradle, and JUnit selected. Open generated project in VSCode with the Extension Pack for Java installed. Try to run project tests either via Test Explorer or by clicking Run Test icons in source editor gutter. No test gets executed and the following error appear in the console:

org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)
EngineDiscoveryOrchestrator.java:160
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:132)
EngineDiscoveryOrchestrator.java:132
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:107)
EngineDiscoveryOrchestrator.java:107
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:78)
EngineDiscoveryOrchestrator.java:78
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:99)
DefaultLauncher.java:99
	at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:77)
DefaultLauncher.java:77
	at org.junit.platform.launcher.core.DelegatingLauncher.discover(DelegatingLauncher.java:42)
DelegatingLauncher.java:42
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.discover(SessionPerRequestLauncher.java:56)
SessionPerRequestLauncher.java:56
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.<init>(JUnit5TestReference.java:46)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createUnfilteredTest(JUnit5TestLoader.java:88)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createTest(JUnit5TestLoader.java:69)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.loadTests(JUnit5TestLoader.java:56)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:513)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: org.junit.platform.commons.JUnitException: ClassSelector [className = 'com.example.Mn4gradleTest', classLoader = jdk.internal.loader.ClassLoaders$AppClassLoader@251a69d7] resolution failed
	at org.junit.platform.launcher.listeners.discovery.AbortOnFailureLauncherDiscoveryListener.selectorProcessed(AbortOnFailureLauncherDiscoveryListener.java:39)
AbortOnFailureLauncherDiscoveryListener.java:39
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:103)
EngineDiscoveryRequestResolution.java:103
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:83)
EngineDiscoveryRequestResolution.java:83
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryRequestResolver.java:113)
EngineDiscoveryRequestResolver.java:113
	at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:46)
DiscoverySelectorResolver.java:46
	at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:69)
JupiterTestEngine.java:69
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152)
EngineDiscoveryOrchestrator.java:152
	... 15 more
Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Stream org.junit.platform.commons.support.ReflectionSupport.streamNestedClasses(java.lang.Class, java.util.function.Predicate)'
	at org.junit.jupiter.engine.discovery.ClassSelectorResolver.lambda$toResolution$12(ClassSelectorResolver.java:138)
ClassSelectorResolver.java:138
	at org.junit.platform.engine.support.discovery.SelectorResolver$Match.expand(SelectorResolver.java:668)
SelectorResolver.java:668
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$enqueueAdditionalSelectors$1(EngineDiscoveryRequestResolution.java:110)
EngineDiscoveryRequestResolution.java:110
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
ForEachOps.java:183
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
ReferencePipeline.java:179
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:4853)
Collections.java:4853
	at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4861)
Collections.java:4861
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
AbstractPipeline.java:509
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
AbstractPipeline.java:499
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
ForEachOps.java:150
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
ForEachOps.java:173
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
AbstractPipeline.java:234
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
ReferencePipeline.java:596
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.enqueueAdditionalSelectors(EngineDiscoveryRequestResolution.java:109)
EngineDiscoveryRequestResolution.java:109
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:95)
EngineDiscoveryRequestResolution.java:95
	... 20 more

[jdneo]

It's probably caused by some API changes between JUnit 5.9 & 5.10.

I was trying to update the JDT Test to the latest version but unfortunately it will cause the legacy JUnit tests fails. See: #1608

[graemerocher]

Why is the IDE embedding a version of JUnit that is incompatible with the version used by the project? This seems poorly thought out as these versions will unlikely align

[jdneo]

I need to correct my statement before; the problem is not caused by the embedded JUnit libraries.

The test runner will use the project's JUnit dependencies if they are available. Only when the required dependencies are not declared by the project, the test runner will use the embedded ones.

The problem is caused due to some concept mismatch between the JDT project and the Gradle project.

If I run ./gradlew dependencies from the sample project, I can see both JUnit 5.9.3 and 5.10.0 are declared in different source sets. That is fine for a Gradle project, because in Gradle projects, each source set can have its own classpath.

Unfortunately, the concept of source set does not exist for a JDT project. One JDT project can only have one .classpath file which persists the classpath of the whole project. So, when importing a Gradle project to a JDT project, a 'workaround' is done to mitigate this kind of mismatch, that is to merge the classpath from all Gradle source sets into one.

For the sample project, the problem happens after merging. Both 5.9.3 and 5.10.0 are used in the classpath. Then who comes first wins -- the mismatch happens!

From the JDT side, there are two options to solve the issue:

• Support multiple classpath scopes for a JDT project
• Support test delegation to build tools. (In this case, it's Gradle)

Maybe, from Micronaut side, a workaround is to align the JUnit version for all the sourceset. But I admit that, to entirely solve the issue, something needs to be done at JDT side.

[graemerocher]

so if you run:

/gradlew dependencies --configuration runtimeClasspath | grep junit    
|    +--- org.junit:junit-bom:5.9.3
|    |    +--- org.junit:junit-bom:5.9.3

The only thing that is there is a BOM which should not be in the classpath. if you run:

./gradlew dependencies --configuration testRuntimeClasspath | grep junit
|    +--- org.junit:junit-bom:5.9.3 -> 5.10.0
|    |    +--- org.junit.jupiter:junit-jupiter:5.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-engine:5.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-console:1.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-launcher:1.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-params:5.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-commons:1.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-engine:1.10.0 (c)
|    |    \--- org.junit.platform:junit-platform-reporting:1.10.0 (c)
|    |    +--- org.junit:junit-bom:5.9.3 -> 5.10.0 (*)
|    |    +--- io.micronaut.test:micronaut-test-junit5:4.0.2 (c)
+--- org.junit.jupiter:junit-jupiter-api -> 5.10.0
|    +--- org.junit:junit-bom:5.10.0 (*)
|    \--- org.junit.platform:junit-platform-commons:1.10.0
|         \--- org.junit:junit-bom:5.10.0 (*)
+--- io.micronaut.test:micronaut-test-junit5 -> 4.0.2
|    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 -> 5.10.0 (*)
+--- org.graalvm.buildtools:junit-platform-native:0.9.25
|    +--- org.junit:junit-bom:5.10.0 (*)
|    +--- org.junit.platform:junit-platform-console:1.9.3 -> 1.10.0
|    |    +--- org.junit:junit-bom:5.10.0 (*)
|    |    \--- org.junit.platform:junit-platform-reporting:1.10.0
|    |         +--- org.junit:junit-bom:5.10.0 (*)
|    |         \--- org.junit.platform:junit-platform-launcher:1.10.0
|    |              +--- org.junit:junit-bom:5.10.0 (*)
|    |              \--- org.junit.platform:junit-platform-engine:1.10.0
|    |                   +--- org.junit:junit-bom:5.10.0 (*)
|    |                   \--- org.junit.platform:junit-platform-commons:1.10.0 (*)
|    +--- org.junit.platform:junit-platform-launcher:1.9.3 -> 1.10.0 (*)
|    \--- org.junit.jupiter:junit-jupiter -> 5.10.0
|         +--- org.junit:junit-bom:5.10.0 (*)
|         +--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
|         +--- org.junit.jupiter:junit-jupiter-params:5.10.0
|         |    +--- org.junit:junit-bom:5.10.0 (*)
|         |    \--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
|         \--- org.junit.jupiter:junit-jupiter-engine:5.10.0
|              +--- org.junit:junit-bom:5.10.0 (*)
|              +--- org.junit.platform:junit-platform-engine:1.10.0 (*)
|              \--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
\--- org.junit.jupiter:junit-jupiter-engine -> 5.10.0 (*)

All the JUnit dependencies are promoted from 5.9.3 to 5.10.0 (the correct version). So it seems to me that the classpath produced by the JDT tooling is simply incorrect and the collection algorithm for dependencies incorrect and inconsistent with the behaviour of Gradle which is a massive problem

[jdneo]

The problem happens due to org.junit.platform:junit-platform-commons

+--- io.micronaut.platform:micronaut-platform:4.1.2
|    +--- org.junit:junit-bom:5.9.3
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 (c)
|    |    \--- org.junit.platform:junit-platform-commons:1.9.3 (c)

Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Stream org.junit.platform.commons.support.ReflectionSupport.streamNestedClasses(java.lang.Class, java.util.function.Predicate)'

[graemerocher]

if you look at the graph the correct version junit-platform-commons in my aforementioned comment is selected by the Gradle dependency configuration hence why it works in Gradle and not in VSCode

The bug appears to be that VScode is erroneously including the dependencies of io.micronaut.platform:micronaut-platform:4.1.2 which is a BOM not a dependency.

The classpath computation is clearly broken here.

We can of course "fix" this on the Micronaut side but the bug will still be there waiting to be hit again

[graemerocher]

A workaround is to add testImplementation("org.junit.platform:junit-platform-commons:1.10.0") to the Gradle build but this is a workaround and should never be necessary in the first place since VSCode/JDT is computing an erroneous classpath that differs from the one Gradle computes

[graemerocher]

Seems this is related to #1020 which again sounds like the classpath computation for Gradle is simply incorrect

[jdneo]

I want to clarify that I'm not denying that current JDT cannot handle the problem perfectly. As I already said above:

But I admit that, to entirely solve the issue, something needs to be done at JDT side.

All the above comment I left is the analysis that I found why the classpath is wrong in JDT, and provide a possible workaround if it's still blocking before the issue is solved from JDT side. Please don't get me wrong.

[MahatmaFatalError]

I have the same problem with a Spring Boot gradle project in Eclipse. Using org.junit:junit-bom:5.10.1 that lead to org.junit.platform:junit-platform-commons:1.10.1

testImplementation("org.junit.platform:junit-platform-commons:1.10.0") fixed the problem, but I am puzzled why...

[jdneo]

Dup with #1045 (comment).

There will be some update around this.

See: microsoft/build-server-for-gradle#119

[jdneo]

The Gradle Test Delegation (both run and debug) has supported now.

To use this feature, you need to install the latest Test Runner for Java and Gradle for Java extension.

To delegate the tests to Gradle, you can set the default testing profile in Testing explorer:

If you do not want to change the default testing profile, you can trigger an one-time execution via: