Class test items displayed in TestResults display no output

[adam-fowler]

Describe the bug
When you select a test item associated with a class in the TestResults pane. It outputs

The test case did not report any output.

Expected behavior
It should either display the results for all the tests in that class, or the test item should not be displayed in the TestResults pane.

[plemarquand]

Its there to support a single binary test product for projects that contain both XCTest and swift-testing tests. @grynspan knows more details.

More information here: swiftlang/swift-package-manager#7766

[grynspan]

Sorry, how can I help?

[plemarquand]

I guess I'm wondering about the genesis of the swiftpm-testing-helper. It was created as a test runner for both XCTest and Swift-testing tests so they could rub under the same executable, correct?

[grynspan]

Not so much! So, on Linux and Windows, the test product is a flat executable and runs directly. SwiftPM emits a main() function into the executable that calls into XCTest or into Swift Testing depending on what arguments are passed. There is no helper executable needed.

Now, one of our second-tier goals with Swift Testing was for it to share as much logic as possible across platforms. So on all platforms regardless of what kind of test product is produced, the main() function Swift Testing emits into the binary is identical.

However, on Darwin, the test product expected by XCTest is a loadable, but not executable, bundle. The main() function from such a bundle cannot be directly executed—it's just another exported symbol in a library, after all.

Xcode includes an executable, aptly named xctest, that knows how to open these bundles, find XCTest tests, and run them. (There are also other ways to run tests via Xcode but they're not relevant to SwiftPM.) XCTest doesn't use the main() function at all! It also doesn't know anything about Swift Testing or how to discover or run its tests.

Enter swiftpm-testing-helper: it loads an arbitrary loadable bundle/library, finds its main() function, and calls it. It does nothing else. The main() function, being emitted into the test product exactly the same for Darwin as for Linux and Windows, takes over and runs Swift Testing tests.

In a perfect world, Darwin would let us make .xctest bundles both loadable and executable, and we wouldn't need the helper, but for other unrelated reasons that would be a terrible and dangerous idea, so them's the breaks.