Skip to content

Commit 9523361

Browse files
Document & expose printing of custom AbstractTestSet (#53215)
This was previously alluded to in the docs as being possible ("just record if we're not the top-level parent"), but trying to do that didn't actually do anything because the printing in the `DefaultTestSet` ignored anything that wasn't a `Test.Result` or `Test.DefaultTestSet`. This is particularly problematic if any of the child-testsets had failures, because those failures would never be shown to a user. Co-authored-by: Sukera <[email protected]> Co-authored-by: Ian Butterworth <[email protected]>
1 parent 65d4ce2 commit 9523361

File tree

4 files changed

+328
-75
lines changed

4 files changed

+328
-75
lines changed

stdlib/Test/docs/src/index.md

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -322,8 +322,12 @@ function finish(ts::CustomTestSet)
322322
# just record if we're not the top-level parent
323323
if get_testset_depth() > 0
324324
record(get_testset(), ts)
325+
return ts
325326
end
326-
ts
327+
328+
# so the results are printed if we are at the top level
329+
Test.print_test_results(ts)
330+
return ts
327331
end
328332
```
329333

@@ -338,6 +342,45 @@ And using that testset looks like:
338342
end
339343
```
340344

345+
In order to use a custom testset and have the recorded results printed as part of any outer default testset,
346+
also define `Test.get_test_counts`. This might look like so:
347+
348+
```julia
349+
using Test: AbstractTestSet, Pass, Fail, Error, Broken, get_test_counts, TestCounts, format_duration
350+
351+
function Test.get_test_counts(ts::CustomTestSet)
352+
passes, fails, errors, broken = 0, 0, 0, 0
353+
# cumulative results
354+
c_passes, c_fails, c_errors, c_broken = 0, 0, 0, 0
355+
356+
for t in ts.results
357+
# count up results
358+
isa(t, Pass) && (passes += 1)
359+
isa(t, Fail) && (fails += 1)
360+
isa(t, Error) && (errors += 1)
361+
isa(t, Broken) && (broken += 1)
362+
# handle children
363+
if isa(t, AbstractTestSet)
364+
tc = get_test_counts(t)::TestCounts
365+
c_passes += tc.passes + tc.cumulative_passes
366+
c_fails += tc.fails + tc.cumulative_fails
367+
c_errors += tc.errors + tc.cumulative_errors
368+
c_broken += tc.broken + tc.cumulative_broken
369+
end
370+
end
371+
# get a duration, if we have one
372+
duration = format_duration(ts)
373+
return TestCounts(true, passes, fails, errors, broken, c_passes, c_fails, c_errors, c_broken, duration)
374+
end
375+
```
376+
377+
```@docs
378+
Test.TestCounts
379+
Test.get_test_counts
380+
Test.format_duration
381+
Test.print_test_results
382+
```
383+
341384
## Test utilities
342385

343386
```@docs

0 commit comments

Comments
 (0)