Test Documentation Generator is a maven-plugin that delivers a set of annotation for your test code.
You can annotate your test classes and test methods and provide details about test scenario.
Plugin itself then parse the data from the annotations and generates Markdown
files for readable documentation and fmf
format metadata for automated reporting of the test cases to external systems.
To generate documentation of the test class, you can use the following annotations:
@SuiteDoc
- is the main annotation, which consists of all other annotation and should be used right above the method. It contains three fields -description
,steps
, andlabels
, that are set using other annotations.@Desc
- overall description of the test, it can contain anything.@Contact
- contact info with fieldsname
andemail
.@Step
- particular step done in a test, contains two fields -value
that contains the step information,expected
is for the expected result of the step.@Label
- label used for specify area of tests defined within the test class.
Example of how the test can be annotated:
@SuiteDoc(
description = @Desc("My test suite containing various tests"),
contact = @Contact(name = "Jakub Stejskal", email = "[email protected]"),
beforeTestSteps = {
@Step(value = "Deploy uber operator across all namespaces, with custom configuration", expected = "Uber operator is deployed"),
@Step(value = "Deploy management Pod for accessing all other Pods", expected = "Management Pod is deployed")
},
afterTestSteps = {
@Step(value = "Delete management Pod", expected = "Management Pod is deleted"),
@Step(value = "Delete uber operator", expected = "Uber operator is deleted")
},
labels = {
@Label(value = "upgrade"),
@Label(value = "clients")
}
)
public static class DummyTest {
// ...
}
To generate documentation of the test method, you can use the following annotations:
@TestDoc
- is the main annotation, which consists all other annotation and should be used right above the method. It contains three fields -description
,steps
, andlabels
, that are set using other annotations.@Desc
- overall description of the test, it can contain anything.@Contact
- contact info with fieldsname
andemail
.@Step
- particular step done in a test, contains two fields -value
that contains the step information,expected
is for the expected result of the step.@Label
- label used for group tests covers similar areas.
Example of how the test can be annotated:
@TestDoc(
description = @Desc("Test checking that the application works as expected."),
contact = @Contact(name = "Jakub Stejskal", email = "[email protected]"),
steps = {
@Step(value = "Create object instance", expected = "Instance of an object is created"),
@Step(value = "Do a magic trick", expected = "Magic trick is done with success"),
@Step(value = "Clean up the test case", expected = "Everything is cleared"),
@Step(value = "Do a magic cleanup check", expected = "Everything magically work")
},
labels = {
@Label(value = "security"),
@Label(value = "upgrade"),
}
)
void testMyCode(ExtensionContext extensionContext) {
// ...
}
Syntax grammar (i.e., EBNF) of the test case
// Lexer rules
WS : [ \t\r\n]+ -> skip; // Ignore whitespace, tabs, carriage returns, and newlines
STRING : '"' (~["\\])* '"'; // Matches quoted strings correctly handling all characters
NUMBER : [0-9]+; // For numbers, if needed
// Parser rules
testDocAnnotation : '@TestDoc' '(' testDocBody ')';
testDocBody : testDocAttribute ( ',' testDocAttribute )* ;
testDocAttribute : descriptionAttribute
| contactAttribute
| stepsAttribute
| labelsAttribute
;
descriptionAttribute : 'description' '=' '@Desc' '(' STRING ')';
contactAttribute : 'contact' '=' '@Contact' '(' contactBody ')';
contactBody : 'name' '=' STRING ',' 'email' '=' STRING;
stepsAttribute : 'steps' '=' '{' step ( ',' step )* '}';
step : '@Step' '(' 'value' '=' STRING ',' 'expected' '=' STRING ')';
labelsAttribute : 'labels' '=' '{' label ( ',' label )* '}';
label : '@Label' '(' 'value' '=' STRING ')';
The generation is handled by maven-plugin that is available at Maven central.
All released versions could be found there.
For every commit we also publish -SNAPSHOT
version to GitHub packages.
To start using the plugin, you will need to add it to your pom file together with maven-dependency-plugin
for building the dependencies of the test classes:
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>io.skodjob</groupId>
<artifactId>test-docs-generator-maven-plugin</artifactId>
<version>${generator.version}</version>
<executions>
<execution>
<phase>post-integration-test</phase>
<goals>
<goal>test-docs-generator</goal>
</goals>
</execution>
</executions>
<configuration>
<testsPath>./dummy-module/src/test/java/io/</testsPath>
<docsPath>./docs/</docsPath>
<generateFmf>true</generateFmf>
</configuration>
</plugin>
maven-dependency-plugin
is needed for proper loading the classes.
Without it the plugin will return NoClassDefFound
exception and will fail.
Plugin works with the following parameters:
testsPath
- path to the built classes, from where all the names of the tests are taken.docsPath
- path to the place where the documentation should be generated.generateFmf
- boolean value whether generator should generate alsofmf
metadata or justMarkdown
.generateDirs
- boolean value whether generator should generate folder for each part of package name or not.
To use -SNAPSHOT
versions you have to have the plugin built on your local environment or use GitHub packages for a dependency resolution.
You can for example create the following setting.xml
:
<settings>
<servers>
<server>
<id>github</id>
<username>x-access-token</username>
<password>${env.GITHUB_TOKEN}</password>
</server>
</servers>
<pluginGroups>
<pluginGroup>io.skodjob</pluginGroup>
</pluginGroups>
</settings>
And specify GitHub in your pom:
<repositories>
<repository>
<id>github</id>
<name>GitHub Apache Maven Packages</name>
<url>https://maven.pkg.github.com/skodjob/test-metadata-generator</url>
</repository>
</repositories>
Each label could be described in separated file with more details.
In case the files with name in format <LABEL>.md
exists in directory labels
in your test-docs path defined via docsPath
, generator will link these files.
Anyone will be able to see the list of tests that covers a specific label.
Each test case documentation will allow to users to simply click on label and see its definition.
In case the file doesn't exist, the (description file doesn't exist)
will be put right after the label. For example: clients (description file doesn't exist)
.
This function is only part of mark-down docs generation.