feat: export smoke test results in prometheus format #1030
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In addition to the html report, the smoke test results can now be exported in prometheus format. The exporter is implemented as a custom Playwright reporter that writes the results to a file and optionally sends them to a Prometheus Pushgateway. The configuration for the Prometheus Pushgateway is read from environment variables to enable easy integration with CI/CD using secrets. This should ensure easy integration of the smoke tests into existing monitoring solutions by either configuring the Prometheus Pushgateway in the CI workflow or by scraping the results file from the Github Actions artifacts. Additionally, we enable the json reporter for the smoke tests to provide more easily parseable results for further processing if needed. All reporters are now configured to save their output to the `webui//playwright-report` directory to easily distinguish them from the test result artifacts, such as screenshots and videos, that are saved in the `webui/test-results` directory. Contributed on behalf of STMicroelectronics Signed-off-by: Olaf Lessenich <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In addition to the html report, the smoke test results can now be exported in prometheus format. The exporter is implemented as a custom Playwright reporter that writes the results to a file and optionally sends them to a Prometheus Pushgateway.
The configuration for the Prometheus Pushgateway is read from environment variables to enable easy integration with CI/CD using secrets.
This should ensure easy integration of the smoke tests into existing monitoring solutions by either configuring the Prometheus Pushgateway in the CI workflow or by scraping the results file from the Github Actions artifacts.
Additionally, we enable the json reporter for the smoke tests to provide more easily parseable results for further processing if needed.
All reporters are now configured to save their output to the
webui//playwright-reportdirectory to easily distinguish them from the test result artifacts, such as screenshots and videos, that are saved in thewebui/test-resultsdirectory.Contributed on behalf of STMicroelectronics