Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store tests results as json artefact (for the rsmp gem workflow) #495

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

emiltin
Copy link
Contributor

@emiltin emiltin commented Jan 9, 2025

Adds a json output when running the rspec command. The resulting file is then uploaded as an artefact.

@emiltin emiltin added the tooling GitHub Actions, deployment and other internal tooling label Jan 9, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@emiltin
Copy link
Contributor Author

emiltin commented Jan 9, 2025

Next step is to associate the output artefacts with the core and sxl version used when testing, so the doc generation can show which versions were tested.

Currently the results is saved to a file with names like:

results-gem-tlc-ubuntu-latest-core-3.1.3-run-12689397028-177-1.json
results-gem-tlc-ubuntu-latest-core-3.2.2-run-12689393344-132-1.json

This includes the core version, but not the sxl. The reason is that the core version is set by the github action matrix, but the sxl is set in the test config. We could perhaps read the sxl from the config and include it in the artefact name.

But it might not be convenient or even possible to list available artefact, so we might need another way to directly list the core and sxl verisons tested. Maybe a separate action step can handle this and create an artefact that lists the core/sxl versions tested.

@emiltin
Copy link
Contributor Author

emiltin commented Jan 16, 2025

It's possible to output annotations. See e.g. https://github.com/Drieam/rspec-github.
Should check if that's a better way to output results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tooling GitHub Actions, deployment and other internal tooling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant