-
Notifications
You must be signed in to change notification settings - Fork 804
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Fix Test workflow concurrency for code coverage #41061
Conversation
A few adjustments are needed to make the coverage stuff more reliable: 1. For trunk runs, don't cancel for concurrency. We need each trunk commit to upload. 2. If the actual run is cancelled, don't upload partial data.
Thank you for your PR! When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:
This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖 The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available. Follow this PR Review Process:
Still unsure? Reach out in #jetpack-developers for guidance! |
@@ -54,7 +56,7 @@ jobs: | |||
|
|||
# Note matrix-job outputs are kind of weird. Last-to-run job that sets a non-empty value wins. | |||
outputs: | |||
did-coverage: ${{ ( steps.process-coverage.conclusion == 'success' && steps.upload-artifacts.conclusion == 'success' ) && 'true' || '' }} | |||
did-coverage: ${{ ( steps.run-tests.conclusion != 'cancelled' && steps.process-coverage.conclusion == 'success' && steps.upload-artifacts.conclusion == 'success' ) && 'true' || '' }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we even try to run process-coverage
if run-tests
is cancelled? 🤔 Though I guess if it runs that's an easy way to determine if coverage was enabled and I'm not sure if matrix.script == 'test-coverage'
is accessible here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For manual analysis, it probably doesn't hurt to save whatever coverage data we did manage to generate. But, particularly for trunk runs, it'll throw things off if that partial data gets uploaded.
The check here is now threefold: (1) we finished running the actual tests, (2) we successfully processed the coverage data, and (3) we successfully uploaded the artifact. If 2 or 3 is false, the upload job won't even be able to run successfully because there will be no data. If 1 is false, it's partial data that we don't want to upload.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems reasonable.
Proposed changes:
A few adjustments are needed to make the coverage stuff more reliable:
Other information:
Jetpack product discussion
Does this pull request change what data or activity we track or use?
Testing instructions: