Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add reproducibility percentage to daily build summary #1113

Open
wants to merge 50 commits into
base: master
Choose a base branch
from
Open

Add reproducibility percentage to daily build summary #1113

wants to merge 50 commits into from

Conversation

adamfarley
Copy link
Contributor

Where we have implemented testing for a reproducible build, this code ensures that the results of that testing are included in the daily build summary.

If we did try to run the tests, but failed for whatever reason, we provide a short summary of the relevant platforms and provide a lot more information in the "details" link to the jenkins job that generates the summary, as well as useful links.

Resolves #1091

@adamfarley
First draft of reproducibility result parser
2cb8a03 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Bug fix
a1dc161 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Refactoring
b0f5f3b 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Refactor
5ad9759 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Bug hunting
51bccf9 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
More bug hunting
0fff5aa 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo fix
16caecb 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo fix
193eea3 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo
cf8899c 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Debug output
481c6f0
@adamfarley
Typo
70d55b3 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo
ca6f841 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo
d278f31 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Debug output
44f4436
@adamfarley
Debug messages
e0fd8c7
@adamfarley
temporary change for debugging
fb2b253 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Bug fix
ee4a206 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Debug
d6578ec
@adamfarley
Bug fix
173ad03 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Determining reproducibility percentages
997b224 
By platform and version, with the addition of a useful map
for easy conversion between platform formats.

Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Too many square brackets
cf6c2bf 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Unexpected character - not enough backslashes?
6ec3f5c
@adamfarley
Making backslash escapes uniform
0a858fa 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Escaping non-variable dollar sign
e9c4de7 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Escaping another dollar sign
2acfbcc 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo fix
c7cfc76 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Another typo
aae584c 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Adding comment
d332f76 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing superfluous def
5706bdf 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Debug messages
7be6cbf 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Debug
8dbfa9f 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Fix for permissions issue accessing pipelines
9483f78 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Making platformConversionMap more accessible
dab5473 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Fixing typo in platform names
748a20c 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Fixing regex
f07bf61 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Another Typo
6dade4e 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Making escapes consistent
f2daa16 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing reference to restricted method
2b0b6ea 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Finding better way to test for match
706c83b 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing reference to out-of-scope variable
934c956 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing debug messages and tidying output
a240955 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Syntax typo fix
dd35d92 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Typo fix
530ac8e 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Re-adding test code
a19c797 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing use of init, which jenkins disallows
f15069c 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Improved the output to make it more concise and useful
4460a60 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
More concise output
9e04805 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Removing the last of the debug code
44d4bd9 
Signed-off-by: Adam Farley <[email protected]>
@adamfarley
Correcting enable tests value
de2fcb6 
Signed-off-by: Adam Farley <[email protected]>
@github-actions GitHub Actions
Copy link

Thank you for creating a pull request!

Please check out the information below if you have not made a pull request here before (or if you need a reminder how things work).

Code Quality and Contributing Guidelines

If you have not done so already, please familiarise yourself with our Contributing Guidelines and Code Of Conduct, even if you have contributed before.

Tests

Github actions will run a set of jobs against your PR that will lint and unit test your changes. Keep an eye out for the results from these on the latest commit you submitted. For more information, please see our testing documentation.

In order to run the advanced pipeline tests (executing a set of mock pipelines), it requires an admin to post run tests on this PR.
If you are not an admin, please ask for one's attention in #infrastructure on Slack or ping one here.
To run full set of tests, use "run tests"; a subset of tests on specific jdk version, use "run tests quick 11,21"

@github-actions github-actions bot added code-tools Issues that are miscellaneous enhancements or bugs with our utilities that assist our build scripts generation Issues that provide enhancements or fixes to the job generators testing labels Sep 17, 2024
@adamfarley adamfarley mentioned this pull request Sep 17, 2024
Open
andrew-m-leonard
andrew-m-leonard reviewed Sep 19, 2024
View reviewed changes
tools/nightly_build_and_test_stats.groovy
def platformResult = "???% - Build not found. Pipeline link: " + pipelineLink

// Does this platform have a build in this pipeline?
if ( trssBuildJobNames.length() > 10 ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure of the significance of "10" here?

andrew-m-leonard
andrew-m-leonard reviewed Sep 19, 2024
View reviewed changes
tools/nightly_build_and_test_stats.groovy
def trssTestJobNames = sh(returnStdout: true, script: "wget -q -O - ${trssURL}/api/getAllChildBuilds?parentId=${buildJob._id}\\&buildNameRegex=^${testJobTitle}\$")

// Did this build have tests?
if ( trssTestJobNames.length() > 10 ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto "10", if significant maybe add reason in comment?

andrew-m-leonard
andrew-m-leonard reviewed Sep 19, 2024
View reviewed changes
Copy link
Contributor

@andrew-m-leonard andrew-m-leonard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment on the significance of "10" ?

@andrew-m-leonard
Copy link
Contributor

I've updated the live Slack job to use this PR, to see how it looks for a few days...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrew-m-leonard andrew-m-leonard andrew-m-leonard left review comments

@karianna karianna karianna approved these changes

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
code-tools Issues that are miscellaneous enhancements or bugs with our utilities that assist our build scripts generation Issues that provide enhancements or fixes to the job generators testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Update Adoptium #build daily Slack status with Reproducible build results
3 participants
@adamfarley @andrew-m-leonard @karianna