feat: add support for evaluation jobs/pipelines #520

zdtsw · 2022-11-29T16:20:05Z

Highlight change:

split current jdkX.groovy config into two parts:

jdkX.groovy will be used for nightly builds
jdkX_evaluation.groovy will also be run as the same time as jdkX.groovy, but only contain unstable/non-temurin variant builds

daily build stats should only count whats in the jdkX.groovy, (can be another type of build stats if required for evaluation, and we already have ones for openj9 etc variants)
weekly archive should only use jdkX.groovy as input and archive results
introduce new weekly-evaluation use jdkX_prototpye.groovy as input for weekly cycle long test.

Detail Config updates
aarch64 windows is added back as part of evaluation
openj9, etc are moved into evaluation
official release of temurin should refer to whats stated in jdkX_release.groovy

in short:

jdkX_release.groovy is the one we used to include what we release to customer, it is only used for release time
jdkX.groovy included everything from jdkX_release + some others we do not release but should be counted into stats
jdkX_evaluation.groovy has config not in either of above two config files: not ready to be included into nightly, not temurin builds, etc

Trigger updates

openjdkX-pipeline relesaseType to Nightly use triggerSchedule_nightly
weekly-openjdkX-pipeline relesaeType to "Release" triggerSchedule_weekly
weekly-prototpye-openjdkX-pipeline releasetype to Nightly Not Publish use triggerSchedule_weekly_evaluation: this will disable the remotetrigger to JCK (isRelease = false)
evaluation-openjdkX-pipeline releaseType to Nightly Not Publish use triggerSchedule_evaluation

first two share the same config file and targetConfiguration, later two share the same config file and targetConfiguration

Test results
Test pipeline generator: https://ci.adoptopenjdk.net/view/work-in-progress/job/wen-build-pipeline-generator/

generates build-scripts-testing/evaluation-openjdk20-pipeline

Tet job generator: https://ci.adoptopenjdk.net/view/work-in-progress/job/wen-evaluation-pipeline_jobs_generator_jdk/

generates build-scripts-testing/evaluation/jobs/jdk20/jdk20-evaluation-linux-aarch64-hotspot etc

result job to create downstream evaluation jobs

pipeline generator relation
build-pipeline-generator https://ci.adoptopenjdk.net/job/build-scripts/job/utils/job/build-pipeline-generator/ will be used to generate all [weekly-]openjdkXX-pipeline => AS-IS no change
new evaluation-pipeline-generator will be used to generate new [weekly-]prototpye-openjdkXX-pipeline

to test it out function, created https://ci.adoptopenjdk.net/view/work-in-progress/job/wen-evaluation-pipeline-generator/

which generates

Post Merge Action

Once this PR is merged, https://ci.adoptopenjdk.net/job/build-scripts/job/utils/job/build-pipeline-generator/ should be triggered automatically and generate two new type of pipelines: prorotype and weekly-evaluation.
we manually enable https://ci.adoptopenjdk.net/job/build-scripts/job/utils/job/evaluation-pipeline_jobs_generator_jdk/ and others. (at the moment, they are disabled)
someone with high permission should do a cleanup in the https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/ (and other version folder) for the jobs moved to evaluation
smoketest will be auto created when the first time downstream build job runs. e.g https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/evaluation/job/jobs/job/jdk20/job/jdk20-evaluation-linux-aarch64-hotspot/

Ref: #473

github-actions · 2022-11-29T16:20:17Z

Thank you for creating a pull request!

Please check out the information below if you have not made a pull request here before (or if you need a reminder how things work).

Code Quality and Contributing Guidelines

If you have not done so already, please familiarise yourself with our Contributing Guidelines and Code Of Conduct, even if you have contributed before.

Tests

Github actions will run a set of jobs against your PR that will lint and unit test your changes. Keep an eye out for the results from these on the latest commit you submitted. For more information, please see our testing documentation.

In order to run the advanced pipeline tests (executing a set of mock pipelines), it requires an admin to post run tests on this PR.
If you are not an admin, please ask for one's attention in #infrastructure on Slack or ping one here.
To run full set of tests, use "run tests"; a subset of tests on specific jdk version, use "run tests quick 11,19"

zdtsw · 2022-12-02T08:13:32Z

consider : #517 (comment)

zdtsw · 2022-12-12T12:14:57Z

run tests quick 19

eclipse-temurin-bot · 2022-12-12T13:29:47Z

 PR TESTER RESULT 

✅ All pipelines passed! ✅

andrew-m-leonard · 2022-12-20T10:13:09Z

pipelines/jobs/configurations/jdk17u_pipeline_config.groovy


+        aarch64Windows: [


Should this be added?

if we do not have it enabled in jdk17u.groovy or jdk17u_prototype.groovy, it wont be used in the build

@andrew-m-leonard if the question is "should Windows aarch64" be in the prototype pipeline then my answer is yes, please. I'm working on adding a second machine to the build farm this week to resolve the redundancy issues we had.

andrew-m-leonard · 2022-12-20T10:13:29Z

pipelines/jobs/configurations/jdk19u_pipeline_config.groovy


+        aarch64Windows: [


Not sure this should be added?

andrew-m-leonard · 2022-12-20T10:15:13Z

pipelines/jobs/configurations/jdk20_pipeline_config.groovy


+        aarch64Windows: [


Not sure should be added?

andrew-m-leonard

Just the query on aarch64Windows being added...?

zdtsw · 2022-12-20T10:23:50Z

Just the query on aarch64Windows being added...?

yes, think we can have these in the config.groovy and use jdkXu.groovy or jdkX_prototype.groovy as enabler.
actually it does not query API for the downstream jobs, but only the openjdk-pipeline.

…into issue/473

gdams · 2022-12-20T12:05:01Z

This all looks good to me, only suggestion is perhaps prototype isn't the best word. Maybe incubator or something?

zdtsw · 2022-12-20T12:29:38Z

This all looks good to me, only suggestion is perhaps prototype isn't the best word. Maybe incubator or something?

I dont mind changing name to incubator, how you think @andrew-m-leonard ?

smlambert · 2022-12-20T14:47:02Z

Throwing in the suggestion to use the term "evaluation pipeline", instead of prototype or incubator.

Suggested in #543 (comment)

Anything we do not release should go in the ~~prototype~~ ~~incubator~~ ~~nonrelease~~ ~~experimental~~ evaluation pipeline, since we are evaluating them for various reasons (other distros to aid in triage and improve automation / train models) and new platforms for stabilization.

zdtsw · 2022-12-21T14:15:06Z

Throwing in the suggestion to use the term "evaluation pipeline", instead of prototype or incubator.

Suggested in #543 (comment)

Anything we do not release should go in the ~~prototype~~ ~~incubator~~ ~~nonrelease~~ ~~experimental~~ evaluation pipeline, since we are evaluating them for various reasons (other distros to aid in triage and improve automation / train models) and new platforms for stabilization.

renamed from prototype to evaluation

karianna · 2022-12-28T08:02:44Z

@zdtsw Will need a rebase

zdtsw · 2022-12-29T15:59:36Z

@zdtsw Will need a rebase

merged master into issue/473, cannot do rebase because this branch issue/473 was created in origin repo, it will mess up anyone forked origin

karianna

LGTM

* feat: add support for evaluation jobs/pipelines - remove isPRBuilder and isReleaseByuilder, replace with new parameter jobType * update: set releaseType of prototype pipeline to "do not publish" * update: add creation of weekly-evalucation pipeline * update: add back win aarch64 to jdk20 prototype - move riscv from jdk20 nightly to evalucation Co-authored-by: Martijn Verburg <[email protected]>

github-actions bot added generation Issues that provide enhancements or fixes to the job generators jenkins-pipeline testing aarch windows labels Nov 29, 2022

github-actions bot added the openj9 label Dec 2, 2022

zdtsw force-pushed the issue/473 branch from 948ebdf to 13144e2 Compare December 2, 2022 09:00

github-actions bot added generation Issues that provide enhancements or fixes to the job generators testing and removed generation Issues that provide enhancements or fixes to the job generators testing labels Dec 2, 2022

zdtsw force-pushed the issue/473 branch from 13144e2 to 3778084 Compare December 8, 2022 15:34

github-actions bot added generation Issues that provide enhancements or fixes to the job generators testing and removed testing generation Issues that provide enhancements or fixes to the job generators labels Dec 8, 2022

zdtsw force-pushed the issue/473 branch from c81840c to 05ddaf0 Compare December 9, 2022 07:38

github-actions bot added testing and removed testing labels Dec 9, 2022

zdtsw changed the title ~~WIP: add support for prototype jobs/pipelines~~ feat: add support for prototype jobs/pipelines Dec 9, 2022

zdtsw force-pushed the issue/473 branch from 05ddaf0 to 1f6640e Compare December 9, 2022 07:58

github-actions bot added testing and removed testing labels Dec 9, 2022

zdtsw force-pushed the issue/473 branch from 58b6dca to 32fab1a Compare December 12, 2022 15:03

zdtsw added 2 commits December 16, 2022 14:59

update: add clear information

5a0c27d

docs: update

cffedf2

andrew-m-leonard reviewed Dec 20, 2022

View reviewed changes

pipelines/jobs/configurations/jdk19u_pipeline_config.groovy

aarch64Windows: [

Copy link

Contributor

andrew-m-leonard Dec 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this should be added?

andrew-m-leonard reviewed Dec 20, 2022

View reviewed changes

pipelines/jobs/configurations/jdk20_pipeline_config.groovy

aarch64Windows: [

Copy link

Contributor

andrew-m-leonard Dec 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure should be added?

andrew-m-leonard reviewed Dec 20, 2022

View reviewed changes

zdtsw requested a review from smlambert December 20, 2022 10:52

zdtsw added 3 commits December 20, 2022 12:06

Merge branch 'master' into issue/473

67755b1

Merge branch 'issue/473' of github.com:adoptium/ci-jenkins-pipelines …

b3a0f11

…into issue/473

fix: linter

14dc5a1

update: rename prototype to evaluation according to review comments

d60087f

update:rename file from prototype to evaluation

6c35778

zdtsw requested review from andrew-m-leonard and gdams December 21, 2022 14:27

zdtsw changed the title ~~feat: add support for prototype jobs/pipelines~~ feat: add support for evaluation jobs/pipelines Dec 21, 2022

gdams approved these changes Dec 21, 2022

View reviewed changes

zdtsw requested a review from steelhead31 December 23, 2022 09:33

Merge branch 'master' into issue/473

2205ded

smlambert mentioned this pull request Feb 14, 2023

jdk8 aarch64_alpine-linux test jobs fail due to DETECTED_JDK_VERSION mismatch from available artifact adoptium/infrastructure#2934

Closed

karianna approved these changes Jan 2, 2023

View reviewed changes

zdtsw merged commit 8e3800e into master Jan 2, 2023

adamfarley mentioned this pull request Jun 14, 2023

Removing chance jobReleaseType can cause duplicate values #729

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for evaluation jobs/pipelines #520

feat: add support for evaluation jobs/pipelines #520

zdtsw commented Nov 29, 2022 •

edited

Loading

github-actions bot commented Nov 29, 2022

zdtsw commented Dec 2, 2022

zdtsw commented Dec 12, 2022

eclipse-temurin-bot commented Dec 12, 2022

andrew-m-leonard Dec 20, 2022

zdtsw Dec 20, 2022

gdams Dec 20, 2022

andrew-m-leonard Dec 20, 2022

andrew-m-leonard Dec 20, 2022

andrew-m-leonard left a comment

zdtsw commented Dec 20, 2022 •

edited

Loading

gdams commented Dec 20, 2022

zdtsw commented Dec 20, 2022

smlambert commented Dec 20, 2022

zdtsw commented Dec 21, 2022

karianna commented Dec 28, 2022

zdtsw commented Dec 29, 2022

karianna left a comment

feat: add support for evaluation jobs/pipelines #520

feat: add support for evaluation jobs/pipelines #520

Conversation

zdtsw commented Nov 29, 2022 • edited Loading

github-actions bot commented Nov 29, 2022

Thank you for creating a pull request!

Code Quality and Contributing Guidelines

Tests

zdtsw commented Dec 2, 2022

zdtsw commented Dec 12, 2022

eclipse-temurin-bot commented Dec 12, 2022

 PR TESTER RESULT 

andrew-m-leonard Dec 20, 2022

Choose a reason for hiding this comment

zdtsw Dec 20, 2022

Choose a reason for hiding this comment

gdams Dec 20, 2022

Choose a reason for hiding this comment

andrew-m-leonard Dec 20, 2022

Choose a reason for hiding this comment

andrew-m-leonard Dec 20, 2022

Choose a reason for hiding this comment

andrew-m-leonard left a comment

Choose a reason for hiding this comment

zdtsw commented Dec 20, 2022 • edited Loading

gdams commented Dec 20, 2022

zdtsw commented Dec 20, 2022

smlambert commented Dec 20, 2022

zdtsw commented Dec 21, 2022

karianna commented Dec 28, 2022

zdtsw commented Dec 29, 2022

karianna left a comment

Choose a reason for hiding this comment

zdtsw commented Nov 29, 2022 •

edited

Loading

zdtsw commented Dec 20, 2022 •

edited

Loading