Update top level README.md so it is more consumable for new users

[sxa]

Various changes:

• Focus on the different build pipeline types so they can be understood, and include:

  • How they are structured in jenkins
  • How they are triggered
  • Which pipelines are related to each one

• Move the details of the configuration files into a separate readme file

• Collapse the section on the metadata generation (Should that be moved elsewhere? It's fully duplicated in the temurin-build README.md although with some formatting changes (e.g. that one still references "openjdk-build" - the repo had those references updated in https://github.com/adoptium/ci-jenkins-pipelines/pull/285/files). Differences here:
  diff.txt

• Add links to the markdown files in the docs directory (Maybe they need a once-over too to verify that the information is still up to date. Some of the diagrams could potentially be inlined into the top level README)

Further items to be clarified via the TODO items in this PR.
Note: The README.md in the configuration directory has been forked from the top level README.md using the process from https://stackoverflow.com/questions/1043388/record-file-copy-operation-with-git/46484848#46484848 to preserve the "forked" file's history.

For discussion:

• What naming convention should we use for the top level pipelines (openjdkXX-pipelines) vs individual build pipelines (jdkXX-linux-aarch64-temurin)
• Add a small diagram of the folder structure?
• Add evaluation jobs to the table that the bottom? Plus JDk22/23? (Also RISC-V needs a bit of work now it's production)

[github-actions]

Thank you for creating a pull request!

Please check out the information below if you have not made a pull request here before (or if you need a reminder how things work).

Code Quality and Contributing Guidelines

If you have not done so already, please familiarise yourself with our Contributing Guidelines and Code Of Conduct, even if you have contributed before.

Tests

Github actions will run a set of jobs against your PR that will lint and unit test your changes. Keep an eye out for the results from these on the latest commit you submitted. For more information, please see our testing documentation.

In order to run the advanced pipeline tests (executing a set of mock pipelines), it requires an admin to post run tests on this PR.
If you are not an admin, please ask for one's attention in #infrastructure on Slack or ping one here.
To run full set of tests, use "run tests"; a subset of tests on specific jdk version, use "run tests quick 11,21"

[sxa]

More 'TODO:' / "Memo to self" items:

• Add information on buildScripts (~800 lines starting at 1500) vs build (which is inside buildScripts and starts at around line 1930 and goes to the end and contains the queue stage) in openjdk_build_pipeline.groovy
• Add something about the buildConfig object import common.IndividualBuildConfig (which is the parameter passed in during the job execution)
• useAdoptShellScripts value? It's in an if block which is over 150 lines in openjdk_build_pipeline_groovy and covers signing - the else` block is about 20 lines.
• There are four invocations of ./${DEFAULTS_JSON['scriptDirectories']['buildfarm']}" - perhaps they should all be explained?
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1623 in d33d46f

  if (useAdoptShellScripts) {

  
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1627 in d33d46f

  if ((buildConfig.TARGET_OS == 'mac' || buildConfig.TARGET_OS == 'windows') && buildConfig.JAVA_TO_BUILD != 'jdk8u') {

  
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1637 in d33d46f

  context.sh(script: "./${ADOPT_DEFAULTS_JSON['scriptDirectories']['buildfarm']}")

  ^^^ Building an exploded image for signing
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1754 in d33d46f

  context.sh(script: "./${ADOPT_DEFAULTS_JSON['scriptDirectories']['buildfarm']}")

  ^^^ Assembling the exploded and signed image
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1756 in d33d46f

  } else {

  ^^^ Not windows
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1764 in d33d46f

  context.sh(script: "./${ADOPT_DEFAULTS_JSON['scriptDirectories']['buildfarm']}")

  
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1776 in d33d46f

  } else {

  ^^^ Use Adopt scripts = false
  

  ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

  Line 1788 in d33d46f

  context.sh(script: "./${DEFAULTS_JSON['scriptDirectories']['buildfarm']}")

Docker handling is in the queue phase:

ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

Line 2006 in d33d46f

if (buildConfig.DOCKER_IMAGE) {

and the non-docker situation is at

ci-jenkins-pipelines/pipelines/build/common/openjdk_build_pipeline.groovy

Line 2148 in d33d46f

} else {

build() invokes buildScripts() either directly or under a context.docker.build(). On Windows, this causes a problem as there is a context('eclipse-codesign') within buildScripts() which cannot handle being run in a docker context.
formMetadata/writeMetadata notes

[steelhead31]

Looks good, I like that its been split like this :)

[sxa]

(Force push done via the UI to in order to rebase, and re-trigger linter checks)
Noting that my earlier comments about things that should be explained will be a lot clearer with the eye catchers (openjdk_build_pipeline:) in #1117