Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: separated updatecli pipeline #606

Conversation

lemeurherve
Copy link
Member

@lemeurherve lemeurherve commented Apr 18, 2023

@lemeurherve lemeurherve enabled auto-merge (squash) April 18, 2023 10:16
@lemeurherve
Copy link
Member Author

Copy link
Contributor

@dduportal dduportal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good idea, but you can't merge "as it": this PR break the whole updatecli for this repository, because it changes the environment in which updatecli was executed before so it could, or could not, break things.

The current main branch currently executes updatecli inside a VM agent which label is linux-amd64-docker:

  • Top-level agent directive (e.g. default agent):
    label "linux-amd64-docker"
  • No agent directive in the Side Task "top level" stage:
    stage('Side Tasks') {
    environment {
    DRYRUN = "${env.BRANCH_IS_PRIMARY ? 'false' : 'true'}"
    }
    parallel {
    stage('Packer Init') {
    steps {
    // Call the initializing function once for the default agent
    script {
    packerInitPlugins()
    }
    }
    }
    stage('GC on AWS us-east-2') {
    agent {
    kubernetes {
    yaml podAgentDefinition
    }
    }
    environment {
    AWS_ACCESS_KEY_ID = credentials('packer-aws-access-key-id')
    AWS_SECRET_ACCESS_KEY = credentials('packer-aws-secret-access-key')
    AWS_DEFAULT_REGION = 'us-east-2'
    }
    steps {
    catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
    sh './cleanup/aws.sh'
    sh './cleanup/aws_images.sh 1 dev'
    sh './cleanup/aws_images.sh 7 staging'
    sh './cleanup/aws_images.sh 60 prod'
    sh './cleanup/aws_snapshots.sh'
    }
    }
    }
    stage('GC on Azure') {
    agent {
    kubernetes {
    yaml podAgentDefinition
    }
    }
    environment {
    PACKER_AZURE = credentials('packer-azure-serviceprincipal')
    }
    steps {
    catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
    sh 'az login --service-principal -u "$PACKER_AZURE_CLIENT_ID" -p "$PACKER_AZURE_CLIENT_SECRET" -t "$PACKER_AZURE_TENANT_ID"'
    sh 'az account set -s "$PACKER_AZURE_SUBSCRIPTION_ID"'
    sh './cleanup/azure.sh'
    }
    }
    }
    stage('Updatecli') {
    steps {
    catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
    script {
    // TODO: Implement https://github.com/jenkins-infra/pipeline-library/issues/518 to allow using the updatecli() library function
    withCredentials([
    usernamePassword(
    credentialsId: 'github-app-updatecli-on-jenkins-infra',
    usernameVariable: 'USERNAME_VALUE', // Setting this variable is mandatory, even if of not used when the credentials is a githubApp one
    passwordVariable: 'UPDATECLI_GITHUB_TOKEN'
    )
    ]) {
    sh 'updatecli version'
    sh 'updatecli diff --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
    if (env.BRANCH_IS_PRIMARY) {
    sh 'updatecli apply --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
    }
    }
    }
    }
    }
    }
    }
    }
  • No agent directive in the updatecli stage:
    stage('Updatecli') {
    steps {
    catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
    script {
    // TODO: Implement https://github.com/jenkins-infra/pipeline-library/issues/518 to allow using the updatecli() library function
    withCredentials([
    usernamePassword(
    credentialsId: 'github-app-updatecli-on-jenkins-infra',
    usernameVariable: 'USERNAME_VALUE', // Setting this variable is mandatory, even if of not used when the credentials is a githubApp one
    passwordVariable: 'UPDATECLI_GITHUB_TOKEN'
    )
    ]) {
    sh 'updatecli version'
    sh 'updatecli diff --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
    if (env.BRANCH_IS_PRIMARY) {
    sh 'updatecli apply --values ./updatecli/values.yaml --config ./updatecli/updatecli.d'
    }
    }
    }
    }

Also, don't forget that the updatecli() pipeline library method takes care of spawning its own pod agent (default image is https://github.com/jenkins-infra/pipeline-library/blob/9e0f0a2f09a7c91b26824e706a8963598b685742/vars/updatecli.groovy#L10, you can change it)

=> There was a good reason to NOT use the pipeline library before (the comment points to an issue :) )
=> Proposal: use the same pipeline code as before , but in the new pipeline for this PR to work.
And/or implement the feature request in the shared library.

=> both case needs you to use agent { label ... } to spin up a VM with Docker inside otherwise you break https://github.com/jenkins-infra/packer-images/blob/main/updatecli/updatecli.d/docker-ce.yaml

lemeurherveCB and others added 9 commits April 18, 2023 17:51
…infra#608

Revert "Revert "feat(azure/arm): enable arm64 azure build" (jenkins-infra#607)"

This reverts commit 302702d.
* feat: add Launchable

* remove python3 version pinning on Linux

* add python3 path

* fixes

* Apply suggestions from review

Co-authored-by: Damien Duportal <[email protected]>

* try with Script path

* simple python filename

* fix Windows paths

* symbolic link on launchable into /usr/local/bin

* install launchable globally on Windows

* fix python scripts path

* remove launchable sanity check without fullpath

* cleanup

* Apply suggestion from review

* Update provisioning/ubuntu-provision.sh

---------

Co-authored-by: Damien Duportal <[email protected]>
chore: Update key "awscli_version" from file "/tmp/updatecli/github/j...

... enkins-infra/packer-images/provisioning/tools-versions.yml"

Made with ❤️️ by updatecli

Co-authored-by: Jenkins Infra Bot (updatecli) <[email protected]>
Co-authored-by: Damien Duportal <[email protected]>
chore: Update key "windows_pwsh_version" from file "/tmp/updatecli/gi...

... thub/jenkins-infra/packer-images/provisioning/tools-versions.yml"

Made with ❤️️ by updatecli

Co-authored-by: Jenkins Infra Bot (updatecli) <[email protected]>
Co-authored-by: Damien Duportal <[email protected]>
chore: Update key "updatecli_version" from file "/tmp/updatecli/githu...

... b/jenkins-infra/packer-images/provisioning/tools-versions.yml"

Made with ❤️️ by updatecli

Co-authored-by: Jenkins Infra Bot (updatecli) <[email protected]>
@lemeurherve lemeurherve marked this pull request as draft April 19, 2023 17:24
kubernetes {
yamlFile 'podTemplate.yaml'
}
docker { image 'jenkinsciinfra/helmfile:2.5.92' }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
docker { image 'jenkinsciinfra/helmfile:2.5.92' }
label "linux-amd64-docker"

This updatecli process requires a Docker Engine, which means it requires a VM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants