Skip to content

Commit

Permalink
(PE-39307) Legacy compilers support smoke tests (#505)
Browse files Browse the repository at this point in the history
* feat(workflow): add GitHub Actions workflow for converting compilers to legacy

- Trigger on pull requests affecting specific paths and on manual dispatch
- Provision and set up a test cluster using Bolt and Rake
- Install Puppet Enterprise on the test cluster
- Convert one compiler to legacy and verify the conversion
- Tear down the test cluster after the workflow completes

* feat(workflow): add GitHub Actions workflow for upgrading PE with legacy compilers

- Trigger on pull requests affecting specific paths and on manual dispatch
- Provision and set up a test cluster using Bolt and Rake
- Install Puppet Enterprise with legacy compilers on the test cluster
- Verify the configuration of legacy compilers
- Upgrade Puppet Enterprise and verify the configuration post-upgrade
- Tear down the test cluster after the workflow completes

* refactor(peadm_spec): improve error handling in add_inventory_hostnames plan

- Replace parallelize with map for better readability
- Add error handling for hostname command execution
- Add error handling for inventory file update command

* fix(workflow): correct legacy compiler selection in test-legacy-upgrade.yaml

- Replace `head -n 2` with `sed -n 2p` for accurate selection of the second compiler

* chore(workflows): remove push trigger from legacy workflows

The push trigger has been removed from the test-legacy-compilers and test-legacy-upgrade workflows. This change ensures that these workflows are only triggered manually via workflow_dispatch.

fix(workflow): correct conditional syntax in test-legacy-upgrade.yaml

- Replace incorrect `AND` with `&&` for accurate conditional checks
  • Loading branch information
CoMfUcIoS authored Sep 26, 2024
1 parent 39d5f7d commit 4596a6c
Show file tree
Hide file tree
Showing 3 changed files with 375 additions and 5 deletions.
161 changes: 161 additions & 0 deletions .github/workflows/test-legacy-compilers.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
---
name: Convert compiler to legacy
on:
pull_request:
paths:
- .github/workflows/**/*
- spec/**/*
- lib/**/*
- tasks/**/*
- functions/**/*
- types/**/*
- plans/**/*
- hiera/**/*
- manifests/**/*
- templates/**/*
- files/**/*
- metadata.json
- Rakefile
- Gemfile
- provision.yaml
- .rspec
- .rubocop.yml
- .puppet-lint.rc
- .fixtures.yml
branches: [main]
workflow_dispatch:
inputs:
ssh-debugging:
description: Boolean; whether or not to pause for ssh debugging
required: true
default: 'false'
jobs:
convert_compiler:
name: Convert compilers to legacy
runs-on: ubuntu-20.04
env:
BOLT_GEM: true
BOLT_DISABLE_ANALYTICS: true
LANG: en_US.UTF-8
steps:
- name: Start SSH session
if: ${{ github.event.inputs.ssh-debugging == 'true' }}
uses: luchihoratiu/debug-via-ssh@main
with:
NGROK_AUTH_TOKEN: ${{ secrets.NGROK_AUTH_TOKEN }}
SSH_PASS: ${{ secrets.SSH_PASS }}
- name: Checkout Source
uses: actions/checkout@v2
- name: Activate Ruby 2.7
uses: ruby/setup-ruby@v1
with:
ruby-version: '2.7'
bundler-cache: true
- name: Print bundle environment
if: ${{ github.repository_owner == 'puppetlabs' }}
run: |
echo ::group::info:bundler
bundle env
echo ::endgroup::
- name: Provision test cluster
timeout-minutes: 15
run: |
echo ::group::prepare
mkdir -p $HOME/.ssh
echo 'Host *' > $HOME/.ssh/config
echo ' ServerAliveInterval 150' >> $HOME/.ssh/config
echo ' ServerAliveCountMax 2' >> $HOME/.ssh/config
bundle exec rake spec_prep
echo ::endgroup::
echo ::group::provision
bundle exec bolt plan run peadm_spec::provision_test_cluster \
--modulepath spec/fixtures/modules \
provider=provision_service \
image=almalinux-cloud/almalinux-8 \
architecture=large-with-dr
echo ::endgroup::
echo ::group::certnames
bundle exec bolt plan run peadm_spec::add_inventory_hostnames \
--inventory spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
inventory_file=spec/fixtures/litmus_inventory.yaml
echo ::endgroup::
echo ::group::info:request
cat request.json || true; echo
echo ::endgroup::
echo ::group::info:inventory
sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true
echo ::endgroup::
- name: Set up yq
uses: frenck/action-setup-yq@v1
with:
version: v4.30.5
- name: Install PE on test cluster
timeout-minutes: 120
run: |
bundle exec bolt plan run peadm_spec::install_test_cluster \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
architecture=large-with-dr \
console_password=${{ secrets.CONSOLE_PASSWORD }} \
version=2023.7.0
- name: Wait as long as the file ${HOME}/pause file is present
if: ${{ always() && github.event.inputs.ssh-debugging == 'true' }}
run: |
while [ -f "${HOME}/pause" ] ; do
echo "${HOME}/pause present, sleeping for 60 seconds..."
sleep 60
done
echo "${HOME}/pause absent, continuing workflow."
- name: Convert one compiler to legacy
timeout-minutes: 120
run: |
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
echo "primary: $primary"
echo "compiler: $compiler"
echo ::group::convert_compiler_to_legacy
bundle exec bolt plan run peadm::convert_compiler_to_legacy \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
primary_host=$primary \
legacy_hosts=$compiler
echo ::endgroup::
- name: Check if compiler is converted
timeout-minutes: 120
run: |
echo ::group::inventory
sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true
echo ::endgroup::
echo ::group::get_peadm_config
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
bundle exec bolt task run peadm::get_peadm_config \
--targets $primary \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
--format json > peadm_config.json
cat peadm_config.json
echo ::endgroup::
echo ::group::smoke_test
legacy_compiler=$(yq '.items[0].value.params.legacy_compilers[0]' peadm_config.json)
if [ "$compiler" != "$legacy_compiler" ]; then
echo "Compiler conversion failed, expected $compiler, got $legacy_compiler"
exit 1
fi
echo ::endgroup::
- name: Tear down test cluster
if: ${{ always() }}
continue-on-error: true
run: |-
if [ -f spec/fixtures/litmus_inventory.yaml ]; then
echo ::group::tear_down
bundle exec rake 'litmus:tear_down'
echo ::endgroup::
echo ::group::info:request
cat request.json || true; echo
echo ::endgroup::
fi
204 changes: 204 additions & 0 deletions .github/workflows/test-legacy-upgrade.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
---
name: Upgrade PE with legacy compilers
on:
pull_request:
paths:
- .github/workflows/**/*
- spec/**/*
- lib/**/*
- tasks/**/*
- functions/**/*
- types/**/*
- plans/**/*
- hiera/**/*
- manifests/**/*
- templates/**/*
- files/**/*
- metadata.json
- Rakefile
- Gemfile
- provision.yaml
- .rspec
- .rubocop.yml
- .puppet-lint.rc
- .fixtures.yml
branches: [main]
workflow_dispatch:
inputs:
ssh-debugging:
description: Boolean; whether or not to pause for ssh debugging
required: true
default: 'false'
jobs:
upgrade_with_legacy_compilers:
name: Upgrade PE with legacy compilers
runs-on: ubuntu-20.04
env:
BOLT_GEM: true
BOLT_DISABLE_ANALYTICS: true
LANG: en_US.UTF-8
steps:
- name: Start SSH session
if: ${{ github.event.inputs.ssh-debugging == 'true' }}
uses: luchihoratiu/debug-via-ssh@main
with:
NGROK_AUTH_TOKEN: ${{ secrets.NGROK_AUTH_TOKEN }}
SSH_PASS: ${{ secrets.SSH_PASS }}
- name: Checkout Source
uses: actions/checkout@v2
- name: Activate Ruby 2.7
uses: ruby/setup-ruby@v1
with:
ruby-version: '2.7'
bundler-cache: true
- name: Print bundle environment
if: ${{ github.repository_owner == 'puppetlabs' }}
run: |
echo ::group::info:bundler
bundle env
echo ::endgroup::
- name: Provision test cluster
timeout-minutes: 15
run: |
echo ::group::prepare
mkdir -p $HOME/.ssh
echo 'Host *' > $HOME/.ssh/config
echo ' ServerAliveInterval 150' >> $HOME/.ssh/config
echo ' ServerAliveCountMax 2' >> $HOME/.ssh/config
bundle exec rake spec_prep
echo ::endgroup::
echo ::group::provision
bundle exec bolt plan run peadm_spec::provision_test_cluster \
--modulepath spec/fixtures/modules \
provider=provision_service \
image=almalinux-cloud/almalinux-8 \
architecture=large-with-dr
echo ::endgroup::
echo ::group::certnames
bundle exec bolt plan run peadm_spec::add_inventory_hostnames \
--inventory spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
inventory_file=spec/fixtures/litmus_inventory.yaml
echo ::endgroup::
echo ::group::info:request
cat request.json || true; echo
echo ::endgroup::
echo ::group::info:inventory
sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true
echo ::endgroup::
- name: Set up yq
uses: frenck/action-setup-yq@v1
with:
version: v4.30.5
- name: Create the params.json file
run: |
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
legacy_compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | sed -n 2p)
replica=$(yq '.groups[].targets[] | select(.vars.role == "replica") | .name' spec/fixtures/litmus_inventory.yaml)
echo -n '{ "download_mode": "direct", "primary_host": "'$primary'", "replica_host": "'$replica'", "legacy_compilers": ["'$legacy_compiler'"], "compiler_hosts": ["'$compiler'"], "version": "2023.7.0", "console_password": "'${{ secrets.CONSOLE_PASSWORD }}'" }' > params.json
- name: Install PE with legacy compilers
timeout-minutes: 120
run: |
echo ::group::params.json
jq '.console_password = "[redacted]"' params.json || true
echo ::endgroup::
echo ::group::install
bundle exec bolt plan run peadm::install \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
--params @params.json
echo ::endgroup::
- name: Wait as long as the file ${HOME}/pause file is present
if: ${{ always() && github.event.inputs.ssh-debugging == 'true' }}
run: |
while [ -f "${HOME}/pause" ] ; do
echo "${HOME}/pause present, sleeping for 60 seconds..."
sleep 60
done
echo "${HOME}/pause absent, continuing workflow."
- name: Check if compilers are configured
timeout-minutes: 120
run: |
echo ::group::inventory
sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true
echo ::endgroup::
echo ::group::get_peadm_config
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
legacy_compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | sed -n 2p)
bundle exec bolt task run peadm::get_peadm_config \
--targets $primary \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
--format json > peadm_config.json
cat peadm_config.json
echo ::endgroup::
echo ::group::smoke_test
configured_legacy_compiler=$(yq '.items[0].value.params.legacy_compilers[0]' peadm_config.json)
configured_compiler=$(yq '.items[0].value.params.compiler_hosts[0]' peadm_config.json)
if [ "$configured_legacy_compiler" != "$legacy_compiler" ] && [ "$configured_compiler" != "$compiler" ]; then
echo "Compilers are not configured, expected $legacy_compiler and $compiler, got $configured_legacy_compiler and $configured_compiler"
exit 1
fi
echo ::endgroup::
- name: Create the upgrade params.json file
run: |
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
legacy_compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | sed -n 2p)
replica=$(yq '.groups[].targets[] | select(.vars.role == "replica") | .name' spec/fixtures/litmus_inventory.yaml)
echo -n '{ "primary_host": "'$primary'", "replica_host": "'$replica'", "compiler_hosts": ["'$compiler'", "'$legacy_compiler'"], "version": "2023.8.0"}' > upgrade_params.json
- name: Upgrade PE with legacy compilers
run: |
echo ::group::upgrade_params.json
cat upgrade_params.json
echo ::endgroup::
echo ::group::upgrade
bundle exec bolt plan run peadm::upgrade \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
--params @upgrade_params.json
echo ::endgroup::
- name: Check if we still have legacy compilers configured
timeout-minutes: 120
run: |
echo ::group::inventory
sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true
echo ::endgroup::
echo ::group::get_peadm_config
primary=$(yq '.groups[].targets[] | select(.vars.role == "primary") | .name' spec/fixtures/litmus_inventory.yaml)
compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | head -n 1)
legacy_compiler=$(yq '.groups[].targets[] | select(.vars.role == "compiler") | .name' spec/fixtures/litmus_inventory.yaml | sed -n 2p)
bundle exec bolt task run peadm::get_peadm_config \
--targets $primary \
--inventoryfile spec/fixtures/litmus_inventory.yaml \
--modulepath spec/fixtures/modules \
--no-host-key-check \
--format json > peadm_config.json
cat peadm_config.json
echo ::endgroup::
echo ::group::smoke_test
configured_legacy_compiler=$(yq '.items[0].value.params.legacy_compilers[0]' peadm_config.json)
configured_compiler=$(yq '.items[0].value.params.compiler_hosts[0]' peadm_config.json)
if [ "$configured_legacy_compiler" != "$legacy_compiler" ] && [ "$configured_compiler" != "$compiler" ]; then
echo "Compilers are not configured, expected $legacy_compiler and $compiler, got $configured_legacy_compiler and $configured_compiler"
exit 1
fi
echo ::endgroup::
- name: Tear down test cluster
if: ${{ always() }}
continue-on-error: true
run: |-
if [ -f spec/fixtures/litmus_inventory.yaml ]; then
echo ::group::tear_down
bundle exec rake 'litmus:tear_down'
echo ::endgroup::
echo ::group::info:request
cat request.json || true; echo
echo ::endgroup::
fi
15 changes: 10 additions & 5 deletions spec/acceptance/peadm_spec/plans/add_inventory_hostnames.pp
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,15 @@
$t = get_targets('*')
wait_until_available($t)

parallelize($t) |$target| {
$fqdn = run_command('hostname -f', $target)
$target.set_var('certname', $fqdn.first['stdout'].chomp)
$command = "yq eval '(.groups[].targets[] | select(.uri == \"${target.uri}\").name) = \"${target.vars['certname']}\"' -i ${inventory_file}"
run_command($command, 'localhost')
$t.map |$target| {
$fqdn = run_command('hostname -f', $target).first
if $fqdn['exit_code'] != 0 {
fail("Failed to get FQDN for target ${target.name}: ${fqdn['stderr']}")
}
$command = "yq eval '(.groups[].targets[] | select(.uri == \"${target.uri}\").name) = \"${fqdn['stdout'].chomp}\"' -i ${inventory_file}"
$result = run_command($command, 'localhost').first
if $result['exit_code'] != 0 {
fail("Failed to update inventory file for target ${uri}: ${result['stderr']}")
}
}
}

0 comments on commit 4596a6c

Please sign in to comment.