Rewrite (#264)

* initial commit * validating * fixing linting * initial pass at tarball * cleanup * adding logic to determine join token and which node is up * fixing some linting * fixing some linting * allowing rpm downgrade and forcing handlers on failure * rpm install logic change * large number of changes 01 * large number of changes 02 * readding throttles * Add first molecule scenario * Move VPC subnet to env var * Convert default scenario to a template and ubuntu-2404 * Remove unused scenario files * updating full rpm logic * fixed rpm logic * linting round 1 * Add supported platforms * fixing VIP join logic * Easy win yamllints * yes/no are not bools * Create galaxy.yml * Update variable in example hosts.yml * Install_rke2_version was renamed to rke2_install_version * set become: false on local file lookup * Docs update (#266) * Add network attached instructions * Update tarball docs * Github does not support titled admons * Clear some lint issues * Clear some galaxy errors * Clear some new lint issues * Fix ansible semver string * Use a supported ansible version * Update file and folder names * Update README instruction for inventory use * Add vale linting, normalized file extensions, and doc tweaks * Revert become change, remove superfluous docs in main readme * adjusting config verbiage --------- Co-authored-by: Adam Leiner <[email protected]> * Ci review (#269) * Add network attached instructions * Update tarball docs * Github does not support titled admons * Clear some lint issues * Clear some galaxy errors * Clear some new lint issues * Fix ansible semver string * Use a supported ansible version * Update file and folder names * Import linting, split linting tasks * Update README instruction o inventory use * Add vale linting, normalized file extensions, and doc tweaks * Tweak workflow lints * Merge workflows back into one file * Test sudo access * Rename Rocky tests, install Terraform * Fix rocky task dep * Install Terraform attempt #2 * Test reusable workflows * Add OS option * Add OS option 2 * Inherit secrets * Install yq * Install yq #2 * Install yq #3 * Install yq #4 * Use YQ for inventory generation * Fix missing close bracket * Print hosts file for debugging * You should cat hosts file for debugging not exec it * Runn test * Fix inventory file location * Fix inventory and add ssh_user * Fix second inventory * Add rocky 9 * Update CI to reference JOBID * Attempt move to AWS commercial * Attempt move to AWS commercial 2 * Attempt move to AWS commercial 3 * Fix destroy command and defaults * Fix AWS Region * Tweak TF and simplify for debuggin * Fix new var * Remove new var * Remove new var 2 * Fix bad EOF * Fix delete job * Reduce EC2 counts * Add Rocky 9 back * Name sec groups * Add Ubuntu * Fix using wrong OS * Add SLES * Fix issue where too in SLES does not have .bashrc * Fix bad permission * Update README, Remove SLES, Test Lint requirements * Remove broken lint check * Updat edocs * increase yaml lint line-length * removing sles tests until they are reimplemented --------- Co-authored-by: Adam Leiner <[email protected]> Co-authored-by: Adam Leiner <[email protected]> * Fix vars * Fix CI reading .ansible directory * #169 (#274) Co-authored-by: Adam Leiner <[email protected]> * Add a non-vague warning banner for v2 release * Update RGS branding --------- Co-authored-by: Adam Leiner <[email protected]> Co-authored-by: Mike DAmato <[email protected]> Co-authored-by: Daemonslayer2048 <[email protected]> Co-authored-by: Jacob Hanafin <[email protected]> Co-authored-by: Jacob Hanafin <[email protected]>
rancherfederal · Jan 15, 2025 · b1ea650 · b1ea650
1 parent 058fd83
commit b1ea650
Show file tree

Hide file tree

Showing 116 changed files with 2,742 additions and 1,927 deletions.
diff --git a/.ansible-lint b/.ansible-lint
@@ -8,4 +8,5 @@ warn_list:
   - var-naming
   - yaml[comments-indentation]
 skip_list:
-  - experimental
+  - experimental
+  - yaml[line-length]
diff --git a/.ansible-lint-ignore b/.ansible-lint-ignore
@@ -1,4 +1,8 @@
 # This file contains ignores rule violations for ansible-lint
 
 roles/testing/tasks/troubleshooting.yml ignore-errors
-inventory/sample/hosts.yml yaml[line-length]
+inventory/sample/hosts.yml yaml[line-length]
+inventory/sample/hosts.yml yaml[comments-indentation]
+roles/rke2/tasks/add-audit-policy-config.yml no-handler
+roles/rke2/tasks/add-pod-security-admission-config.yml no-handler
+roles/rke2/tasks/add-registry-config.yml no-handler
diff --git a/.github/vale/Readability/AutomatedReadability.yml b/.github/vale/Readability/AutomatedReadability.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the Automated Readability Index (%s) below 8."
+link: https://en.wikipedia.org/wiki/Automated_readability_index
+
+formula: |
+  (4.71 * (characters / words)) + (0.5 * (words / sentences)) - 21.43
+
+condition: "> 8"
diff --git a/.github/vale/Readability/ColemanLiau.yml b/.github/vale/Readability/ColemanLiau.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the Coleman–Liau Index grade (%s) below 9."
+link: https://en.wikipedia.org/wiki/Coleman%E2%80%93Liau_index
+
+formula: |
+  (0.0588 * (characters / words) * 100) - (0.296 * (sentences / words) * 100) - 15.8
+
+condition: "> 9"
diff --git a/.github/vale/Readability/FleschKincaid.yml b/.github/vale/Readability/FleschKincaid.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the Flesch–Kincaid grade level (%s) below 8."
+link: https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests
+
+formula: |
+  (0.39 * (words / sentences)) + (11.8 * (syllables / words)) - 15.59
+
+condition: "> 8"
diff --git a/.github/vale/Readability/FleschReadingEase.yml b/.github/vale/Readability/FleschReadingEase.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the Flesch reading ease score (%s) above 70."
+link: https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests
+
+formula: |
+  206.835 - (1.015 * (words / sentences)) - (84.6 * (syllables / words))
+
+condition: "< 70"
diff --git a/.github/vale/Readability/GunningFog.yml b/.github/vale/Readability/GunningFog.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the Gunning-Fog index (%s) below 10."
+link: https://en.wikipedia.org/wiki/Gunning_fog_index
+
+formula: |
+  0.4 * ((words / sentences) + 100 * (complex_words / words))
+
+condition: "> 10"
diff --git a/.github/vale/Readability/LIX.yml b/.github/vale/Readability/LIX.yml
@@ -0,0 +1,17 @@
+extends: metric
+message: "Try to keep the LIX score (%s) below 35."
+
+link: https://en.wikipedia.org/wiki/Lix_(readability_test)
+# Very Easy: 20 - 25
+#
+# Easy: 30 - 35
+#
+# Medium: 40 - 45
+#
+# Difficult: 50 - 55
+#
+# Very Difficult: 60+
+formula: |
+  (words / sentences) + ((long_words * 100) / words)
+
+condition: "> 35"
diff --git a/.github/vale/Readability/SMOG.yml b/.github/vale/Readability/SMOG.yml
@@ -0,0 +1,8 @@
+extends: metric
+message: "Try to keep the SMOG grade (%s) below 10."
+link: https://en.wikipedia.org/wiki/SMOG
+
+formula: |
+  1.0430 * math.sqrt((polysyllabic_words * 30.0) / sentences) + 3.1291
+
+condition: "> 10"
diff --git a/.github/vale/Readability/meta.json b/.github/vale/Readability/meta.json
@@ -0,0 +1,4 @@
+{
+  "feed": "https://github.com/errata-ai/Readability/releases.atom",
+  "vale_version": ">=2.13.0"
+}
diff --git a/.github/vale/config/vocabularies/RGS/accept.txt b/.github/vale/config/vocabularies/RGS/accept.txt
@@ -0,0 +1,18 @@
+# Common/valid Slang
+[C|c]onfig
+airgap
+
+# Acronyms
+STIG
+
+# Tools
+[A|a]nsible
+
+# Kubernetes
+Kubernetes
+[K|k]ubeconfig
+
+# Linux
+[F|f]apolicyd
+containerd
+SELinux
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -1,40 +1,72 @@
 ---
-name: Lint Test
+name: Lint
 
 on:
   push:
 
-env:
-  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
-  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-  AWS_DEFAULT_REGION: us-gov-west-1
-
 jobs:
 
-  lint-test:
-    name: Lint for push
+  ansible-lint:
+    name: Ansible
     runs-on: ubuntu-latest
 
     steps:
-      - name: Check out the codebase.
+      - name: Check out the codebase
         uses: actions/checkout@v3
 
       - name: Set up Python.
         uses: actions/setup-python@v4
         with:
           python-version: '3.x'
 
-      - name: Install test dependencies.
-        run: pip3 install yamllint ansible-lint ansible
+      - name: Install ansible-lint
+        run: pip3 install ansible-lint ansible
 
       - name: Version check
         run: |
           ansible --version
           ansible-lint --version
+
+      - name: Run ansible-lint
+        run: ansible-lint ./roles
+
+  yaml-lint:
+    name: YAML
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the codebase
+        uses: actions/checkout@v3
+
+      - name: Set up Python.
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Install yamllint
+        run: pip3 install yamllint
+
+      - name: Version check
+        run: |
           yamllint --version
 
-      - name: Run yamllint.
+      - name: Run yamllint
         run: yamllint .
 
-      - name: Run ansible-lint.
-        run: ansible-lint
+  vale-lint:
+    name: Vale
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the codebase
+        uses: actions/checkout@v3
+
+      - name: Install vale
+        run: sudo snap install vale
+
+      - name: Version check
+        run: |
+          vale --version
+
+      - name: Run yamllint
+        run: vale --glob='*.md' ./docs ./README.md
diff --git a/.github/workflows/reusable_integration_testing.yml b/.github/workflows/reusable_integration_testing.yml
@@ -0,0 +1,172 @@
+---
+name: Reusable Cluster Testing
+
+on:
+  workflow_call:
+    inputs:
+      os:
+        required: true
+        type: string
+      ssh_user:
+        required: true
+        type: string
+
+jobs:
+
+  cluster-test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out the codebase.
+        uses: actions/checkout@v2
+
+      - name: Add terraform repository
+        run: |
+          wget -O - https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
+          echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
+
+      - name: Set up Python
+        id: setup_python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.9
+
+      - name: Install dependencies
+        run: |
+          sudo apt update -y
+          sudo apt install terraform -y
+          sudo snap install yq
+          python -m pip install --upgrade pip
+          pip3 install ansible pytest-testinfra
+
+      - name: Ansible version check
+        run: ansible --version
+
+      - name: Terraform version check
+        run: terraform --version
+
+      - name: yq version check
+        run: yq --version
+
+      - name: Terraform Init
+        id: init
+        run: |
+          cd testing/
+          terraform init
+
+      - run: 'echo "$SSH_KEY" > .key'
+        shell: bash
+        env:
+          SSH_KEY: ${{secrets.SSH_PRIVATE_KEY}}
+
+      - run: chmod 400 .key
+
+      - name: Terraform private key
+        run: cp .key testing/.key
+
+      - name: Terraform Validate
+        id: validate
+        run: |
+          cd testing/
+          terraform validate -no-color
+
+      - name: Terraform Plan
+        id: plan
+        run: |
+          cd testing/
+          terraform plan -var "GITHUB_RUN_ID=$GITHUB_RUN_ID" -var "os=${{ inputs.os }}" -no-color
+        continue-on-error: true
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-2
+
+      - name: Terraform Plan Status
+        if: steps.plan.outcome == 'failure'
+        run: exit 1
+
+      - name: Terraform Apply
+        run: |
+          cd testing/
+          terraform apply -var "GITHUB_RUN_ID=$GITHUB_RUN_ID" -var "os=${{ inputs.os }}" -auto-approve
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-2
+
+      - name: Create inventory
+        run: |
+          mkdir ./${{ inputs.os }}
+          export RKE2_SERVER=$(aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" "Name=tag:Owner,Values=rke2-ansible-github-actions" "Name=tag:NodeType,Values=Server" "Name=tag:github_run,Values=$GITHUB_RUN_ID" "Name=tag:os_test,Values=${{ inputs.os }}" --query "Reservations[*].Instances[*].PublicIpAddress" --output text | head -1)
+          export RKE2_AGENT=$(aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" "Name=tag:Owner,Values=rke2-ansible-github-actions" "Name=tag:NodeType,Values=Agent" "Name=tag:github_run,Values=$GITHUB_RUN_ID" "Name=tag:os_test,Values=${{ inputs.os }}" --query "Reservations[*].Instances[*].PublicIpAddress" --output text | head -1)
+          yq --null-input '.rke2_cluster.children.rke2_servers.hosts.master-01.ansible_host = strenv(RKE2_SERVER)' > ./${{ inputs.os }}/hosts.yml
+          yq -i '.all.vars.rke2_kubernetes_api_server_host = strenv(RKE2_SERVER)' ./${{ inputs.os }}/hosts.yml
+          yq -i '.rke2_cluster.children.rke2_agents.hosts.worker-01.ansible_host = strenv(RKE2_AGENT)' ./${{ inputs.os }}/hosts.yml
+          echo "remote_user=${{ inputs.ssh_user }}" >> ansible.cfg
+          echo "private_key_file=.key" >> ansible.cfg
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-2
+
+      - name: Check hosts.yml and ansible.cfg
+        run: |
+          cat ./${{ inputs.os }}/hosts.yml
+          cat ansible.cfg
+
+      - name: Run playbook
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} -vv --private-key .key site.yml
+
+      - name: Run playbook again for idempotency
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} -vv --private-key .key site.yml
+
+      - name: Run Ansible Tests
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} --verbose --skip-tags "troubleshooting" --private-key .key testing.yml
+
+      - name: Run Python Tests
+        run: |
+          export DEFAULT_PRIVATE_KEY_FILE=.key
+          pytest --hosts=rke2_servers --ansible-inventory=./${{ inputs.os }}/hosts.yml --force-ansible --connection=ansible --sudo testing/basic_server_tests.py
+          pytest --hosts=rke2_agents --ansible-inventory=./${{ inputs.os }}/hosts.yml --force-ansible --connection=ansible --sudo testing/basic_agent_tests.py
+
+      - name: Update inventory hosts.yml with added host
+        run: |
+          export RKE2_AGENT2=$(aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" "Name=tag:Owner,Values=rke2-ansible-github-actions" "Name=tag:NodeType,Values=ExtraNode" "Name=tag:github_run,Values=$GITHUB_RUN_ID" "Name=tag:os_test,Values=${{ inputs.os }}" --query "Reservations[*].Instances[*].PublicIpAddress" --output text | head -1)
+          yq -i '.rke2_cluster.children.rke2_agents.hosts.worker-02.ansible_host = strenv(RKE2_AGENT2)' ./${{ inputs.os }}/hosts.yml
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-2
+
+      - name: Run playbook again with added host
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} --verbose --private-key .key site.yml
+
+      - name: Run Ansible Tests with added host
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} --verbose --skip-tags "troubleshooting" --private-key .key testing.yml
+
+      - name: Run Python Tests with added host
+        run: |
+          export DEFAULT_PRIVATE_KEY_FILE=.key
+          pytest --hosts=rke2_servers --ansible-inventory=./${{ inputs.os }}/hosts.yml --force-ansible --connection=ansible --sudo testing/basic_server_tests.py
+          pytest --hosts=rke2_agents --ansible-inventory=./${{ inputs.os }}/hosts.yml --force-ansible --connection=ansible --sudo testing/basic_agent_tests.py
+
+      - name: Run troubleshoot tasks
+        if: ${{ failure() }}
+        run: |
+          ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i ./${{ inputs.os }}/hosts.yml -u ${{ inputs.ssh_user }} -vvv --tags "troubleshooting" --private-key .key testing.yml
+
+      - name: Delete Stack
+        if: ${{ always() }}
+        run: |
+          cd testing/
+          terraform destroy -var "GITHUB_RUN_ID=$GITHUB_RUN_ID" -var "os=${{ inputs.os }}" -auto-approve
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_DEFAULT_REGION: us-east-2
+