Frontend charm integration tests fix with a self hosted runner #54

mz2 · 2023-10-26T16:11:08Z

Integration tests for the frontend charm now actually work also on GitHub!

The workflow looks to be stable, with a dozen passes witnessed today.

It does however take ~10mins to run and I figured we would therefore not start running it on every commit to every branch (6+ mins of the time is taken by the microk8s setup -- we can drop this in a big way with custom images used for the CI runs when that becomes available). It runs on:

all branches once a PR review has been posted for the branch for which there is a PR open
all branches with a PR which touch resources under ./frontend/charm (the integration tests can capture failures also in resources outside that directory, hence the rule above also).
on the main branch
on version tagged commits
manually via workflow dispatch

The workflow was refactored to reuse two now introduced reusable composite actions in https://github.com/canonical/certification-github-workflows (the first PR to that repo should be reviewed and merged first before this goes in).

NOTE! When squashing this into main, please edit the commit message heavily down in volume since this contains a few months worth of desperate debugging attempts of these integration tests that continued to work locally just fine but failed to be executed specifically on GitHub up until now. I have simplified the version history with some squashing but since there are several merge commits there I didn't want to do anything more drastic than that.

…ning.

…n blocked status (more scenarios needed to actually relate the frontend to the backend).

…ger the integration tests with.

… action is required.

…state where relation is not yet connected. Simplified the charm logic, no _stored state needed (fetching the hostname and port from the relation's application data bag).

Let's try with the microk8s version and libpyjuju < 3.1 that seems to work for rout53-acme-operator where the integration test pipeline is pulled from

…rver into integration-test-fix

.github/workflows/publish_charms.yml

…:canonical/test_observer into integration-test-fix-with-hosted-runners

…f these changes

mz2 · 2023-12-09T22:22:54Z

frontend/charm/requirements.txt

@@ -1 +1,2 @@
-ops >= 2.2.0
+ops >= 2.8.0
+macaroonbakery==1.3.2


Added as a workaround to https://github.com/canonical/test_observer/actions/runs/7142353669/job/19451476954 as instructed at https://chat.canonical.com/canonical/pl/9xzxm61rgtry5d7sfx9xz7ryyc

mz2 · 2023-12-09T22:27:20Z

frontend/charm/tox.ini

@@ -66,7 +66,7 @@ commands =
 description = Run integration tests
 deps =
    pytest
-    juju
+    juju<=3.2


This is intentionally <= 3.2 as opposed to for example >= 2.9 as is set in the charm metadata: the integration test environment is set to be 3.x (2.9 was not really behaving in the github runner context, which is possible to go investigate but probably not worth our while unless we get stuck in production 2.9 for a long time unexpectedly).

mz2 · 2023-12-09T22:29:35Z

.github/workflows/frontend_charm_integration_tests.yml

-          provider: microk8s
+        uses: actions/checkout@v4
+      - name: Set up microk8s
+        uses: canonical/certification-github-workflows/.github/actions/microk8s-setup@microk8s-setup-action


The setup steps turned out to be more extensive on self hosted runners, and since this is all repetitive boilerplate that becomes relevant again when we want to integration test any other charmed part of our stack, I moved the whole lot to a reusable composite action. Same story below re: the archive-charm-testing-artifacts

…R review has been submitted

…es to resources under frontend/charm

mz2 · 2023-12-09T22:55:34Z

terraform/test-observer.tf

@@ -1,7 +1,7 @@
 terraform {
  required_providers {
    juju = {
-      version = "~> 0.7.0"


The changes below showed up as necessary when the provider was updated: the series was deprecated (and base which replaces is similarly nonsensical for k8s charms) => series removed below.

This change is not strictly speaking necessary for the charm integration tests (though it was my intention to also test out in succession also that the terraform plan continues to be able to spin a new environment up that stabilises in an all-ready state).

can we do this on PS5 though? as in can we use tf provider for juju version 0.10.0?

I think this should indeed Just Work, in that the next time you deploy using this plan, it will upgrade the provider to the latest version that is described here. Deploying to staging will confirm that safely, I'd say.

omar-selo

It's amazing that you got it to work! finally this has been annoying for a long time

.github/workflows/publish_api.yml

.github/workflows/publish_frontend.yml

frontend/charm/requirements.txt

omar-selo · 2023-12-11T07:55:21Z

terraform/test-observer.tf

@@ -1,7 +1,7 @@
 terraform {
  required_providers {
    juju = {
-      version = "~> 0.7.0"


can we do this on PS5 though? as in can we use tf provider for juju version 0.10.0?

.github/workflows/frontend_charm_integration_tests.yml

mz2 · 2023-12-11T14:46:29Z

Superseded by #80

mz2 and others added 30 commits June 7, 2023 11:49

Define the dependencies more exactly to get the integration tests run…

0d44757

…ning.

juju 3.1/stable -> 3.0/stable

97e8d4c

Install strictly confined versions of juju and microk8s.

a055701

Use microk8s 1.27-strict/stable

672ea62

Adds package repo access to the integration test job

6c11eb5

Add the containerd template modification

5c149e2

Adds some missing sudoing

11cf30b

From >> to tee -a

6e8375a

Drop the docker login

97f1531

Oops, escaping quotes did not work quite right there.

1478f45

Adjust the integration test scenario, expecting rather than raising o…

b03e015

…n blocked status (more scenarios needed to actually relate the frontend to the backend).

Drops the debugging intended branch from the list of branches to trig…

55b9436

…ger the integration tests with.

Change to maintenance status instead for clarity, since configuration…

cde6b80

… action is required.

Adds the "integration-test-fix" branch into the rule still

1ed9d5f

Adds a 503 response / maintenance status to the frontend application …

0a30cee

…state where relation is not yet connected. Simplified the charm logic, no _stored state needed (fetching the hostname and port from the relation's application data bag).

Removs the redundant import StoredState

bf2859c

Add a microk8s status --wait-ready

2fc065f

Adds missing sudo

06a258e

Merge branch 'main' into integration-test-fix

4af3b2a

Merge branch 'main' into integration-test-fix

0db5736

Merge branch 'main' into integration-test-fix

8417af7

Let's try with the microk8s version + libpyjuju < 3.1

a3a5404

Let's try with the microk8s version and libpyjuju < 3.1 that seems to work for rout53-acme-operator where the integration test pipeline is pulled from

Merge branch 'integration-test-fix' of github.com:canonical/test_obse…

e90849c

…rver into integration-test-fix

Back to 5.13/stable for lxd

234b1aa

2.9/stable -> 3.0/stable

fd3ed35

Trying with the non-strictly confined microk8s

63f4d6f

Adds "publish-charm" job

acf47ce

Adds promotion action

b03dddb

Swap to self-hosted runners

7348559

LXD 5.13 => 5.19

9521314

mz2 force-pushed the integration-test-fix-with-hosted-runners branch from c115fae to fb4817a Compare December 9, 2023 21:45

mz2 changed the title ~~Integration test fix with hosted runners~~ Frontend charm integration tests fix with a self hosted runner Dec 9, 2023

mz2 commented Dec 9, 2023

View reviewed changes

.github/workflows/publish_charms.yml Outdated Show resolved Hide resolved

mz2 added 7 commits December 10, 2023 00:16

Update .github/workflows/publish_charms.yml

5a75762

Merges in origin/main

4f5cc55

Merge branch 'integration-test-fix-with-hosted-runners' of github.com…

188f2ba

…:canonical/test_observer into integration-test-fix-with-hosted-runners

Remove publish_charms.yml that has been long since been separately added

418cdbd

Removes release-charms.yml that has long been superseded

ff57b22

Removes also frontend_charm_promote.yml that is not really in scope o…

61e6d4f

…f these changes

From 3.2 requirement => 2.9 requirement

9e0ff27

mz2 commented Dec 9, 2023

View reviewed changes

Removes the stray reformatting of frontend charm.py

943e0e6

mz2 commented Dec 9, 2023

View reviewed changes

mz2 mentioned this pull request Dec 9, 2023

Frontend charm integration test run bugfix #20

Closed

mz2 added 4 commits December 10, 2023 00:35

Remove another example of accidental reformatting

acb0dd8

Run the integration test workflow manually, on main branch and when P…

f9528e3

…R review has been submitted

Run frontend integration tests also on branches where there are chang…

3ded998

…es to resources under frontend/charm

Removes a stray change to frontend_charm_analysis.yml

5bcbd1c

mz2 commented Dec 9, 2023

View reviewed changes

mz2 requested a review from omar-selo December 9, 2023 22:57

omar-selo reviewed Dec 11, 2023

View reviewed changes

.github/workflows/frontend_charm_integration_tests.yml Outdated Show resolved Hide resolved

mz2 added 4 commits December 11, 2023 13:53

An explanatory comment re: pinning macaroonbakery to 1.3.2

ad2ae8b

Adds docker.io mirroring

09e1425

Fix the trigger so the workflow triggers on approved PRs

e465185

Reference the setup steps from main

4814f96

mz2 mentioned this pull request Dec 11, 2023

Frontend charm integration tests fix with a self hosted runner #80

Merged

mz2 closed this Dec 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Frontend charm integration tests fix with a self hosted runner #54

Frontend charm integration tests fix with a self hosted runner #54

mz2 commented Oct 26, 2023 •

edited

Loading

mz2 Dec 9, 2023

mz2 Dec 9, 2023 •

edited

Loading

mz2 Dec 9, 2023 •

edited

Loading

mz2 Dec 9, 2023 •

edited

Loading

mz2 Dec 9, 2023

omar-selo Dec 11, 2023

mz2 Dec 11, 2023

omar-selo left a comment

omar-selo Dec 11, 2023

mz2 commented Dec 11, 2023

Frontend charm integration tests fix with a self hosted runner #54

Frontend charm integration tests fix with a self hosted runner #54

Conversation

mz2 commented Oct 26, 2023 • edited Loading

mz2 Dec 9, 2023

Choose a reason for hiding this comment

mz2 Dec 9, 2023 • edited Loading

Choose a reason for hiding this comment

mz2 Dec 9, 2023 • edited Loading

Choose a reason for hiding this comment

mz2 Dec 9, 2023 • edited Loading

Choose a reason for hiding this comment

mz2 Dec 9, 2023

Choose a reason for hiding this comment

omar-selo Dec 11, 2023

Choose a reason for hiding this comment

mz2 Dec 11, 2023

Choose a reason for hiding this comment

omar-selo left a comment

Choose a reason for hiding this comment

omar-selo Dec 11, 2023

Choose a reason for hiding this comment

mz2 commented Dec 11, 2023

mz2 commented Oct 26, 2023 •

edited

Loading

mz2 Dec 9, 2023 •

edited

Loading

mz2 Dec 9, 2023 •

edited

Loading

mz2 Dec 9, 2023 •

edited

Loading