Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: run e2e tests with testcontainers #19003

Draft
wants to merge 33 commits into
base: master
Choose a base branch
from

Conversation

blakepettersson
Copy link
Member

@blakepettersson blakepettersson commented Jul 9, 2024

This change allows for the use of testcontainers to run E2E tests. Why would we want this? Mainly for three reasons:

  1. IDE friendlier
  2. Debugger friendlier
  3. Much easier to setup

It's a "one click" way (or a simple go test e2e way) to run E2E tests. To run using testcontainers, run go test e2e or make test-e2e-local - there is no need to do anything else.

This has a bunch of changes, but the majority of them is simply extracting the command flags from the affected controller commands to separate builders, which are then used for instantiating each controller programatically. The commands make use of these builders to set the flags as they previously did, so from a user perspective nothing changes.

Redis, K3S and the E2E server are run as testcontainers when running the E2E tests. The controllers are run in-process but spun off into separate goroutines.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

Copy link

bunnyshell bot commented Jul 9, 2024

🔴 Preview Environment stopped on Bunnyshell

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

  • 🔵 /bns:start to start the environment
  • 🚀 /bns:deploy to redeploy the environment
  • /bns:delete to remove the environment

Copy link

bunnyshell bot commented Jul 9, 2024

✅ Preview Environment created on Bunnyshell but will not be auto-deployed

See: Environment Details

Available commands (reply to this comment):

  • 🚀 /bns:deploy to deploy the environment

@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch 3 times, most recently from cba2c14 to 6654416 Compare July 9, 2024 22:28
Copy link

codecov bot commented Jul 9, 2024

Codecov Report

Attention: Patch coverage is 23.83107% with 505 lines in your changes missing coverage. Please review.

Project coverage is 50.64%. Comparing base (a071f93) to head (6654416).
Report is 10 commits behind head on master.

Files Patch % Lines
cmd/argocd-server/commands/argocd_server.go 0.00% 180 Missing ⚠️
...ntroller/commands/argocd_application_controller.go 0.00% 139 Missing ⚠️
.../argocd-repo-server/commands/argocd_repo_server.go 0.00% 130 Missing ⚠️
...t-controller/commands/applicationset_controller.go 74.83% 28 Missing and 10 partials ⚠️
server/server.go 44.44% 5 Missing ⚠️
cmd/argocd/commands/headless/headless.go 0.00% 4 Missing ⚠️
util/tls/tls.go 0.00% 4 Missing ⚠️
cmd/argocd/commands/admin/cluster.go 0.00% 2 Missing ⚠️
reposerver/gpgwatcher.go 0.00% 2 Missing ⚠️
util/cache/cache.go 96.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19003      +/-   ##
==========================================
- Coverage   50.65%   50.64%   -0.01%     
==========================================
  Files         315      315              
  Lines       43286    43260      -26     
==========================================
- Hits        21926    21911      -15     
+ Misses      18861    18846      -15     
- Partials     2499     2503       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch 7 times, most recently from 42d5c4c to a50a826 Compare July 17, 2024 09:21
- name: Run E2E testsuite
run: |
set -x
make test-e2e-local
goreman run stop-all || echo "goreman trouble"
Copy link
Member

@rumstead rumstead Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added to make sure the argocd processes would shut down gracefully and have the golang coverage from the e2e "flushed" to the files. We should just confirm that test-containers send the proper signals to cause the flush to happen.

(Just commenting about the contributors meeting today as a reminder :))

@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch from ceb6b4e to 83a93a5 Compare August 7, 2024 21:12
@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch 2 times, most recently from 6b1e642 to 8afb28d Compare August 27, 2024 08:51
@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch from 9cd89f9 to fb6f94e Compare September 24, 2024 19:34
This change allows for the use of testcontainers to run E2E tests. Why
would we want this? Mainly for three reasons:

1. IDE friendlier
2. Debugger friendlier
3. Much easier to setup

It's a "one click" way (or a simple `go test e2e` way) to run E2E tests,
set the environment variable `ARGOCD_E2E_USE_TESTCONTAINERS=true` and
you're off to the races.

This has a bunch of changes, but the majority of them is simply creating
builders for each of the affected controller commands to separate builders,
which are used to be able instantiate each controller programatically.
The commands make use of these builders to set the flags as they
previously did.

Redis, K3S and the E2E server are run as testcontainers when running the
E2E tests. The controllers are run in-process but spun off into separate
goroutines.

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
This works just as nicely and reduces the scope of the changes.

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
This is required since we're running the controllers outside a container.

Signed-off-by: Blake Pettersson <[email protected]>
This is to prevent the "dubious ownership" errors in Git.

Signed-off-by: Blake Pettersson <[email protected]>
Since there is no need to set a specific serviceaccount right now for a
given controller, revert these changes.

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
The repo-server and api server both needs to be able to have their
ports configured.

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Hardcode helm index port for now, but randomize api server and repo server ports.

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
The tests are still failing due to the Helm index still referring to the 9080 port for some reason...

Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch from c4f34fe to 0707781 Compare November 2, 2024 23:45
Signed-off-by: Blake Pettersson <[email protected]>
Signed-off-by: Blake Pettersson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants