Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Test rework permanent next-target #3517

Closed
wants to merge 1 commit into from

Conversation

apostasie
Copy link
Contributor

@apostasie apostasie commented Oct 8, 2024

Note:
This PR is a dump of all next- things for test rework.
It will never be merged, and will be forked-out into smaller PR.

Github actions changes:

  • cosmetic efforts on test display titles (more compact, more consistent)
  • lint on all platforms instead of just linux
  • removal of all nick/retries
  • separation of flaky / non-flaky in different steps - non-flaky now run without any retry - flaky do run with gotestsum retry, but no longer nick/retries

Integration test entrypoint:

  • now lives inside ./hack/test-integration.sh
  • additionally displays slow tests at the end of the run

Dockerfile changes:

  • removed CMD - replaced in favor of a direct call to ./hack/test-integration.sh
  • removed useless stages

Currently, test suites have been ran locally (no-flaky, on the host directly) 100 times without failure:

  • ./cmd/nerdctl < 1 sec per run
  • ./cmd/nerdctl/completion < 1 sec per run
  • ./cmd/nerdctl/issues < 10 secs per run
  • ./cmd/nerdctl/system < 10 secs per run
  • ./cmd/nerdctl/volume < 10 secs per run
  • ./cmd/nerdctl/network < 10 secs per run
  • ./cmd/nerdctl/image < 30 secs per run
  • ./cmd/nerdctl/ipfs

WORK IN PROGRESS rn:

  • ./cmd/nerdctl/builder
  • ./cmd/nerdctl/container

NOT part of this PR:

  • ./cmd/nerdctl/login
  • ./cmd/nerdctl/compose

@apostasie apostasie force-pushed the finally-disable-retries branch 7 times, most recently from 20b08e0 to 521d09a Compare October 8, 2024 22:29
runs-on: ubuntu-24.04
timeout-minutes: 20
lint-go:
timeout-minutes: "${{ fromJSON(env.SHORT_TIMEOUT) }}"
Copy link
Contributor Author

@apostasie apostasie Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoever came-up with this notion that it would be cool to have the world CIs built on top of yaml + shell + a layer of crap deserves to be repeatedly whacked with a wet trout.

@apostasie apostasie force-pushed the finally-disable-retries branch 18 times, most recently from cd2c625 to b6022fc Compare October 9, 2024 23:30
@apostasie apostasie changed the title [WIP] Test rework, part 5: separate flakies Test rework, part 5: separate flakies Oct 10, 2024
@apostasie apostasie marked this pull request as ready for review October 10, 2024 01:17
@apostasie
Copy link
Contributor Author

@AkihiroSuda technically ready for review, although we should first merge the "global lock for CNI", along with "part 4" - then I will rebase and break it in cleaner commits.

@apostasie apostasie force-pushed the finally-disable-retries branch 3 times, most recently from ab11cef to 9be06cf Compare October 11, 2024 21:35
@apostasie apostasie marked this pull request as draft October 11, 2024 21:35
@apostasie apostasie force-pushed the finally-disable-retries branch 3 times, most recently from 1c16e3b to 161dc04 Compare October 11, 2024 22:07
@apostasie apostasie changed the title Test rework, part 5: separate flakies [WIP] Test rework, part 5: separate flakies Oct 11, 2024
@apostasie apostasie force-pushed the finally-disable-retries branch 9 times, most recently from ece4022 to 1fd7aae Compare October 14, 2024 00:14
@apostasie apostasie changed the title [WIP] Test rework, part 5: separate flakies [WIP] Test rework permanent next-target Oct 14, 2024
@apostasie
Copy link
Contributor Author

Has been fully subsumed into breakout PRs.

@apostasie apostasie closed this Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant