Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(amazonq): reset state after each testgen e2e test #6522

Merged
merged 14 commits into from
Feb 17, 2025

Conversation

chungjac
Copy link
Contributor

@chungjac chungjac commented Feb 6, 2025

Problem

  1. The /test command can occasionally create test files in directories unrelated to the source file location (this is a known science issue that will be addressed)
  2. Unit tests generated after the "Clicks on accept" test may possibly accumulate in the CI since testFixtures isn't reset for each build. This can lead to bloated test environments as these tests should be isolated and start with a clean state.

Solution

  1. Add a dedicated testgen folder within testFixtures to isolate from other files. This prevents cross-contamination (ex: python tests appearing in java directories)
  2. Clean up generated unit tests after "Clicks on accept" test

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

test run
@chungjac
Copy link
Contributor Author

chungjac commented Feb 6, 2025

/runIntegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 7, 2025

/runintegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 7, 2025

/runIntegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 10, 2025

/runintegrationTests

1 similar comment
@chungjac
Copy link
Contributor Author

chungjac commented Feb 11, 2025

/runintegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 12, 2025

/runIntegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 14, 2025

/runIntegrationTests

@chungjac
Copy link
Contributor Author

chungjac commented Feb 15, 2025

/runIntegrationTests

@chungjac chungjac changed the title test(amazonq): flaky valid language tests test(amazonq): reset state after each testgen e2e test Feb 17, 2025
@chungjac chungjac marked this pull request as ready for review February 17, 2025 18:17
@chungjac chungjac requested a review from a team as a code owner February 17, 2025 18:17
@nkomonen-amazon
Copy link
Contributor

/runIntegrationTests

Copy link
Contributor

@nkomonen-amazon nkomonen-amazon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for handling! I just triggered the tests for a final time

@nkomonen-amazon nkomonen-amazon merged commit d9b9861 into aws:master Feb 17, 2025
30 of 32 checks passed
s7ab059789 pushed a commit to s7ab059789/aws-toolkit-vscode that referenced this pull request Feb 19, 2025
## Problem
1. The /test command can occasionally create test files in directories
unrelated to the source file location (this is a known science issue
that will be addressed)
2. Unit tests generated after the "Clicks on accept" test may possibly
accumulate in the CI since testFixtures isn't reset for each build. This
can lead to bloated test environments as these tests should be isolated
and start with a clean state.

## Solution
1. Add a dedicated testgen folder within testFixtures to isolate from
other files. This prevents cross-contamination (ex: python tests
appearing in java directories)
2. Clean up generated unit tests after "Clicks on accept" test
---

- Treat all work as PUBLIC. Private `feature/x` branches will not be
squash-merged at release time.
- Your code changes must meet the guidelines in
[CONTRIBUTING.md](https://github.com/aws/aws-toolkit-vscode/blob/master/CONTRIBUTING.md#guidelines).
- License: I confirm that my contribution is made under the terms of the
Apache 2.0 license.
@chungjac chungjac deleted the flaky-test branch March 13, 2025 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants