Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove extraneous detail from snapshots for transformer "exec" tests #7704

Closed
overlookmotel opened this issue Dec 6, 2024 · 0 comments · Fixed by #7715
Closed

Remove extraneous detail from snapshots for transformer "exec" tests #7704

overlookmotel opened this issue Dec 6, 2024 · 0 comments · Fixed by #7715
Labels
A-transformer Area - Transformer / Transpiler

Comments

@overlookmotel
Copy link
Contributor

Snapshots for transformer "exec" tests contain extraneous details which make snapshot changes hard to read, and causes erroneous conformance task failures.

Irrelevant details cause CI failures

For example, #7687 adds a new exec test. The problem is that the snapshot contains irrelevant details, which cause changes in the snapshot on every run, and so an erroneous failure:

- RUN  v2.1.2 /Users/overlookmotel/Programming/crates/oxc/tasks/transform_conformance
+ RUN  v2.1.2 /home/runner/work/oxc/oxc/tasks/transform_conformance
 
- ✓ fixtures/oxc/babel-plugin-transform-class-properties-test-fixtures-this-in-computed-key-exec.test.js  (1 test) 2ms
+ ✓ fixtures/oxc/babel-plugin-transform-class-properties-test-fixtures-this-in-computed-key-exec.test.js  (1 test) 5ms
 
-   Start at  10:54:43
-   Duration  160ms (transform 12ms, setup 0ms, collect 6ms, tests 2ms, environment 0ms, prepare 38ms)
+   Start at  10:59:00
+   Duration  242ms (transform 19ms, setup 0ms, collect 12ms, tests 5ms, environment 0ms, prepare 61ms)

https://github.com/oxc-project/oxc/actions/runs/12197559497/job/34027460327?pr=7687

I'm not sure if we can get vitest to not output these irrelevant details, or whether we'd need to post-process vitest's output to remove them before outputting snapshot.

Test numbering creates large diffs

Snapshots include a number before each test failure e.g.:

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/28]⎯

This creates a lot of churn in the diff when you fix a test, which makes it harder to see the relevant changes. Because you get a ton of irrelevant changes like:

- ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/28]⎯
+ ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/27]⎯

- ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/28]⎯
+ ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/27]⎯

- ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/28]⎯
+ ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/27]⎯

This is not so important, it's just a bit messy.

@Boshen I'm not familiar with vitest, so not sure how best to tackle these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-transformer Area - Transformer / Transpiler
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants