chore(tasks/transform-conformance): support override to replace takeover mode

[Dunqing]

This PR does the following things.

1. Move the override output of the snapshots folder to the overrides folder.
2. Support override mode to replace takeover mode
3. The update_fixtures.js no longer uses overrides's output.js to replace Babel's output.js.

How does override mode work?

When running each test, it checks whether an output file for that test exists in the ⁠overrides directory. If it does, the output file will be used to compare with the transformed code.

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “0-merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[Dunqing]

• chore(tasks/transform-conformance): update only when the output differs from the original output #7779
• chore(tasks/transform-conformance): support --override flag #7774
• chore(tasks/transform-conformance): support override to replace takeover mode #7771 👈 (View in Graphite)
• test(transformer): conformance runner use HelperLoaderMode::External in takeover mode #7807
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #7771 will not alter performance

_{Comparing 12-10-chore_tasks_transform-conformance_support_override_to_replace_takeover_mode (b089e8b) with main (450bb33)}

Summary

✅ 29 untouched benchmarks

[overlookmotel]

I see one problem. There are 3 overrides for class properties transform where the options.json file is overridden too. To replace update_fixture.js, the new mechanism would need to look for options.json files in the overrides folder too, and use that as the source of truth if there is one (note: required for exec tests too).

This explains the difference in snapshots for class properties transform in this PR.

2 of the overrides for class properties transform have no output.js files in them, but I've changed that in #7777. If you want to rebase on top of that, it'll solve that particular problem.

What explains the difference in snapshots for the other transforms, I cannot say. Looks like something's not working!

[overlookmotel]

PS I can clean up update_fixtures.js to remove dead code after this PR is merged. Probably faster for me to do it, as I'm familiar with it.

[Dunqing]

What explains the difference in snapshots for the other transforms, I cannot say. Looks like something's not working!

Looks like the takeover mode hasn't stored any information in snapshots, we even don't know if the semantic check is passed or not, now we does.

[overlookmotel]

An unwelcome discovery! 😭

But yes better to know than not to know.

But every single test for object-rest-spread is now "Output mismatch"! I guess that's where --override comes in. Going through them all and checking our output is actually correct is going to be fun...

[Dunqing]

But every single test for object-rest-spread is now "Output mismatch"! I guess that's where --override comes in. Going through them all and checking our output is actually correct is going to be fun...

Fixed! By using --override 😂.

[Dunqing]

The private-loose/foobar/input.js test still mismatch, can you take a look more?

[overlookmotel]

The private-loose/foobar/input.js test still mismatch, can you take a look more?

Looks like the problem is that it's not looking for options.json in overrides folder and using that over the original Babel one if it's there. i.e. Currently you can override output.js, but not options.json.

Sorry if I was unclear above. #7777 solved the problem of when overrides folder contains only an options.json file and no output.js file (solved by just adding output.js files to those override folders). But still options.json in overrides needs to be obeyed!

[Dunqing]

The private-loose/foobar/input.js test still mismatch, can you take a look more?

Looks like the problem is that it's not looking for options.json in overrides folder and using that over the original Babel one if it's there. i.e. Currently you can override output.js, but not options.json.

Sorry if I was unclear above. #7777 solved the problem of when overrides folder contains only an options.json file and no output.js file (solved by just adding output.js files to those override folders). But still options.json in overrides needs to be obeyed!

Solved! Thank you for finding the root cause!

[graphite-app]

Merge activity

• Dec 11, 11:07 AM EST: @graphite-app we removed the merge queue label because we could not find a Graphite account associated with your GitHub profile.

You must have a Graphite account in order to use the merge queue. Create an account and try again using this link

• Dec 11, 11:18 AM EST: A user added this pull request to the Graphite merge queue.
• Dec 11, 11:22 AM EST: A user merged this pull request with the Graphite merge queue.

[overlookmotel]

I rebased this on top of #7807, so it's easier to see that all the old takeover snapshots have just moved to overrides, and this change hasn't actually altered the output.js files.