[webview_flutter_web] Migrate integration tests to package:web.

[ditman]

This PR:

• Fixes a DOM timing issue that was preventing the Legacy Widget from building.
• Makes the integration_tests for the package less flaky, now that they seem to be timing out unexpectedly in CI.

Issues

• Fixes [webview_flutter_web] Legacy mode accessing DOM before it's ready. flutter#151693
• Should prevent this flake from happening: bad / good (Linux_web web_platform_tests_shard_3 stable)

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene page, which explains my responsibilities.
• I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• I signed the CLA.
• The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• I linked to at least one issue that this PR fixes in the description above.
• I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
• I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style, or this PR is exempt from CHANGELOG changes.
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making, or this PR is test-exempt.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[ditman]

Now the non-legacy test should actually do something in CI. The legacy one is only left as a manual check. No coverage is lost, since neither of the tests were actually verifying anything earlier (the bug that is being fixed here should have been caught by a functional legacy test!)

This should also ease the Linux_web web_platform_tests_shard_3 flakes. In my machine this PR seems to not flake, unlike what's currently in the repo!

[ditman]

Nah, this is still flaking :/

• https://ci.chromium.org/ui/p/flutter/builders/try/Linux_web%20web_platform_tests_shard_1%20master/9401/overview

[ditman]

$ flutter --version
Flutter 3.22.2 • channel stable • [email protected]:flutter/flutter.git
Framework • revision 761747bfc5 (5 weeks ago) • 2024-06-05 22:15:13 +0200
Engine • revision edd8546116
Tools • Dart 3.4.3 • DevTools 2.34.3
$ flutter clean; run_integration_test.sh | ts -s
00:00:03 
00:00:03 ============================================================
00:00:03 || Running for packages/webview_flutter/webview_flutter_web
00:00:03 ============================================================
00:00:03 
00:00:03 Starting chromedriver on port 4444
00:00:03 Running command: "flutter drive -d web-server --web-port=7357 --browser-name=chrome --web-renderer=canvaskit --driver test_driver/integration_test.dart --target integration_test/webview_flutter_test.dart" in /work/flutter/packages/packages/webview_flutter/webview_flutter_web/example
00:00:03 Resolving dependencies...
00:00:04 Downloading packages...
00:00:04   collection 1.18.0 (1.19.0 available)
00:00:04   leak_tracker 10.0.4 (10.0.5 available)
00:00:04   leak_tracker_flutter_testing 3.0.3 (3.0.5 available)
00:00:04   material_color_utilities 0.8.0 (0.12.0 available)
00:00:04   meta 1.12.0 (1.15.0 available)
00:00:04   platform 3.1.4 (3.1.5 available)
00:00:04   string_scanner 1.2.0 (1.3.0 available)
00:00:04   test_api 0.7.0 (0.7.3 available)
00:00:04   vm_service 14.2.1 (14.2.4 available)
00:00:04 Got dependencies!
00:00:04 9 packages have newer versions incompatible with dependency constraints.
00:00:04 Try `flutter pub outdated` for more information.
00:00:04 Launching integration_test/webview_flutter_test.dart on Web Server in debug mode...
00:00:31 Waiting for connection from debug service on Web Server...         26.3s
00:00:31 integration_test/webview_flutter_test.dart is being served at http://localhost:7357
00:00:31 The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
00:00:38 All tests passed.
00:00:38 Application finished.
00:00:39 Stopping chromedriver
00:00:39 
00:00:39 
00:00:39 ------------------------------------------------------------
00:00:39 Run overview:
00:00:39   packages/webview_flutter/webview_flutter_web - ran
00:00:39 
00:00:39 Ran for 1 package(s)
00:00:39 
00:00:39 
00:00:39 No issues found!

vs CI. 🤷

[ditman]

The problem seems to stem from some DDC modules not being ready/available when the app starts. It doesn't always reproduce, but it can be seen when the app fails to start with --no-headless flag set:

(I've seen it also fail with a very similar trace, but complaining about IntegrationTestWidgetsFlutterBinding not being defined... Smells like a race condition! Are we attempting to start the app before the compilation is ready??)

I wonder if testing in --profile mode would yield better results (in general)

[ditman]

I wonder if testing in --profile mode would yield better results (in general)

It probably does, but we can't use that mode, because asserts don't work in --profile, and we have some tests that check that assertions are thrown.

[ditman]

Try by inlining the bootstrap file.

[ditman]

master may have a solution for this. Updating branch.

[ditman]

(Updating master seems to have been beneficial!)

I'm going to try next to revert the inlining of the bootstrap, it might not be needed!

[Rexios80]

I wonder if testing in --profile mode would yield better results (in general)

It probably does, but we can't use that mode, because asserts don't work in --profile, and we have some tests that check that assertions are thrown.

@ditman Related to asserts with WASM integration tests: flutter/flutter#151426

[stuartmorgan]

LGTM

[ditman]

Turns out the flakiness in --canvaskit mode is now fixed in master, but the fix won't be backported to the current stable (v3.22).

@stuartmorgan should I roll back the --canvaskit mode in tests until the fix lands in the next stable? Until then, web tests in stable are extra flaky. WDYT?

[stuartmorgan]

Turns out the flakiness in --canvaskit mode is now fixed in master, but the fix won't be backported to the current stable (v3.22).

@stuartmorgan should I roll back the --canvaskit mode in tests until the fix lands in the next stable? Until then, web tests in stable are extra flaky. WDYT?

This is looking really bad on stable so far; we should make CI run canvaskit only on master ASAP.

[ditman]

Working on this right now.

[ditman]

This is looking really bad on stable so far; we should make CI run canvaskit only on master ASAP.

@stuartmorgan PR posted #7146