Investigate "integration"-style UI tests

[markerikson]

Bringing over a question from Discord:

We used Next on my last project. In addition to Cypress for E2E tests and typical Jest unit tests, we also wrote what I described as "integration"-level tests that loaded an entire page in Jest+RTL and interacted with it.

Initially we used next-page-tester, which took a route URL and simulated all of Next's data loading methods. Unfortunately, that stopped working as of Next 12 due to reliance on the internals. So, shortly before I left we switched to just loading the topmost "content" component per page.

This technique relied on having exported copies of some of the AJAX responses, so that we could use msw to mock out the requests.

Since we only ran our E2E tests nightly rather than on PR, that gave us a safety net during PR/build pushes.

How feasible is something like that here?

I'm guessing the problem is that Replay seems to rely on a live running browser instance in the cloud as a backend, and thus it might be difficult to mock API responses.

As an example: in #5782, I just went in and swapped out the logic for showing the "Quick Open" popup.

Ideally, I'd like to have an "integration-y" test that loads the Replay UI, hits "CTRL-P", asserts that the popup is visible, types in a few characters and asserts that we're trying to filter, and then fake-clicks outside to close it. I don't need the entire rest of the processing chain working there, but there would have to be enough mock data loaded that the UI can get to the point where hitting CTRL-P is valid and maybe return a filtered search result or something in the list

We discussed this a bit further in a frontend meeting. It sounds as if there's some provision for mocking specific Apollo and client-type functions in tests/mock/, but that can get fragile. Having canned API response data and using msw and mock-socket to feed them back may be easier.

Prioritization is a big question. At the moment I'd say it might be worth having me investigate this after the thunk conversion work is done.

Thoughts?

[abstractalgo]

To mention some of the things we've discussed:

The existence of test/mock/src/handlers for mocked protocol messages. But, as we all discussed together, mocking the entire backend and staying up-to-date with all of its changes is probably a no-go. Bootstrapping frontend app with some sort of dev server (or a real one) would pretty much be required. Currently, there're a lot of dependencies between different parts of the system, and what would it be required to only bootstrapping the minimal required app state and UI to test specific features is probably not even fully known.

Partially considered could be tests that intercept network requests (via msw or whatever else) to see if our app handles different scenarios gracefully and could help detect and eliminate those that we missed (also being reported in Sentry e.g.). This may or may not be a smaller effort than separately testing large number of UI modules, and would either way be a good step in that direction.