Playwright snapshot testing

What is snapshot testing?

Snapshot testing allows us to assert on how a page or element should look after various user flows or actions. Some items are difficult to assert specific statements on. For example we could assert that an element is visible which will pass a test. However we may not know if the element actually looks the way we expect it to.

Hence with by using snapshots it gives an extra layer of confidence that the software is acting as it should. Snapshot testing has mainly been used on echart elements with mocked data. So the graph will always look the same and we can catch any functional regressions.

Functionality

Method to wait for all network activity to cease, before taking a screenshot.

Example of snapshot method called in test.

Above is a simple test which is asserting that once all the user flows have been executed such as setting the base_date the final UI should look like so:

Here some mock data has been added to make PM2.5 cause a plot of AQI 3 on the in-situ line.

There is also some added logic to ensure that the page has finished loading before taking the screenshot.

How to take a snapshot

1. The most reliable way to take a snapshot is to use the toMatchSnapshot('') Playwright method. inside the quotes will be whatever the image shall be named and it's format for example toMatchSnapshot('cool-cat.png')
2. Note that this snapshot method requires that you run it through command line npx playwright test with filename or test name as optionals. Now upon running that command the test should fail like so:

Since the test cannot find an image to compare against, it will add the image that it got from running the test regardless.

3. Now update and tweak any tests until the actual image provided satisfies requirements. This will be your golden image which is basically just your expected image. Once satisfied with that image use the --update-snapshots flag which will then set that image in your codebase as the comparison snapshot.

For example I have a test called "Epic Test" of which i am now satisfied with the comparison image. I can enter npx playwight test -g "Epic Test" --update-snapshots which will now set that image as the golden standard for specifically that test.

4. Since we currently have playwright working across 3 browsers (Chromium, Firefox, Webkit) then playwright will create 3 golden images each corresponding to a browser:

This is done as each browser acts differently. One common difference is how Webkit (Safari) will have a snapshot that is 2x the size of Chromium and Firefox dimension wise. This is due to how browsers and viewports may have different device pixel ratios

Maintenance

Updating snapshots requires some time to manually check that the new updated image is correct.

1. Sit down with developers to confirm whether or not certain snapshots should be updated based on new implementation.

• This could also mean going through a code review to spot new functionality that could affect any snapshots.

2. If there are new changes then you may need to update your test steps or locators to facilitate this.
3. Once happy, run the test with --update-snapshots to create a new comparison image. The name will remain the same, but the snapshot will update.

Notes

Note that upon taking a snapshot it will be added to a new directory named after the related .spec.ts file.