Skip to content

Screenshot Tests

Jump to bottom
Stefan Arentz edited this page Mar 3, 2016 · 15 revisions

Running L10N Snapshots

Follow BUILDING.md first and make sure you can run the app in the simulator.

This is good for a test run with just one device and one language:

snapshot --project Client.xcodeproj --scheme L10nSnapshotTests \
  --erase_simulator \
  --devices "iPhone 6s" --languages "nl"

It is best to test with a non-english language to make sure you catch cases where you may incorrectly depend on English localizable ui items.

How do tests run

dekdjekde

Naming tests and snapshot images

Please give tests a descriptive name that makes it clear what we are screenshotting. Like test01IntroScreen or test55ClearPrivateData.

Tests have a sequence number in the name to ensure that they execute in the right order. Xcode runs the tests alphabetically so if one test depends on the state of another test, this is a simple way to ensure that order is maintained.

Please name snapshot() images like Settings-01, Settings-02, etc. This way they are grouped together and and up as files named like iPhone4s-Settings-01.png which makes it easy to post-process them and group them.

Adding accessibility identifiers

The most reliable way to find a button in a UI Test is by using the accessibilityIdentifier on elements.

Please give the accessibilityIdentifier a name that looks like ClassName.componentName.

For example, the button to toggle private mode now has a accessibilityIdentifier named "TabTrayController.togglePrivateMode". This is long, but it makes it very clear what the button is. It leaves no room for ambiguity or confusion.

Then you can simply use code like this:

func test04PrivateBrowsingTabsEmptyState() {
    let app = XCUIApplication()
    app.buttons["URLBarView.tabsButton"].tap() // Open tabs tray
    app.buttons["TabTrayController.togglePrivateMode"].tap() // Switch to private mode
    snapshot("PrivateBrowsingTabsEmptyState-01")
}

Things to know about Screenshot Tests

  • You can set enviroinment variables in the setUp() method. Currently only MOZ_SKIP_WHATSNEW and MOZ_WIPE_PROFILE exist.
  • The setUp() function is called before every test runs? IS THIS CORRECT?
  • Create separate test functions to capture managable units. For example do not combine screenshots for many parts of the app in a single test. Instead create multiple tests.
  • The order of the tests is very important. Xcode runs the tests sorted by name. This is why the tests are currently numbered like func test01Intro() and func test02SettingsScreen().

UI Testing Resources

Some random documents that I found useful:

Clone this wiki locally