Data driven tests in Thucydides

Introduction

Data-driven, or parameterized, testing involves using multiple sets of data for a single test case. For example, it may involve testing field validation with different possible values. These sorts of test are particularly useful for user interfaces using forms with large numbers of fields, or where the behaviour of the application can vary depending on the values entered by the user.

Data-driven testing with Thucydides

In JUnit 4, you can use the Parameterized test runner to perform data-driven tests. In Thucydides, you use the ThucydidesParameterizedRunner. This runner is very similar to the JUnit Parameterized test runner, except that you use the TestData annotation to provide test data, and you can use all of the other Thucydides annotations (Managed, ManagedPages, Steps and so on). This test runner will also generate proper Thucydides HTML and XML reports for the executed tests.

An example of a data-driven Thucydides test is shown below. In this test, we are checking that valid ages and favorite colors are accepted by the signon page of an (imaginary) application. To test this, we use several combinations of ages and favorite colors, specified by the testData() method. These values are represented as instance variables in the test class, and instantiated via the constructor.

@RunWith(ThucydidesParameterizedRunner.class)
public class WhenEnteringPersonalDetails {

    @TestData
    public static Collection<Object[]> testData() {
        return Arrays.asList(new Object[][]{
                {25, "Red"},
                {40, "Blue"},
                {36, "Green"},
        });
    }

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.myapp.com")
    public Pages pages;

    @Steps
    public SignupSteps signup;

    private Integer age;
    private String favoriteColor;

    public WhenEnteringPersonalDetails(Integer age, String favoriteColor) {
        this.age = age;
        this.favoriteColor = favoriteColor;
    }

    @Test 
    public void valid_personal_details_should_be_accepted() {
        signup.navigateToPersonalDetailsPage();
		signup.enterPersonalDetails(age, favoriteColor);
    }  
}

Reporting on data-driven web tests

When you generate reporting on data-driven web tests, the reports display full test outcomes and screenshots for each set of data. An overall story report is displayed for the data-driven test, which a test case for each row of test data. The test data used for each test is displayed in the report.

Running data-driven tests in parallel

Data-driven web tests can be long, especially if you need to navigate to a particular page before testing a different field value each time. In most cases, however, this is necessary, as it is unsafe to make assumptions about the state of the web page after a previous data-driven test. One effective way to speed them up, however, is to run them in parallel. You can configure ThucydidesParameterizedRunner tests to run in parallel by using the Concurrent annotation.

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent
public class WhenEnteringPersonalDetails {...

By default, this will run your tests concurrently, by default using two threads per CPU core. If you want to fine-tune the number of threads to be used, you can specify the 'threads' annotation property.

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent(threads="4")
public class WhenEnteringPersonalDetails {...

You can also express this as a value relative to the number of available processors. For example, to run 4 threads per CPU, you could specify the following:

@RunWith(ThucydidesParameterizedRunner.class)
@Concurrent(threads="4x")
public class WhenEnteringPersonalDetails {...