Support for Gherkin test cases for QA on FHIR Core - Notice E Shelf Readiness

[f-odhiambo]

We need to see how best to implement this framework to FHIR Core for the Support for Gherkin test cases for QA on FHIR Core - Notice E Shelf Readiness

[mwkhan1983]

Compiling summary about BDD approach to testing using Gherkin & Cucumber. The purpose is to decide best collaborative approach to implement for Gherkin Test to FHIR core and other reference applications for BDD. Please share your thoughts and ideas.

BDD is a development method that encourages communication between teams. This collaborative approach brings the business and the technical aspects of projects together. This method allows teams to better communicate on requirements, detect problems early on, and maintain software over time with ease.

Cucumber reads executable specifications written in plain text and validates that the software does what those specifications say.

Gherkin is a set of grammar rules that makes plain text structured enough for Cucumber to understand.

Gherkin Syntax:

Given steps are used to describe the initial context of the system
When keyword defines the test action which will be executed
Then keyword defines the outcome of previous steps
And keyword is used to extend to your steps

Example:

Feature: Login Functionality

@Login @Regression
Scenario: The user logs in and sees the Home page
Given The “Log In” page is displayed
When  I enter “admin” as userName and “12345” as password
And   The “Login” button is pressed
Then  The “Home” page is displayed
And   The user details are displayed in the “Profile” sec

Cucumber implementation:

@Given("The {} page is displayed")
public void loginPageDisplayed(String login) {
  Assert.assertTrue(helper.isElementDisplayed(login));
}

@When("I enter {} as userName and [] as password")
public void enterCredentials(String user, String pass) {
   Maincall.loginView.enterUserName(user);
   Maincall.loginView.enterPassword(pass);
}

@And("The {} button is pressed)
public void clickLogin(String loginBtn) {
   Maincall.loginView.pressLogin(loginBtn);
}


@Then("The {} page is displayed")
public void homePageDisplayed(String homePage) {
   Assert.assertTrue(MainCall.loginView.verifySearch(homePage), 
   "User not logged in");
}

Listing pros and cons of using BDD below. For detail reading and understanding please see these links:

1. http://www.automated-testing.com/bdd/bdd-pros-and-cons/
2. https://www.functionize.com/blog/what-is-gherkin-how-do-you-write-gherkin-tests/
3. https://sqa.stackexchange.com/questions/33550/what-are-the-pros-cons-of-bdd-testing

Pros of using BDD:

1. Strong collaboration between business users, developers and testers
2. enforce building domain vocabulary and semi formal language (DSL) to express system behavior consistently within the organization.
3. living documentation
4. Early discovery of unknowns through discovery workshops

Cons of using BDD:

1. One of the biggest pro Strong collaboration is also one of the biggest con. Absence of any of business users, developers and testers can cause ambiguities and in ONA context managing collaboration between remote resources vs physically in same space
2. It consumes lot of time. So not good for short projects and where timelines are tight
3. Retrospectively fitting BDD to an existing project can be a time-consuming and difficult process. BDD encourages discovery workshops and writing tests even before development begins. In our case, we already have started with the development and would be retrofitting BDD. We also need to see if our specifications are good enough for BDD.
4. Usually it is developers who automate tests. In our context, developers may not have time to learn Gherkin + Cucumber first and then write tests along with development (and unit testing). If specific quality assurance resources are doing this job, they would need whole business context, understanding of existing code and clear and concise specifications.

Conclusion:
As per my research, in theory BDD sounds great. In practice, organizations rarely continue to use BDD past the initial 3-6 months. It is usually only QA team maintaining and rest of the organization not participating. It is important to think through potential difficulties and have a buy-in from all stakeholders before we commit to implement BDD approach.