New approach for end-to-end tests

[TsayAdobe]

Traditionally we put the test scripts for a website in a git repo. However, for a CMS, end-to-end test scenarios are associated with the content. If the content is changed, it can break the test scenarios. A CMS website usually involves designers, authors, and developers. With the content and test scripts in two places, stakeholders may not be able to define test coverage and review scenario correctness.

I propose a new approach to doing E2E test scenarios for Franklin. Say, you have a page named “example” (e.g., example.docx in SharePoint). Then the page “example-test” (e.g., example.test.docx in SharePoint) has the test scenarios in the test blocks. When you execute E2E test automation for the “example” page, the test framework automatically gets test scenarios from the “example-test” page.

The benefits of having test scripts close to the content:

1. Bring test scripts and the content together in one place. Stakeholders can easily review test scenarios.
2. The .test document becomes the place for communicating ideas and keeping the team in sync. A designer can draft the scenarios. The author can refine each scenario with concrete steps. The developer implements automation based on these steps.
3. Keep quality visible. If an author updates the content or a developer fixes a bug, they should ensure all test scenarios are passed.
4. Authors and developers share the responsibility for quality.
5. With Gherkin language, the .test document serves for documentation of product behaviors as well as a script for automated tests.

Example:

Page: https://www.webplatform4.com/example
Test scenarios: https://www.webplatform4.com/example-test

[trieloff]

This sounds interesting and could be something as a pattern for @dylandepass' StoryBook explorations.

[dylandepass]

Yeah interesting idea, I have some doubts about 4 and authors ability to do this but in a perfect world this would be great. I'm not familiar with Gherkin, does it have a way to convert a human readable statement into a query like this?

My current explorations with storybook cover E2E tests and also cover 1 & 3 above.

https://dylandepass.github.io/rosalind-boilerplate/?path=/story/blocks-header--header
(Check the interactions tab to see the tests, this is loading the actual header from content).

[TsayAdobe]

Proof of Concept
The framework repo: https://github.com/WebPlatformAutomation/Platform-UI
The demo project repo: https://github.com/WebPlatformAutomation/www