Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI/CD packages: Getting started, batch-scan, and command-line scanning #2218

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions accessibility-checker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Review the [accessibility-checker/src/README](src/README.md) for more informatio

Review the [accessibility-checker/boilerplates/README](boilerplates/README.md) and see examples for the following:

- [batch-scan](batch-scan): scan a batch of local files
- [batch-scan](batch-scan): scan a set of local files
- [cucumber-selenium](cucumber-selenium): Using [Cucumber](https://www.npmjs.com/package/cucumber) with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
- [jest](jest): Using a [Jest](https://www.npmjs.com/package/jest) test environment
- [jest-selenium](jest-selenium): Using a [Jest](https://www.npmjs.com/package/jest) test environment with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
Expand All @@ -46,9 +46,10 @@ Review the [accessibility-checker/boilerplates/README](boilerplates/README.md) a

## Baselines

Baselines are a helpful feature of `accessibility-checker` that can also be used in the test environment. The concept involves capturing a scan result as a 'baseline' so that future scans will pass if they match the baseline. If they differ, then the test will fail. This feature is useful for issues that have been determined to be the following:
Baselines are a helpful feature of `accessibility-checker` that can also be used in the test environment. The concept involves capturing a scan result as a _baseline_ so that future scans will pass if they match the baseline. If they differ, then the test will fail. This feature is useful for issues that have been determined to be the following:

- false positives
- `Needs review` issues resolved
- issues scheduled to be fixed later
- capturing new regression issues

Expand Down
17 changes: 10 additions & 7 deletions accessibility-checker/boilerplates/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# accessibility-checker boilerplates

In this folder are a series of boilerplates created for a form
of "Hello World" for different test framework environments. The boilerplates
are:
of "Hello World" for different testing frameworks.
By using a boilerplate, developers can focus on the unique aspects of their project rather than building automated accessibility testing from scratch.
See the boilerplate ReadMe for your automated testing framework to get started:

* [batch-scan](batch-scan): Example usage of scanning a list (batch) of .html files
* [cucumber-selenium](cucumber-selenium): Using [Cucumber](https://www.npmjs.com/package/cucumber) with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [jest](jest): Using a [Jest](https://www.npmjs.com/package/jest) test environment with a baseline
* [jest-selenium](jest-selenium): Using a [Jest](https://www.npmjs.com/package/jest) test environment with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [mocha-selenium](mocha-selenium): Using a [Mocha](https://www.npmjs.com/package/mocha) test environment with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [protractor](protractor): Using a [Protractor](https://www.npmjs.com/package/protractor) test environment
* [webdriverio](webdriverio): Using a [Webdriverio](https://www.npmjs.com/package/webdriverio) test environment
* [jest](jest): Using a [Jest](https://www.npmjs.com/package/jest) testing framework with a baseline
* [jest-customRS](jest-customerRS): Using a [Jest](https://www.npmjs.com/package/jest) testing framework ...
* [jest-selenium](jest-selenium): Using a [Jest](https://www.npmjs.com/package/jest) testing framework with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [mocha-selenium](mocha-selenium): Using a [Mocha](https://www.npmjs.com/package/mocha) testing framework with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [protractor](protractor): Using a [Protractor](https://www.npmjs.com/package/protractor) testing framework with a [Selenium-webdriver](https://www.npmjs.com/package/selenium-webdriver) browser
* [webdriverio](webdriverio): Using a [Webdriverio](https://www.npmjs.com/package/webdriverio) testing framework environment
41 changes: 41 additions & 0 deletions accessibility-checker/boilerplates/batch-scan/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# batch-scan

Scanning a batch of HTML files in a repeatable fashion is a level of automation beyond using the command-line to scan a list of URLs or using the browser's DevTools extension to manually scan and store a multi-scan report:
- See [Extensions](https://github.com/IBMa/equal-access/wiki#extensions) for the list of browser extensions supported to interactively scan multiple pages, states, and store a multi-scan report.
- See [Command-Line multi-scan](https://github.com/IBMa/equal-access/wiki#scanning-multiple-pages) for information about one-time scanning a list of files, directories, or a list of URLs generated from a web crawler.

This boilerplate shows how a local set of HTML files can be repeatedly scanned and the results saved.

### Purpose of files in this boilerplate

**main.js** - In summary, this JavaScript code uses the `accessibility-checker` library to scan HTML files for accessibility issues,
and logs the results to the console
Here's a breakdown of what the code does:

- The `accessibility-checker` library is imported as aChecker
- An array named `sampNames` is defined, containing strings representing the names of HTML files to be scanned. Note that although some sample names are repeated, suggesting that the same HTML file might be scanned multiple times, the file names are repeated in this boilerplate simply as a filler. The same file would have to be changed dynamically to have different results reported.
- An asynchronous function is defined using an immediately invoked arrow function (IIFE). This function performs the accessibility checks.
- Inside the IIFE, a variable `idx` is initialized to keep track of the index for appending unique identifiers to the filenames.
- The `Promise.all()` method is used to run multiple promises concurrently. It takes an array of promises as its argument. In this case, the array is generated by mapping over the sampNames array.
- For each name in `sampNames`, a promise is created using `aChecker.getCompliance()`. This method checks the accessibility compliance of an HTML file.
- The HTML file path is constructed using the current working directory `(process.cwd())` and the _sample name_ with a unique _index_ appended.
- Once all promises are resolved, the results are stored in the results variable.
- The `aChecker.close()` method is called to release any resources used by the `accessibility-checker` library.
- A success message is logged to the console, indicating the number of pages scanned.
- If an error occurs during the scanning process, it is caught and logged to the console using `console.error(err)`.

**package.json** - In summary, this file defines a Node.js project with a specific entry point, version, description, and dependencies. It also includes a custom script for testing the application.
file for a Node.js project named "@ibma/aat-boilerplate". Here's a breakdown of its contents:
- `name`: The name of the project is "`@ibma/aat-boilerplate`.
- `version`: The version of the project is `3.0.0`.
- `description`: The brief description of the project which is "Example usage of accessibility checker in batch scans".
- `main`: The entry point of the application is `src/app.js`.
- `scripts`: The section containing custom commands that can be run using `npm run`. In this case, there's only one script named "test" that runs the command `node main.js`.
- `engines`: The section specifying the required Node.js version for the project, which is ">=10".
- `devDependencies`: The section listing the development dependencies of the project. In this case, there's only one dependency, `accessibility-checker`, which is the local file located at "../../src".

**sample1.html** - A sample HTML file to be scanned.

## Baseline basics

See the [Baseline basics](https://github.com/IBMa/equal-access/wiki#baseline-basics) topic in the Wiki to get started using a baseline to compare scans over time and to prevent new issues from being introduced.
42 changes: 42 additions & 0 deletions accessibility-checker/boilerplates/cucumber-selenium/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# cucumber-selenium


... Scanning a batch of HTML files in a repeatable fashion is a level of automation beyond using the command-line to scan a list of URLs or using the browser's DevTools extension to manually scan and store a multi-scan report:
- See [Extensions](https://github.com/IBMa/equal-access/wiki#extensions) for the list of browser extensions supported to interactively scan multiple pages, states, and store a multi-scan report.
- See [Command-Line multi-scan](https://github.com/IBMa/equal-access/wiki#scanning-multiple-pages) for information about one-time scanning a list of files, directories, or a list of URLs generated from a web crawler.

This boilerplate shows how ... can be repeatedly scanned and the results saved.

### Purpose of files in this boilerplate

**main.js** - In summary, this JavaScript code uses the `accessibility-checker` library to scan HTML files for accessibility issues,
and logs the results to the console
Here's a breakdown of what the code does:

- The `accessibility-checker` library is imported as aChecker
- An array named `sampNames` is defined, containing strings representing the names of HTML files to be scanned. Note that although some sample names are repeated, suggesting that the same HTML file might be scanned multiple times, the file names are repeated in this boilerplate simply as a filler. The same file would have to be changed dynamically to have different results reported.
- An asynchronous function is defined using an immediately invoked arrow function (IIFE). This function performs the accessibility checks.
- Inside the IIFE, a variable `idx` is initialized to keep track of the index for appending unique identifiers to the filenames.
- The `Promise.all()` method is used to run multiple promises concurrently. It takes an array of promises as its argument. In this case, the array is generated by mapping over the sampNames array.
- For each name in `sampNames`, a promise is created using `aChecker.getCompliance()`. This method checks the accessibility compliance of an HTML file.
- The HTML file path is constructed using the current working directory `(process.cwd())` and the _sample name_ with a unique _index_ appended.
- Once all promises are resolved, the results are stored in the results variable.
- The `aChecker.close()` method is called to release any resources used by the `accessibility-checker` library.
- A success message is logged to the console, indicating the number of pages scanned.
- If an error occurs during the scanning process, it is caught and logged to the console using `console.error(err)`.

**package.json** - In summary, this file defines a Node.js project with a specific entry point, version, description, and dependencies. It also includes a custom script for testing the application.
file for a Node.js project named "@ibma/aat-boilerplate". Here's a breakdown of its contents:
- `name`: The name of the project is "`@ibma/aat-boilerplate`.
- `version`: The version of the project is `3.0.0`.
- `description`: The brief description of the project which is "Example usage of accessibility checker in batch scans".
- `main`: The entry point of the application is `src/app.js`.
- `scripts`: The section containing custom commands that can be run using `npm run`. In this case, there's only one script named "test" that runs the command `node main.js`.
- `engines`: The section specifying the required Node.js version for the project, which is ">=10".
- `devDependencies`: The section listing the development dependencies of the project. In this case, there's only one dependency, `accessibility-checker`, which is the local file located at "../../src".

**sample1.html** - A sample HTML file to be scanned.

## Baseline basics

See the [Baseline basics](https://github.com/IBMa/equal-access/wiki#baseline-basics) topic in the Wiki to get started using a baseline to compare scans over time and to prevent new issues from being introduced.
5 changes: 4 additions & 1 deletion accessibility-checker/boilerplates/jest/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# jest

Using a [Jest](https://www.npmjs.com/package/jest) testing framework with a baseline.

## Baseline basics
This folder covers "baseline basics".

We have two tests - one referencing a baseline (stored at `baselines/IMG_BASELINE.json`) and one that does not.
Expand All @@ -18,6 +21,6 @@ At the time of this commit, if you look at the results for `Image missing alt wi
Help: https://able.ibm.com/rules/archives/2024.06.17/doc/en-US/img_alt_valid.html#%7B%22message%22%3A%22The%20image%20has%20neither%20an%20accessible%20name%20nor%20is%20marked%20as%20decorative%20or%20redundant%22%2C%22snippet%22%3A%22%3Cimg%20src%3D%5C%22hello.png%5C%22%3E%22%2C%22value%22%3A%5B%22VIOLATION%22%2C%22FAIL%22%5D%2C%22reasonId%22%3A%22fail_no_alt%22%2C%22ruleId%22%3A%22img_alt_valid%22%2C%22msgArgs%22%3A%5B%5D%7D
```

We can then add to the `document.body.innerHTML` in that test, following the linked `help` above, to remove the violation. In this example, a simple way is to add an appopriate `alt` attribute.
We can then add to the `document.body.innerHTML` in that test, following the linked `help` above, to remove the violation. In this example, a simple way is to add an appropriate `alt` attribute.

A useful exercise would be to extend this example to make use of the provided HelloWidget component.
29 changes: 29 additions & 0 deletions accessibility-checker/boilerplates/protractor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# protractor

Protractor is a wrapper around Selenium Webdriver that provides an automated testing framework,
which simulates user interaction with an Angular web application for a range of browsers and mobile devices.
It provides all features of Selenium WebDriver along with Angular specific features for seamless end to end testing.

## Baseline basics

[THIS IS COPIED OVER FROM JEST, NEEDS TO BE EDITED and VALIDATED]
We have two tests - one referencing a baseline (stored at `baselines/IMG_BASELINE.json`) and one that does not.

A scan was run previously that detected an issue. That scan was saved as a baseline.
The Checker will ignore issues stored in the baseline. This feature allows a team to snapshot where they're at to prevent new issues from being introduced. This also allows a team to fail on potential violations, but then store items in the baseline that they've assessed and determined were being addressed in some way.

When you `npm install` and `npm test` in this folder, you should expect to see one test fail and one test pass.

At the time of this commit, if you look at the results for `Image missing alt without Baseline` you will see a failure including:

```
- Message: The image has neither an accessible name nor is marked as decorative or redundant
Level: violation
XPath: /html[1]/body[1]/div[1]/img[1]
Snippet: <img src="hello.png">
Help: https://able.ibm.com/rules/archives/2024.06.17/doc/en-US/img_alt_valid.html#%7B%22message%22%3A%22The%20image%20has%20neither%20an%20accessible%20name%20nor%20is%20marked%20as%20decorative%20or%20redundant%22%2C%22snippet%22%3A%22%3Cimg%20src%3D%5C%22hello.png%5C%22%3E%22%2C%22value%22%3A%5B%22VIOLATION%22%2C%22FAIL%22%5D%2C%22reasonId%22%3A%22fail_no_alt%22%2C%22ruleId%22%3A%22img_alt_valid%22%2C%22msgArgs%22%3A%5B%5D%7D
```

We can then add to the `document.body.innerHTML` in that test, following the linked `help` above, to remove the violation. In this example, a simple way is to add an appropriate `alt` attribute.

A useful exercise would be to extend this example to make use of the provided HelloWidget component.