[BUG]: Accessibility scanner flagging new violations that the in-browser scan does not find

[dmuelle]

Project

accessibility-checker-engine

Browser

Chrome

Operating system

MacOS

Description

When I ran the DCS accessibility scan after the update to the IBM Accessibility Requirement v7.3 ruleset, I saw many violations flagged target_spacing_sufficient, all for files that have not been modified since the last scan. These are flagged as actual violations, not "needs review".

The flags are attached to autogenerated "Parent topic:" links that are created as part of the IBM Docs build process. The parts of the HTML file that are being flagged are added by the doc build and are not present in the DITA source files.

But the in-browser IBM Accessibility checker for Chrome does not find these violations. They are only showing up in the batch scan.

Steps to reproduce

For example- this is one of the errors in the scan: Undersized target "a" does not have sufficient spacing of 12 CSS pixels from another target "a"

This is the page in IBM Docs: JAX-RS 2.0 integration with Atom. The flag is coming from the atuogenerated Parent topic: Deploying JAX-RS 2.0 applications to Liberty link at the bottom of the page. If you hover over the link, it pulls up the text you see in the error report.

All of the instances I've found are for similar "Parent topic" links.

Here is a link to a recent report with similar flags
https://wfm.dcs.ibm.com/product/SSAW57_liberty_test/reports/accessibility/53064bda8c82f705eb91c2ba8a48d723

Since these flags are not drawn by anything in our source files, I'm trying to determine whether the flags can be ignored, or theres a problem with the scanner, or a problem with the IBM docs framework.

[dmuelle]

Slack comment from IBM Docs admin Kerry Landis:

Note that this does not get flagged in the browser check - that is because once the Docs CSS is applied, it is not a violation. The WFM scan flags it because the scan is done against the raw HTML with no CSS applied to format things like link spacing.

We are hoping this flag can either be ignored by default in the WFM scans or downgraded from violation to "Needs review"

[shunguoy]

slack discussion thread: https://ibm-studios.slack.com/archives/C036P1CTN/p1727885751674519

[joefitterer]

Here's where I see them in our dev (test) environment. They are not flagged in our production documentation because the issues only appeared yesterday https://wfm.dcs.ibm.com/product/SSCT62_dev/reports/accessibility/2c8b5966edf93792462d2433eee1d525

[doveye]

I'm seeing these on links that aren't generated parent topic ones. Eg both the following links result in the error:

We also have a lot of generated parent topic links that aren't causing errors, so not sure what the pattern is.

My in-browser IBM Equal Access accessibility checker (set to use the 7.3 rulest) also doesn't raise these errors, either on the bare html file or in IBM Docs.

[philljenkins]

Slack comment from IBM Docs admin Kerry Landis:

Note that this does not get flagged in the browser check - that is because once the Docs CSS is applied, it is not a violation. The WFM scan flags it because the scan is done against the raw HTML with no CSS applied to format things like link spacing.

@dmuelle commented

We are hoping this flag can either be ignored by default in the WFM scans or downgraded from violation to "Needs review"

As Tom explained in the #accessibility-at-ibm thread,

So, we wouldn't say something is informational that is, in fact, a violation. So, how can the scanned content more accurately reflect the live content?

... as Nick noted, DCS is scanning a pre-publish page [that has the violation] that is different than the live page [with CSS applied that doesn't have the violation]

So from the Checker perspective, the scanner is working as designed and there's nothing to fix in the Checker tooling or rules. The fix would need to be in the DCS WFM process and usage of the Checker.

[nfernho]

Today @tombrunet and I meet to discuss this issue. There is a lot confusion around this so I hope this clears it up.

1. WFM will never ever apply special rules to the accessibility "Rule set" that we check against. Please understand that WFM is a "middle man" in this scenario. All WFM does is automate the IBM accessibility scanning (ACheker) so writers do not have to "manually do anything". WFM also provides a "nice" GUI that reads the report out of AChecker and allows users to ignore specific results per file in the report. That is all our tool is doing. We do nothing else. If the report is not working, that is a WFM bug. That is it. If you are not happy with results of the report, that is not anything WFM will ever fix. WFM reads the RuleSet that AChecker provides.

WFM is not in the business of understanding what is and what is not an accessibility violation. That is not the point of our tool.

@philljenkins - Thus, with what I stated above, your comment is never going to happen:

The fix would need to be in the DCS WFM process and usage of the Checker.

2. The main confusion that this error brought is that "on the live website" the problem does not exist, but it does in the report that WFM generates. This is because of what Kerry stated:

Note that this does not get flagged in the browser check - that is because once the Docs CSS is applied, it is not a violation. The WFM scan flags it because the scan is done against the raw HTML with no CSS applied to format things like link spacing.

In other words, this is NOT apples to apples. ACheck is not scanning the same file from the WFM scan that shows up on the live website.

3. What Tom has asked me for this, the .html file that ACheck is scanning from WFM. I have provided that file. This can help Tom identify if there is a bug in AChecker or not.

rwlp_jaxrs2.0_atom_wfm_scanned.html.zip

4. Depending on what Tom finds in #3 will determine what the solution to this problem is. However, if writers cannot wait for a solution from Tom, a work around would be to use the WFM report capabilities to "ignore" the violation.

5. Finally, I want to be very clear, that due to IBM Strategic Directions we will not be doing anything to address any accessibility issues with WFM outside of Sev1 Critical Failures (ie: the tool doesn't run). In other words, as I previously stated, WFM will not be fixing this issue.

[tombrunet]

@nfernho That example triggers a needs review, not a violation, so, still not entirely clear what's firing a violation. That said, I see some things in that example that shouldn't be flagging. We can look to fix that, but, not sure if it'll fix the violation scenario since that's not showing on our end in that page.

[shunguoy]

Triage: waiting for more information to investigate what exactly happened. Also, we need to fix the issue of the target size overlapping.

[doveye]

Earlier, I said "My in-browser IBM Equal Access accessibility checker (set to use the 7.3 ruleset) also doesn't raise these errors, either on the bare html file or in IBM Docs."

However, I've found that sometimes it does, and sometimes it doesn't. Eg here are 2 files that were both listed by the batch checker (results in WFM) as having the error (twice in the case of the second file):

I downloaded both of these files from COS and ran the Chrome browser checker on them. The first topic didn't show the error, the second one showed only the 2nd instance of the error (compared to the WFM report).

So compared to the batch checker, which I believe is running on these same files, 2 errors are not reported. This suggests that the two forms of the checker aren't producing the same results?

The structure of the relevant html looks the same to me in both files. For the file that has 2 instances of the error, I can't see why the first instance is even a problem. There's nothing that's close to the link (it's the links at the bottom of the file that are causing the errors).

In any case, what's the actual problem? The rule that's supposedly being violated is to ensure that "controls are easier to activate with a mouse or by touch" by not being too close together. But the only clickable thing here is a link. It might be close to an icon (it's not clear whether that's the trigger though) but it's not close to another control so why is this error being thrown at all?

html-files-with-error.zip

First file:

<aside role="complementary" aria-labelledby="q016800___title__1"><nav role="navigation">
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <img alt="[AIX, Linux, Windows]" src="ngalw.gif"><a data-hd-platform="ulwbis" data-hd-product="SSFKSJ_9.4.0" href="q016780_.html" title="Three types of channel exit are available to the IBM MQ MQI client environment on AIX, Linux, and Windows.">Channel-exit programs for MQI channels</a></div>
</div>
</nav></aside></article></main></body></html>

Second file:

<section class="section postreq" role="region" aria-labelledby="taskq118930___postreq__1"><div class="tasklabel"><h2 class="sectiontitle tasklabel" id="taskq118930___postreq__1">What to do next</h2></div><a href="q132610_.html" title="Certificates used by Advanced Message Security (AMS) for signing and encryption are stored in z/OS SAF key rings. You need to create these key rings and certificates before you can use AMS.">Creating key rings for Advanced Message Security</a></section>

</div>

<aside role="complementary" aria-labelledby="q118930___title__1"><nav role="navigation">
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <img alt="[z/OS]" src="ngzos.gif"><a data-hd-otherprops="ams" data-hd-platform="zos" data-hd-product="SSFKSJ_9.4.0" href="q019097_.html" title="Use these topics as a step by step guide for configuring Advanced Message Security (AMS).">Configuring Advanced Message Security for z/OS</a></div>
</div>
</nav></aside></article></main></body></html>

[philljenkins]

Fundamentally, accessibility scanning files without CSS applied is incomplete and may not be a valid scan in all cases.
CSS is fundamentally there for the look, feel, and often accessibility of the content.

Isn't there a way to apply the Doc's CSS before/during the WFM process so that the Checker scans a more complete and valid page of content that better reflects what the end user will receive?