Feature request: accesskit_consumer API documentation

[molenick]

Hi! Very cool project, really glad this exists and eager to learn more about how to interact with the accesskit_consumer API. I'm using egui and have been exploring its integration with accesskit and am pretty impressed with the out-of-the-box experience with Voiceover in my app.

I've started poking at the source a bit, but haven't quite found the right entry point to do anything useful yet and was hoping to get pointed at a good place to start.

At a high level what I think needs to happen is something like:

• Get a reference to a window that provides accesskit metadata
• Implement a handler to receive accessibility tree updates from the window
• Do something with the update

If it's welcomed, I'd be happy to document what I learn for the project so it's easier for the next person. Thank you!

[DataTriny]

Hello @molenick,

Documenting AccessKit is becoming more important as it gets integrated into more UI toolkits. Any help with this is much appreciated.

Please note that accesskit_consumer cannot be used on its own, rather it is embedded into the various platform-dependent accessibility backends that we call adapters. When you use egui you rely on accesskit_winit, which is just a wrapper around a winit window and a platform-specific adapter, determined at compile-time. The Tree structure you can find inside accesskit_consumer stores a complete state of the UI and is owned by the platform adapter. Your UI toolkit modifies this tree by pushing tree updates, and the platform adapter ensure that this tree gets exposed to the operating system's accessibility stack.

So, the entrypoint of accesskit_consumer is Tree::new, but platform-specific stuff need to happen before that.

[molenick]

Thank you for the guidance @DataTriny. I found this which I think should generally help me understand a bit more about what I need to do.

As far as documentation is concerned, what's most useful to the project? It seems like the primary developer use cases of accesskit are:

• UI frameworks developers - Platforms like egui that want to support screen readers
• Accessibility tool developers - Tool makers that make screenreaders etc
• UI developers - end users who will need to decide "this image is decorative and should not be included in updates" vs. "this image is informational and should be included in updates"

Right now my use is mostly the third case (ui developer) but I am also interested in the possibility of using accesskit to create ui test automation frameworks like XCUITest (ios et al) and Capybara (web).

For UI development, I'm mostly interested in learning "what do I need to know to implement accessible UI" in egui. From my cursory usage it seems like it just "works" but I imagine there are advanced topics like "suppress this information from tree updates" or "provide accessibility information for widgets that visually communicate information without an explicit text component."

For accessibility tooling/automated testing I'm interested in learning to see how easily accesskit could be leveraged to provide ui and accessibility testing for UIs written in frameworks that support accesskit. Things like: "hello world didn't appear on screen so fail a test" or "the contrast ratio between this text and its background are too low so fail a test".

[DataTriny]

@molenick You are totally right when pointing out the different kind of users we need to consider for the documentation, this is actually something I've been thinking about recently.

The first use case is probably currently the most relevant. Use case 3 is tricky because UI toolkits almost always expose their own API on top of AccessKit, so what we would communicate to application developers might not translate directly into practical advice they can use, assuming they are aware that AccessKit is used under the hood.

The types in the main accesskit crate should therefore be documented with both of these categories of users in mind, and the types which are only meant for UI toolkit developers (like ActionHandler for instance) should probably include a note for UI designers saying that they can ignore these. I even wonder if it would be cleaner to separate these in their own namespace.

I don't think we have enough experience with AccessKit to give advice to assistive technology developers at this time. Because we haven't yet found a generic API we could expose for all platform adapters, UI toolkit developers would have to add dedicated codepaths for these tools. This is fine if you build an embedded screen reader for a game running on a game console for example, not so much for a testing tool or an accessibility tree explorer.