Skip to content

Implement ISelectionProvider and ISelectionItemProvider #14019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Dec 17, 2024

Conversation

chiaramooney
Copy link
Contributor

@chiaramooney chiaramooney commented Oct 22, 2024

Description

Type of Change

  • New feature (non-breaking change which adds functionality)

Why

Accessibility on the new architecture should reach parity with Paper.

Resolves #13064
Resolves #13060
Resolves #13056
Resolves #13052
Resolves #13048
Resolves #13044
Resolves #13040
Resolves #13036
Resolves #13032
Resolves #12047
#13327
#13316

Resolves #13354
Resolves #13353

What

  • Adds support for ISelectionProvider interface
  • Adds support for ISelectionItemProvider interface
  • Adds support for accessibilityState:selected
  • Adjusts AccessibilityState native type on Windows to be std::optional instead of bool
  • Adjusts AccessibilityState native type on Windows to include multiselectable and required fields of type std::optional
  • Adds multiselectable and required fields to AccessibilityState type in JS with type boolean | undefined
  • Adds support for aria-multiselectable and aria-required.
  • Adds support for the following accessibilityActions: addToSelection, removeFromSelection, select
  • Fixes crash in AccessibilityInsights.

Testing

Locally tested for:

  • ISelectionItemProvider: AddToSelection, RemoveFrommSelection, Select, get_IsSelected, get_SelectionContainer
  • ISelectionProvider:  GetSelection, get_CanSelectMultiple, get_IsSelectionRequired
  • Parent as selection container, Non-parent ancestor as selection container
  • Adding/Removing SelectionItem from the Visual Tree
  • Adding/Removing SelectionContainer from Visual Tree
  • Calling addToSelection, removeFromSelection, and select accessibilityActions
  • Adjusting values of accessibilityState multiselectable and required
  • Child views that aren't selectable items
  • Selection of multiple items

Changelog

Should this change be included in the release notes: Yes

@chiaramooney chiaramooney requested a review from a team as a code owner October 22, 2024 23:32
Copy link
Contributor

@danielayala94 danielayala94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I didn't understand when and what should be addRef'd. If you can provide some clarity here, it would be helpful.

My sincere advice here is to move into smart pointers. Then, there won't be a need to manually manage memory for each and every object. In Windows, we rely on the winrt library (see winrt::com_ptr) to work with COM interfaces.

https://learn.microsoft.com/en-us/uwp/cpp-ref-for-winrt/com-ptr
https://learn.microsoft.com/en-us/windows/uwp/cpp-and-winrt-apis/consume-com

@chiaramooney
Copy link
Contributor Author

In general, I didn't understand when and what should be addRef'd. If you can provide some clarity here, it would be helpful.

My sincere advice here is to move into smart pointers. Then, there won't be a need to manually manage memory for each and every object. In Windows, we rely on the winrt library (see winrt::com_ptr) to work with COM interfaces.

https://learn.microsoft.com/en-us/uwp/cpp-ref-for-winrt/com-ptr https://learn.microsoft.com/en-us/windows/uwp/cpp-and-winrt-apis/consume-com

So unfortunately the methods here are defined by UIA (we are just adding implementations for them) so I can't change the type values of the IRawElementProviderFragment** types in most cases.

My understanding is if we are returning a reference to an object then we need to AddRef so the object is not destroyed prematurely. In the cases when we don't add AddRef but it is needed the app will crash when the program executes.


struct AccessibilityState {
bool disabled{false};
std::optional<bool> selected{std::nullopt}; // [Windows] - Do not remove; required for Windows ISelectionItemProvider Implementation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will these ever get upstreamed? Are the new props actually from aria?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard to say. I will file an issue to request they get upstreamed but it seems RN only decided to implement a subset of the aria props. The new props are real aria props. Meta just hasn't integrated them into the platform. Maybe they aren't needed for accessibility on mobile?

Copy link
Contributor

@danielayala94 danielayala94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@chiaramooney chiaramooney merged commit 3388c98 into microsoft:main Dec 17, 2024
59 checks passed
@chiaramooney chiaramooney deleted the cm-selection-provider branch December 17, 2024 18:07
acoates-ms pushed a commit to acoates-ms/react-native-windows that referenced this pull request Jan 10, 2025
)

Adds support for ISelectionProvider interface
Adds support for ISelectionItemProvider interface
Adds support for accessibilityState:selected
Adjusts AccessibilityState native type on Windows to be std::optional instead of bool
Adjusts AccessibilityState native type on Windows to include multiselectable and required fields of type std::optional
Adds multiselectable and required fields to AccessibilityState type in JS with type boolean | undefined
Adds support for aria-multiselectable and aria-required.
Adds support for the following accessibilityActions: addToSelection, removeFromSelection, select
Fixes crash in AccessibilityInsights.
acoates-ms added a commit that referenced this pull request Jan 13, 2025
)

* Implement ISelectionProvider and ISelectionItemProvider (#14019)

Adds support for ISelectionProvider interface
Adds support for ISelectionItemProvider interface
Adds support for accessibilityState:selected
Adjusts AccessibilityState native type on Windows to be std::optional instead of bool
Adjusts AccessibilityState native type on Windows to include multiselectable and required fields of type std::optional
Adds multiselectable and required fields to AccessibilityState type in JS with type boolean | undefined
Adds support for aria-multiselectable and aria-required.
Adds support for the following accessibilityActions: addToSelection, removeFromSelection, select
Fixes crash in AccessibilityInsights.

* [Fabric] Implement IRangeValueProvider and Adjust IValueProvider Implementation (#14212)

* IRangeValue Provider

* Adjust ReadOnly to Use Prop Value

* Complete IRangeValueProvider Implementation

* Change files

* Update Snapshots

* Rework modal to be implemented entirely using public APIs (#14256)

* Rework modal implementation to use public APIs

* shutdown fix

* Change files

* format

* remove dead code

* format

* fixes

* fix

* UIA tree for root component should be kept distinct from main UIA tree

* input offset fix for sub rootviews

* Split modal into two componentview's so that we dont have multiple roots in our componentview tree

* format

* Ensure rootview removes itself from the island, before a new one adds itself on instance reload

* defork some modal test pages

* remove override

* fix

* fix

* snapshots

---------

Co-authored-by: Chiara Mooney <[email protected]>
acoates-ms pushed a commit to acoates-ms/react-native-windows that referenced this pull request Mar 11, 2025
)

Adds support for ISelectionProvider interface
Adds support for ISelectionItemProvider interface
Adds support for accessibilityState:selected
Adjusts AccessibilityState native type on Windows to be std::optional instead of bool
Adjusts AccessibilityState native type on Windows to include multiselectable and required fields of type std::optional
Adds multiselectable and required fields to AccessibilityState type in JS with type boolean | undefined
Adds support for aria-multiselectable and aria-required.
Adds support for the following accessibilityActions: addToSelection, removeFromSelection, select
Fixes crash in AccessibilityInsights.
acoates-ms added a commit that referenced this pull request Mar 12, 2025
* Update Guardrails on Provider Instantiation (#13961)

* Update Provider Guardrails

* Change files

* Format

* Update Snapshots

* Use WebSocket factory in PkgInspectorConnection (#14202)

* Use WebSocket resource factory for pkgr inspector

* Change files

* clang format

* Implement ISelectionProvider and ISelectionItemProvider (#14019)

Adds support for ISelectionProvider interface
Adds support for ISelectionItemProvider interface
Adds support for accessibilityState:selected
Adjusts AccessibilityState native type on Windows to be std::optional instead of bool
Adjusts AccessibilityState native type on Windows to include multiselectable and required fields of type std::optional
Adds multiselectable and required fields to AccessibilityState type in JS with type boolean | undefined
Adds support for aria-multiselectable and aria-required.
Adds support for the following accessibilityActions: addToSelection, removeFromSelection, select
Fixes crash in AccessibilityInsights.

* [Fabric] Implement IRangeValueProvider and Adjust IValueProvider Implementation (#14212)

* IRangeValue Provider

* Adjust ReadOnly to Use Prop Value

* Complete IRangeValueProvider Implementation

* Change files

* Update Snapshots

* std::aligned_storage is deprecated (#14253)

* std::aligned_storage is deprecated

* Change files

* format

---------

Co-authored-by: Andrew Coates <[email protected]>

* Update WinUI3ExperimentalVersion from 1.6.240701003-experimental2 to 1.7.250109001-experimental2 (#14270)

* Update WinUI3ExperimentalVersion from 1.6.240701003-experimental2 to 1.7.250109001-experimental2

* Change files

* Update some type and method names that have changed in WinAppSDK 1.7

---------

Co-authored-by: Jon Thysell <[email protected]>

* Rework modal to be implemented entirely using public APIs (#14256)

* Rework modal implementation to use public APIs

* shutdown fix

* Change files

* format

* remove dead code

* format

* fixes

* fix

* UIA tree for root component should be kept distinct from main UIA tree

* input offset fix for sub rootviews

* Split modal into two componentview's so that we dont have multiple roots in our componentview tree

* format

* Ensure rootview removes itself from the island, before a new one adds itself on instance reload

* defork some modal test pages

* remove override

* Enable basic XamlIsland in rntester within playground app when UseExperimentalWinUI3 is true (#14272)

* Enable basic XamlIsland in rntester within playground app when UseExperimentalWinUI3 is true

* Change files

* Fix build break due to unreferenced parameter

* lint

* Move the implementation of CalendarView to sample-custom-component, and use codegen

* Enable CalendarView only when UseExperimentalWinUI3 is true, and add CalendarView.g.h to git

* clang format

* Update snapshots and remove some unused variables

* Update snapshotPages.test.js.snap

* Remove new page from the visitAllPages test

* Remove Paper-only code from Fabric builds of Microsoft.ReactNative (#14289)

Removes most of the Paper-only code from Fabric builds of Microsoft.ReactNative. This is only a first pass - there may be more that needs to be removed (or stuff that needs to be put back in).

- Bug fix (non-breaking change which fixes an issue)
- Breaking change (fix or feature that would cause existing functionality to not work as expected)

Besides nearly doubling the size of the binary files, all of the Paper-only code is functionally useless in Fabric builds, which at worse is dead weight, and at best is confusing to developers.

Resolves #13405

Updated project files to not load/compile paper-only modules. Updated modules that are used by both to correctly only include the Paper or Fabric code they need.

None, but here are some preliminary binary size numbers for x64 release:

| File | Paper | Fabric (Before) | Fabric (After) |
|:-|:-:|:-:|:-:|
| Microsoft.ReactNative.dll | 4.02 MB | 8.32 MB | 6.47 MB (🔻1.85 MB / 22% ) |
| Microsoft.ReactNative.winmd | 64.0 KB | 121.0 KB | 92.0KB (🔻29 KB / 24%) |

Verified all existing tests still pass and fabric apps work.

Should this change be included in the release notes: _yes_

Remove Paper-only code from Fabric builds of Microsoft.ReactNative

* RNIsland UIA fragment root should report parents fragment root (#14302)

* Fix crash running on server 2016

* RNIsland UIA fragment root should report parents fragment root

* Change files

* fix

* use react tag for runtime id

* fix

* fix

* fix

* Fix crash when currently focused element gets marked as enableFocusRing=false (#14306)

* Fix crash when currently focused element gets marked as enableFocusRing=false

* Change files

* [Fabric] Add Functional Tests for Switch Component (#14297)

* Add Functional Tests for Switch

* Fix Tests

* Update Snapshots

* [Fabric] Use PopupWindowSiteBridge for Modal when USE_EXPERIMENTAL_WINUI3 is true (#14284)

* add modal implementation with PopupWindowSiteBridge

* Change files

* add conditional

* feedback

* Initial support for UIA and tab navigation for a child Island (#14305)

* Basic support for stitching the UIA tree for a ContentIslandComponentView's child

* Updated comment

* Change files

* Support shift+tab, and move Automation event handlers to ContentIslandComponentView

* Allow portals to have independent layout constraints and scale factor (#14315)

* Allow portals to have independent layout constraints and scale factor

* format

* change files

* fix

* Round Focus visuals by default, fix nudge rendering (#14312)

* Round Focus visuals by default, fix nudge rendering

* Change files

* Only build OfficeReact.Win32 with UseFabric enabled (#14319)

This PR changes our build system to only build the version of `OfficeReact.Win32` with Fabric enabled.

- Breaking change (fix or feature that would cause existing functionality to not work as expected)

We are building both versions of Desktop in our CI, but the `-Fabric` version is in fact a strict superset of functionality and the only one that Office uses now.

Changed the default so that Desktop always builds with `RnwNewArch` (and therefore `UseFabric`) enabled. Removed the matrix configurations for DesktopFabric from all CI. Fixed the affected nuspec file. Also fixed Desktop publish to include ARM64EC for Fabric and the an issue caused by #14311.

N/A.

Verified the build works.

Should this change be included in the release notes: _no_

* Fix react devtools hitting an assert on launch (#14320)

* ignore unsupported methods

* delete Super::DispatchCommand call

* Change files

* [Fabric] Fix PlatformColor (#14325)

* Fix platform color

* Change files

* Add missing theme colors

* snapshots

* TurboModule spec checking should accept hstring for string arguments (#14322)

* TurboModule spec checking should accept hstring for string arguments

* Change files

* example

* [Fabric] Add Support for ITextProvider, ITextProvider2, and ITextRangeProvider (#14332)

* Partial Implementation

* Add More Implementation

* More Implementation

* More Implementation

* Add Support for Bounding Rectangles

* Complete Initial Implementation

* Change files

* Code Cleanup

* Code Cleanup

* Update Snapshots + Code Cleanup

* Update Snapshots

* Address Feedback

* Address Feedback

* Update Snapshots

* Fix Tests

---------

Co-authored-by: Jon Thysell <[email protected]>

* [Fabric] Fix modal height (#14343)

* fix modal height

* Change files

* reverse change

* add override

* Fix stale focus rects left in UI when last focusable component is removed (#14359)

* Fix stale focus rects left in UI when last focusable component is removed

* Change files

---------

Co-authored-by: Tatiana Kapos <[email protected]>

* Fix issue when calling arrange on an RN Island during load (#14362)

* Fix issue when calling arrange on an RN Island during load

* Change files

---------

Co-authored-by: Tatiana Kapos <[email protected]>

* Fix tooltips in high dpi (#14397)

* Fix tooltips in high dpi

* Change files

* format

* tooltips dont respect textscalefactor

* Fix build issue building component codegen using clang (#14393)

* Fix build issue building component codegen using clang

* Change files

* Components do not lose hover state if pointer leaves window before it leaves the component (#14375)

* Components do not lost hover state if pointer leaves window before it leaves the component

* format

* Change files

---------

Co-authored-by: Tatiana Kapos <[email protected]>

* Property updates switching between PlatformColors would no-op (#14398)

* Property updates switching between PlatformColors would no-op

* Change files

* format

* [Fabric] Add Support for Role Prop (#14352)

* Add Support for Role Prop

* Change files

* Format

* Update Tests

---------

Co-authored-by: Tatiana Kapos <[email protected]>

* Fix bool operator on transparent colors returning false (#14413)

* Fix crash loading logbox

* Fix bool operator on transparent colors returning false

* Change files

* format

* change files

* codegen updates

* update overrides

* snapshots

---------

Co-authored-by: Chiara Mooney <[email protected]>
Co-authored-by: Julio César Rocha <[email protected]>
Co-authored-by: Tiago <[email protected]>
Co-authored-by: JesseCol <[email protected]>
Co-authored-by: Jon Thysell <[email protected]>
Co-authored-by: Tatiana Kapos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Accessibility Area: ActivityIndicator Area: Component Views Area: Fabric Support Facebook Fabric Area: Image Area: Modal Area: RefreshControl Area: ScrollView Area: Slider Area: Switch Area: Text Area: TextInput Area: View Props https://reactnative.dev/docs/view#props Area: View New Architecture Broad category for issues that apply to the RN "new" architecture of Turbo Modules + Fabric Parity: Fabric vs. Paper RNW Fabric does not look or behave like RNW Paper
Projects
Status: No status
5 participants