Connect hardware keyboard for simulators in UI tests on CI

[jaclync]

Description

CI is failing on trunk for a UI test in iPad simulator, which I can reproduce:

woocommerce-ios/WooCommerce/UITestsFoundation/Screens/Orders/CustomerNoteScreen.swift

Line 36 in a2dd518

XCTAssertTrue(app.keyboards.firstMatch.exists)

It seems to have failed from not being able to find a keyboard. When I ran the same test case while connecting hardware keyboard manually on the failing screen, the test passed.

After some time-boxed search, a solution I found was to enable the hardware keyboard in the UI tests CI script:

defaults write com.apple.iphonesimulator ConnectHardwareKeyboard -bool true

With this change in the UI tests script, UI tests are passing on CI now.

There are still some flaky test cases in the UI tests, I checked a few commits in trunk and this seems to happen sometimes. I tried triggering another build for the PR commit, and the iPhone flaky tests from the first run are gone in the second build. I'm not sure if this change might have an impact on the frequency of flaky tests though, please feel free to share any concerns.

Steps to reproduce

Make sure the CI passes, especially the 2 UI tests steps.

Testing information

Locally, feel free to try reproducing the failure in OrdersTests when asserting the keyboard is open while verifying the order note.

Screenshots

I ran the CI build 3 times on the same PR commit and all passed, one without flaky tests and two with some flaky tests:

---

• I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

Reviewer (or Author, in the case of optional code reviews):

Please make sure these conditions are met before approving the PR, or request changes if the PR needs improvement:

• The PR is small and has a clear, single focus, or a valid explanation is provided in the description. If needed, please request to split it into smaller PRs.
• Ensure Adequate Unit Test Coverage: The changes are reasonably covered by unit tests or an explanation is provided in the PR description.
• Manual Testing: The author listed all the tests they ran, including smoke tests when needed (e.g., for refactorings). The reviewer confirmed that the PR works as expected on all devices (phone/tablet) and no regressions are added.

[dangermattic]

	1 Message
📖	This PR is still a Draft: some checks will be skipped.

Generated by 🚫 Danger

[wpmobilebot]

📲 You can test the changes from this Pull Request in WooCommerce iOS by scanning the QR code below to install the corresponding build.

	App Name	WooCommerce iOS
	Build Number	pr14307-168b266
	Version	20.9
	Bundle ID	com.automattic.alpha.woocommerce
	Commit	168b266
	App Center Build	WooCommerce - Prototype Builds #11444

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[staskus]

Thanks for the fix!

I wonder if we need a test that checks app.keyboards.firstMatch, as I understand it's a workaround way to check if note was added. Your changes will make sure this test succeeds on the CI but it can still fail locally as I understand. Although I tried testing it locally with different keyboard settings and couldn't reproduce it.

I'm not sure if this change might have an impact on the frequency of flaky tests though, please feel free to share any concerns.

The flakiness of tests is strange since Xcode 16 switch. I think it's worth trying to go with your fix regardless of my previous comments. We can observe if the flakiness increase or decrease after the merge.