MAUI Android Device Tests

[vaind]

Integrates MAUI TestUtils, as mentioned in dotnet/maui#3552 (comment) and dotnet/xharness#738

Some tests still fail, see e.g. #1703 (comment) so I've enabled continue-on-error for the Android test run and I suggest creating a follow-up issue to resolve these failures.

closes #1704
closes #1662

#skip-changelog

[mattjohnsonpint]

Hi. I have another big PR with more MAUI stuff coming shortly that includes the deployment flag. But if you can find a way to get device tests going that would be amazing.

Currently, I can only write unit tests that run on net6.0 target, but there's a lot of code we'll want to tests that is only applicable under a specific device target, such as net6.0-android.

I think we want to do something like what the MAUI team is doing here: https://github.com/dotnet/maui/tree/main/src/TestUtils

Looks like others are trying this as well. See dotnet/maui#3552 (comment)

[bruno-garcia]

Hi. I have another big PR with more MAUI stuff coming shortly that includes the deployment flag. But if you can find a way to get device tests going that would be amazing.

Currently, I can only write unit tests that run on net6.0 target, but there's a lot of code we'll want to tests that is only applicable under a specific device target, such as net6.0-android.

I think we want to do something like what the MAUI team is doing here: https://github.com/dotnet/maui/tree/main/src/TestUtils

Looks like others are trying this as well. See dotnet/maui#3552 (comment)

Captured this in this issue: #1704

[vaind]

I've tried integrating the MAUI TestUtils, as mentioned here dotnet/maui#3552 (comment) and here dotnet/xharness#738

but am getting a lot of errors, such as Severity Code Description Project File Line Suppression State Error CS0234 The type or namespace name 'Graphics' does not exist in the namespace 'Microsoft.Maui' (are you missing an assembly reference?) TestUtils.DeviceTests (net6.0-android), TestUtils.DeviceTests (net6.0-ios), TestUtils.DeviceTests (net6.0-maccatalyst), TestUtils.DeviceTests (net6.0-windows10.0.19041.0) C:\dev\sentry-dotnet\MauiTestUtils\src\DeviceTests\AssertionExtensions.Android.cs 10 Active

I assume it has something to do with the imports I had to comment out in the imported projects, but I don't really know how to resolve that...

[mattjohnsonpint]

Thanks for getting this going!

I pushed some changes here that fix the build issues. I also converted tabs to spaces and moved the MauiTestUtils folder under the test folder to not clutter the root.

The tests seems to work when called via xharness:

xharness android test --output-directory=./test_output/net6.0-android/ --app=./test/MauiTestUtils/samples/DeviceTests.Sample/bin/Debug/net6.0-android/com.microsoft.maui.testutils.devicetests-Signed.apk --package-name=com.microsoft.maui.testutils.devicetests

I didn't try the other targets or wire it to CI, but this is a good start.

[vaind]

I've done a couple more fixes to make the Sentry.Maui.Device.TestApp work... However, after trying to integrate the test into the app, I'm getting the following errors:

Severity	Code	Description	Project	File	Line	Suppression State
Error	NETSDK1150	The referenced project '..\Sentry.Testing\Sentry.Testing.csproj' is a non self-contained executable.  A non self-contained executable cannot be referenced by a self-contained executable.  For more information, see https://aka.ms/netsdk1150	Sentry.Maui.Device.TestApp	C:\Program Files\dotnet\sdk\6.0.400-preview.22301.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets	1096	
Error	NETSDK1082	There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'android-x86'.	Sentry.Maui.Device.TestApp	C:\Program Files\dotnet\sdk\6.0.400-preview.22301.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets	430	
Error	NETSDK1082	There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'android-x64'.	Sentry.Maui.Device.TestApp	C:\Program Files\dotnet\sdk\6.0.400-preview.22301.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets	430	
Error	NETSDK1082	There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'android-arm64'.	Sentry.Maui.Device.TestApp	C:\Program Files\dotnet\sdk\6.0.400-preview.22301.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets	430	
Error	NETSDK1082	There was no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'android-arm'.	Sentry.Maui.Device.TestApp	C:\Program Files\dotnet\sdk\6.0.400-preview.22301.10\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets	430	

I understand there are AspNetCore-specific utilities in the Sentry.Testing project and that causes these errors. Do you think we should move these out to a separate project or is there a way to include them conditionally, when not building for net6.0-android?

aspnet maui support tracking issue: dotnet/aspnetcore#35077

[mattjohnsonpint]

IMHO, we shouldn't have any ASP.NET Core dependencies in Sentry.Testing at all. That's supposed to be a common project for all test projects to use. I'll split them out.

[SimonCropp]

@vaind rebasing of develop will get you this change #1762

[mattjohnsonpint]

Yay, it works! 🎉