[QUIC] IsSupportedIsTrue failing on s390x

[ManickaP]

SupportedLinuxPlatforms_IsSupportedIsTrue

It is also failing on helix for linux-armv6:

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-21c096763250462988/System.Net.Quic.Functional.Tests/3/console.f2cd77de.log?helixlogtype=result

and linux-s390x:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-053a9df11f6d452495/System.Net.Quic.Functional.Tests/3/console.2de1fd1f.log?helixlogtype=result

Originally posted by @am11 in #82108 (comment)

(ubuntu.1804.armarch.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:raspbian-10-helix-arm32v6

    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedWindowsPlatforms_IsSupportedIsTrue [SKIP]
      Condition(s) not met: "IsWindows"
    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Net.Quic/tests/FunctionalTests/MsQuicPlatformDetectionTests.cs(63,0): at System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodInvoker.Mono.cs(30,0): at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs(59,0): at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        Using unknown
        Running on Distro=raspbian VersionId=10.0

(ubuntu.2004.s390x.experimental.open) executed on machine ms02 running Linux-5.4.0-137-generic-s390x-with-glibc2.29
    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Net.Quic/tests/FunctionalTests/MsQuicPlatformDetectionTests.cs(63,0): at System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue()
        /_/src/mono/System.Private.CoreLib/src/System/Reflection/MethodInvoker.Mono.cs(30,0): at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, IntPtr* args)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs(59,0): at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
      Output:
        Using unknown
        Running on Distro=ubuntu VersionId=20.4

EDIT:
I did a search in DevOps tests:

AzureDevOpsTests
| where TestName == "System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue"
  and Outcome != "Pass"
| project-keep QueueName, TestRunName, BuildDefinitionName, RunCompleted, BuildReason, Branch
| order by BuildDefinitionName, QueueName, TestRunName, RunCompleted

And seems like these 2 platforms:

• net8.0-linux-Release-s390x-Mono_Release-Ubuntu.2004.S390X.Experimental.Open
• net8.0-linux-Release-armv6-Mono_Release-(Raspbian.10.Armv6.Open)[email protected]/dotnet-buildtools/prereqs:raspbian-10-helix-arm32v6

Are the only remaining failing ones, as of now.

Known Issue Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

It is also failing on helix for linux-armv6:

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-21c096763250462988/System.Net.Quic.Functional.Tests/3/console.f2cd77de.log?helixlogtype=result

and linux-s390x:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-053a9df11f6d452495/System.Net.Quic.Functional.Tests/3/console.2de1fd1f.log?helixlogtype=result

Originally posted by @am11 in #82108 (comment)

Author:	ManickaP
Assignees:	-
Labels:	untriaged, area-System.Net.Quic
Milestone:	-

[ManickaP]

Once #82159 is merged, this should become disabled-test issue.

[ManickaP]

This is another occurrence of #55979, i.e. msquic is silently dropping connections due to the machine being too slow.

I've temporarily added statistics (6709e8e) and they show multiple dropped connections.

I'm moving this to future as we don't have any solution for this problem. Maybe we could even close this against #55979

[wfurt]

That is pretty sad that we cannot make single test reliable. I'm wondering if we can use the counter to adjust the expectation or decrease the concurrency for ARM. (or tune the drop threshold)

cc: @nibanks

[nibanks]

That is pretty sad that we cannot make single test reliable.

You can't run a single test without failing? From my previous understanding, QUIC was getting overwhelmed by too much getting queued, and it was (by design) aborting the connection because it could no longer provide the time guarantees to have a reliable connection. But this wasn't caused by just 1 test.

[wfurt]

In this particular test, there is single listener and many parallel attempts to negotiate connection - this test was added recently to verify handling accept queue inside .NET.

[wfurt]

Failed for me on x64 as well while running tests on my VM over the weekend

    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.UnsupportedPlatforms_ThrowsPlatformNotSupportedException [SKIP]
      Condition(s) not met: "IsQuicUnsupported"
      Assert.Equal() Failure
      Expected: 100
      Actual:   94
      Stack Trace:
        /home/furt/github/wfurt-runtime/src/libraries/System.Net.Quic/tests/FunctionalTests/QuicListenerTests.cs(251,0): at System.Net.Quic.Tests.QuicListenerTests.Listener_BacklogLimitRefusesConnection_ParallelClients_ClientThrows_Core(Int32 backlogLimit, Int32 connectCount)
        /home/furt/github/wfurt-runtime/src/libraries/System.Net.Quic/tests/FunctionalTests/QuicListenerTests.cs(268,0): at System.Net.Quic.Tests.QuicListenerTests.Listener_BacklogLimitRefusesConnection_ParallelClients_ClientThrows_Core(Int32 backlogLimit, Int32 connectCount)

[ManickaP]

For some reason I have mismatched the issues, the comments belong to #82769 🤦

[wfurt]

I'm wondering if the milestone is right @ManickaP. Are we going to support armv6 or s390x architectures?
While #82769 happened on supported HW I think this is one is not.

[jkotas]

Fixed by #115129

[ManickaP]

I'll re-open this and test it out in a PR, we have 3 different platforms disabled against it, so let's see.

[am11]

Hard to tell if it was fixed on s390x as the Send to Helix step hangs and doesn't run any tests: https://dev.azure.com/dnceng-public/public/_build?definitionId=148. @uweigand, it looks like a connection issue? In case it is not fixable, we can disable this step so the leg is green.

[michaldobrodenka]

So ARMv6 is being removed? :( I'm using it in net8.

[uweigand]

Hard to tell if it was fixed on s390x as the Send to Helix step hangs and doesn't run any tests: https://dev.azure.com/dnceng-public/public/_build?definitionId=148. @uweigand, it looks like a connection issue? In case it is not fixable, we can disable this step so the leg is green.

Unfortunately the s390x Helix tests have been broken for a long time now, @directhex tried unsuccessfully to fix them but ran apparently into some network security issue where the outgoing connections from Microsoft/Azure systems to our builders are being blocked on the MS side. @giritrivedi and others on our side are currently working on setting up an alternative test environment that contacts our builders directly from GitHub, but that's not quite ready yet. I guess if there's no chance on the original builders getting fixed we might as well remove them. @leecow any thoughts?

[leecow]

Yes, I think removing them makes sense at this point.

[jkotas]

So ARMv6 is being removed?

We have deleted the CI validation. It has been broken for almost a year. The build images are outdated and tripping compliance checks.

ARMv6 is community-maintained port. If you would like to submit patches to keep it up-to-date and working, we will be happy to accept them.

[jkotas]

I'll re-open this and test it out in a PR, we have 3 different platforms disabled against it, so let's see.

My mistake. Thank you for catching it. I have closed wrong issue. (I meant to close #64674.)