[NativeAOT] GC suspension issues with System.Collections.Concurrent.Tests

[LakshanF]

A likely helpful testcase to go with #67805 that repro's on a win-x64 machine. The steps to repro are as below,

1. Build the libraries from the steps outlined in building and Running library tests.
2. cd src\libraries\System.Collections.Concurrent\tests
3. dotnet.cmd build -c Release /t:Test /p:TestNativeAot=true

The test should hang. The following selected call stacks show the issues with GC suspension,

 29  Id: 8d30.1510 Suspend: 1 Teb: 00000016`04a1a000 Unfrozen
 # Child-SP          RetAddr               Call Site
00 00000016`061ff6f8 00007ffd`b0862a0e     ntdll!NtWaitForSingleObject+0x14
01 00000016`061ff700 00007ff7`2350ebc1     KERNELBASE!WaitForSingleObjectEx+0x8e
02 00000016`061ff7a0 00007ff7`2350efe8     System_Collections_Concurrent_Tests!Thread::WaitForGC+0x41 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 90] 
03 (Inline Function) --------`--------     System_Collections_Concurrent_Tests!Thread::ReversePInvokeAttachOrTrapThread+0xc7 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 1220] 
04 00000016`061ff7d0 00007ff7`2350ef03     System_Collections_Concurrent_Tests!RhpReversePInvokeAttachOrTrapThread2+0xd8 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 1387] 
05 00000016`061ff800 00007ff7`2399d35e     System_Collections_Concurrent_Tests!RhpReversePInvoke+0x93 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 1401] 
06 00000016`061ff830 00007ff7`23515851     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_Thread__OnThreadExit+0xe [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\System.Private.CoreLib\src\System\Threading\Thread.NativeAot.Windows.cs @ 130] 
07 00000016`061ff870 00007ff7`2350f68a     System_Collections_Concurrent_Tests!PalDetachThread+0x41 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\windows\PalRedhawkMinWin.cpp @ 210] 
08 00000016`061ff8a0 00007ffd`b3094a0c     System_Collections_Concurrent_Tests!ThreadStore::DetachCurrentThread+0x4a [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\threadstore.cpp @ 162] 

24  Id: 8d30.8334 Suspend: 1 Teb: 00000016`04a0e000 Unfrozen
 # Child-SP          RetAddr               Call Site
00 (Inline Function) --------`--------     System_Collections_Concurrent_Tests!YieldProcessorNormalizedForPreSkylakeCount+0x18 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\yieldprocessornormalized.h @ 173] 
01 00000016`05bff400 00007ff7`23514cd5     System_Collections_Concurrent_Tests!ThreadStore::SuspendAllThreads+0x182 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\threadstore.cpp @ 262] 
02 00000016`05bff460 00007ff7`235108d0     System_Collections_Concurrent_Tests!GCToEEInterface::SuspendEE+0xa5 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\gcrhenv.cpp @ 811] 
03 00000016`05bff4a0 00007ff7`23743855     System_Collections_Concurrent_Tests!RhGetTotalAllocatedBytesPrecise+0x10 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\GCHelpers.cpp @ 299] 
04 00000016`05bff4f0 00007ff7`239a24cd     System_Collections_Concurrent_Tests!System_Collections_Concurrent_Tests_System_Collections_Concurrent_Tests_ConcurrentStackTests___c__DisplayClass17_0___Concurrent_Push_TryPop_WithSuspensions_b__1+0x45 [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Collections.Concurrent\tests\ConcurrentStackTests.cs @ 205] 
05 00000016`05bff580 00007ff7`239ac991     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_ExecutionContext__RunFromThreadPoolDispatchLoop+0x3d [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ExecutionContext.cs @ 268] 
06 00000016`05bff5d0 00007ff7`239a8012     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_Tasks_Task__ExecuteWithThreadLocal+0x231 [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\Task.cs @ 2349] 
07 00000016`05bff6f0 00007ff7`2399e353     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_ThreadPoolWorkQueue__Dispatch+0x1f2 [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ThreadPoolWorkQueue.cs @ 730] 
08 00000016`05bff780 00007ffd`b30a9d5a     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_ThreadPool__DispatchCallback+0x43 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\System.Private.CoreLib\src\System\Threading\ThreadPool.Windows.cs @ 373] 

16  Id: 8d30.97e8 Suspend: 1 Teb: 00000016`04bfd000 Unfrozen
 # Child-SP          RetAddr               Call Site
00 00000016`053fb908 00007ffd`b0862a0e     ntdll!NtWaitForSingleObject+0x14
01 00000016`053fb910 00007ff7`2350ebc1     KERNELBASE!WaitForSingleObjectEx+0x8e
02 00000016`053fb9b0 00007ff7`2350f053     System_Collections_Concurrent_Tests!Thread::WaitForGC+0x41 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 90] 
03 00000016`053fb9e0 00007ff7`2350ffbb     System_Collections_Concurrent_Tests!RhpWaitForGC2+0x33 [D:\work\Core\CurrentWork3\runtime\src\coreclr\nativeaot\Runtime\thread.cpp @ 974] 
04 00000016`053fba10 00007ff7`239a726c     System_Collections_Concurrent_Tests!RhpGcPollRare+0x2b
05 00000016`053fbaa0 00007ff7`239a3ec8     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_SpinWait__SpinOnceCore+0xdc [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\SpinWait.cs @ 238] 
06 00000016`053fbae0 00007ff7`239aeef8     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_ManualResetEventSlim__Wait_4+0xc8 [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\ManualResetEventSlim.cs @ 528] 
07 00000016`053fbb80 00007ff7`239aeb4f     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_Tasks_Task__WaitAllBlockingCore+0xf8 [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\Task.cs @ 4932] 
08 00000016`053fbbf0 00007ff7`2372eccb     System_Collections_Concurrent_Tests!S_P_CoreLib_System_Threading_Tasks_Task__WaitAllCore+0x2bf [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Private.CoreLib\src\System\Threading\Tasks\Task.cs @ 4847] 
09 00000016`053fbc90 00007ff7`240df18e     System_Collections_Concurrent_Tests!System_Collections_Concurrent_Tests_System_Collections_Concurrent_Tests_ConcurrentStackTests__Concurrent_Push_TryPop_WithSuspensions+0xbb [D:\work\Core\CurrentWork3\runtime\src\libraries\System.Collections.Concurrent\tests\ConcurrentStackTests.cs @ 210] 
0a 00000016`053fbcd0 00007ff7`239141e8     System_Collections_Concurrent_Tests!Internal_CompilerGenerated__Module___InvokeRetV+0x3e

[MichalStrehovsky]

@LakshanF would it be better to fold your test case into #67805? I don't know how to milestone this issue. Suspension is definitely 7.0 but if this doesn't track any new issue, it should be folded under the existing one. Multiple issues for a single bug/feature make tracking hard. Extra issues make it harder to see our progress to zero 7.0 issues.

[jkotas]

The test should hang. The following selected call stacks show the issues with GC suspension,

These stack shows threads that are happily suspended. The thing to look for in cases like this is a thread that is not suspended and actively running managed code.

[LakshanF]

I can see 1 call stack that seems broken and will paste that in #67805 as @MichalStrehovsky suggested

[LakshanF]

Closing this and added the test details as a scenario to the main issue