NativeAOT: Run-time simd checking

[SupinePandora43]

Run-time simd checks can be beneficial in long-running cases (SpanHelpers.SequenceEqual for example)

Run-time simd checks can hurt performance because of redundant checks...

Could it be fixed? Eg;

void Method(){
    if(Avx2.IsSupported) avx2path();
    else softwarepath();
}
Method();
Method();
Method();

Can be transformed into:

if(Avx2.IsSupported){
    Method_avx2path();
    Method_avx2path();
    Method_avx2path();
} else {
    Method_softwarepath();
    Method_softwarepath();
    Method_softwarepath();
}

This could be done by inlining, but it won't work for the most part...

Compiler somehow should produce an unique codegen for every simd for every subsequent method:

void Start(){
    if(Avx2.IsSupported) ...;
    else ...;
    Continue();
}
void Continue(){
    End()
}
void End(){
    if(Avx2.IsSupported) ...;
    else ...;
}

Will produce

Start_Avx2
Start
Continue_Avx2
Continue
End_Avx2
End

But this unbiased codegeneration will cause dramatic size increase.
Could PGO be possibly used to know where checks are actually useful?

category:cq
theme:ready-to-run
skill-level:expert
cost:medium
impact:small

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

AFAIK NativeAOT just targets some low cpu instruction set.
Could compiler emit run-time simd checks based on profiles?

Like:

if(Avx2.IsSupported){ // not jit constant, an actual getter.
    for(...) ExpensiveCallWithAvx2();
} else {
    for(...) ExpensiveCall();
}

Obviously don't emit them if it will not provide a massive performance boost.

... Yet I don't know how much those instruction sets add performance ... Hold on... Bepuphysics2 may benefit from it! (I think)

Author:	SupinePandora43
Assignees:	-
Labels:	area-CodeGen-coreclr, untriaged
Milestone:	-

[EgorBo]

Related: #68038

[MichalStrehovsky]

The compiler already generates runtime checks wherever possible (e.g. ssse3.issupported is a runtime check with default settings). AVX.issupported cannot be a runtime check because of how ISA support is structured in RyuJIT. Not all use of VEX encoding in RyuJIT happens under IsSupported checks.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

AFAIK NativeAOT just targets some low cpu instruction set.
Could compiler emit run-time simd checks based on profiles?

Like:

if(Avx2.IsSupported){ // not jit constant, an actual getter.
    for(...) ExpensiveCallWithAvx2();
} else {
    for(...) ExpensiveCall();
}

Obviously don't emit them if it will not provide a massive performance boost.

... Yet I don't know how much those instruction sets add performance ... Hold on... Bepuphysics2 may benefit from it! (I think)

Author:	SupinePandora43
Assignees:	-
Labels:	area-CodeGen-coreclr, untriaged, area-NativeAOT-coreclr
Milestone:	-

[jakobbotsch]

Would the expectation be that RyuJIT would know only to use VEX encoding on paths that have checked Avx2.IsSupported?
I guess as an alternative NativeAOT could monomorphize these functions containing such checks and link them once on program start (or lazily) to avoid repeated runtime checks.

[jkotas]

When Avx+ instruction set is allowed, the VEX encoding can be used implicitly by RyuJIT pretty much anywhere, e.g. even for zeroing locals in the method prolog.

I think that the simplest variant of this would be for RyuJIT to only use the VEX encoding for explicit Avx and similar hardware intrinsics, and nothing else.

I guess as an alternative NativeAOT could monomorphize these functions containing such checks and link them once on program start (or lazily) to avoid repeated runtime checks.

Yes, that would be a more advanced version. We would need a profitability function that decides where the code duplication is worth it.

[SupinePandora43]

We would need a profitability function that decides where the code duplication is worth it.

Could be done with profiles?!