RTX16 FP16 support

[artths]

Hi!
There is no information in integration overview about RTX16 series and FP16 support. 1650 supports FP16 but outputs black screen when I try it (DX11, SM5.0). Should it work?

[artths]

Also doesn't seem to work on RTX 20 series.

[UUNagato]

I have a similar problem with RTX3060Ti and FP16 setting. And in my case, it's because that FsrRcasH computes a wrong rcpL with APrxMedRcpH1 (line 863, ffx_fsr1.h). Simply replace this with AF1 rcpL=APrxMedRcpF1(AF1_(4.0)*lobe+AF1_(1.0)); can make it work. It seems APrxMedRcpH1 has some problems with it.

[tedlofgren]

GTX 1660Ti using FP16 (A_HALF) have similar issue on DX11, outputs black from RCAS. Works fine on DX12.
Fallback to FP32 works for both DX11 and DX12.

Seems related to FsrRcasH as @UUNagato stated. Changing to the suggested line does indeed output something, however not the expected result (for me at least).

According to FidelityFX-FSR-Overview-Integration (slide 27-28) special care have to be taken when choosing between FP32 and FP16, even when FP16 is supported.

Makes me question using FP16 at all.

[UUNagato]

Not sure if it's the reason but I guess a possible explanation for this bug is that APrxMedRcpH1 assumes we are using 16bit float and integer type. However, writing shader with types like "min16uint" or "min16float" doesn't guarantee the graphics card will run them in 16bits. It's often 32bits and thus APrxMedRcpH1 fails since it assumes the float number is 16bits. (0x778d is a constant for 16bit float layout).
AH1 APrxMedRcpH1(AH1 a) { AH1 b = AH1_AW1(AW1_(0x778d) - AW1_AH1(a)); return b * (-b * a + AH1_(2.0)); }

Replace it with APrxMedRcpF1 will work because the graphics card can always run 32bit float when you write float or float32_t.