Direct integration of AMD AMF H.264 and H.265 encoder

[Xaymar]

The integration of AMD AMF via FFmpeg is lacking, as it appears to be almost completely unmaintained in FFmpeg. This results in poor user experience and a lack of features/functionality compared to the older integration in the original obs-amd-encoder.

[Xaymar]

Useless features we may not want:

• Constrained Profiles
• Interlaced or Checkerboard output
• Multi-View and Stereo coding as OBS Studio supports neither
• SVC Encoding in H.264, as it is a waste of performance and time.
• PSNR, SSIM, Statistics, and BlockQp feedback - maybe useful in debugging, but not useful in encoding.
• Color Conversion as this is done in OBS Studio itself

Useful features end-users may want:

• Asynchronous encoding to OBS Studio to do other work on the encode thread.
• Multi-slice and Intra-Refresh coding for ultra low latency streaming.
• Pre-Analysis support for vastly improved quality from AMF.
• IDR/I-only encoding for lossless capture.
• HDR and Full Range support

[Xaymar]

Regarding Color Format:

As far as I am currently, AMD does not have any encoders capable of 4:2:2, 4:4:4 or RGB encoding. They do have hardware decoders for those formats, but no encoders support. Therefore, the encoder should specify a preferred format and/or handle color conversion in the way libOBS expects.

[Xaymar]

Regarding Multi-Slice and Intra-Refresh

These have no real usage for end users and are a nice to have, but not required for fulfilling this feature request. They allow users to have slightly more resilient or lower latency streams at a massive cost to quality. Additionally it is not confirmed if this encoding mode is even still supported by AMD hardware at all.