Can't encode multiple stream on version 460.106.00 & 460.73.01

[hao-kai]

• Nvidia driver version: 460.73.01
• GPU Model: RTX 2080 Ti
• Docker image: nvidia/cuda:11.1-cudnn8-devel-ubuntu18.04

# ./nvlax_encode -i /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.460.73.01 -o /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.460.73.01
[+] libnvidia-encode.so
[+] patched successfully

and run

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i /tmp/BigBuckBunny.mp4 \
-vf scale_npp=480:320 -c:a copy -c:v h264_nvenc -b:v 5M /tmp/output1.mp4 \
-vf scale_npp=240:120 -c:a copy -c:v h264_nvenc -b:v 8M /tmp/output2.mp4 \
-vf scale_npp=960:540 -c:a copy -c:v h264_nvenc -b:v 4M /tmp/output3.mp4 \
-vf scale_npp=960:540 -c:a copy -c:v h264_nvenc -b:v 4M /tmp/output4.mp4  

It shows

[h264_nvenc @ 0x55ba06f470c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
Error initializing output stream 3:0 -- Error while opening encoder for output stream #3:0 - maybe incorrect parameters such as bit_rate, rate, width or height

But if I use keylase/nvidia-patch, ffmpeg can transcode multiple streams.

I also tested on

• Nvidia driver version: 460.106.00
• GPU Model: RTX 3090
• Docker image: nvidia/cuda:11.1-cudnn8-devel-ubuntu18.04
  But I still cannot transcode.

[illnyang]

I have addressed that in b3699ad. Let me know if it works

[ajnightmare]

- Nvidia driver version: 460.106.00
- GPU Model: RTX 2080 Ti
- Docker image: nvidia/cuda:11.1-cudnn8-devel-ubuntu18.04

My Nvidia driver version varies because when I launch a job, it might land on a different machine.

I restarted the pod and clone nvlax

#  git rev-parse HEAD
b3699ad40c4dfbb9d46c53325d63ae8bf4a94d7f

and installed it.

# ./nvlax_encode -i /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.460.106.00 -o /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.460.106.00
[+] libnvidia-encode.so
[+] patched successfully

And I ran

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i /tmp/BigBuckBunny.mp4 \
-vf scale_npp=480:320 -c:a copy -c:v h264_nvenc -b:v 5M /tmp/output1.mp4 \
-vf scale_npp=240:120 -c:a copy -c:v h264_nvenc -b:v 8M /tmp/output2.mp4 \
-vf scale_npp=960:540 -c:a copy -c:v h264_nvenc -b:v 4M /tmp/output3.mp4 \
-vf scale_npp=960:540 -c:a copy -c:v h264_nvenc -b:v 4M /tmp/output4.mp4  

[h264_nvenc @ 0x5571a558a0c0] OpenEncodeSessionEx failed: out of memory (10): (no details)
Error initializing output stream 3:0 -- Error while opening encoder for output stream #3:0 - maybe incorrect parameters such as bit_rate, rate, width or height

Still failed in the same way.