Nvidia TensorRT image generation is 6x slower!!! - A1111 WebUI - What is going on?

[dezorianguy]

Thank you for your time.

I installed the TensorRT extension within the A1111 WebUI and followed the instructions to "export engine" (screenshot). However, when generating images, it takes approximately three times as long.

For a test run, I first generated an image without the SD Unet activated, followed by one with the SD Unet activated for that specific model (screenshot). The regular generation takes about 0.5 minutes, while the TensorRT-boosted one takes around 6 minutes.

My setup: NVIDIA GeForce RTX 3060 (12GB).

[T9es]

Since I noticed you are using SDXL with this, are you positive you aren't overflowing to RAM? The TRT models do take up approx 5-6gb on the drive. Might be that you CAN'T load all of that into your 12gigs. On my 3090 I'm hovering at 20gb when using one loRA model and one optimized engine.

[dezorianguy]

Thanks for the suggestion. How would I check if that's the case? When generating an image the GPU is at 98-100%, in both TensorRT mode and regular.

[T9es]

What OS are you running? On windows 11 you can just open the task mgr and see everything inside the GUI.

I have no idea what you are using though. That console window looks like some sort of linux distro. If it in fact is linux and I'm not mistaken, open up a terminal and try nvidia-smi -l 1. This should give you the gpu usage every second.

If it's some custom skin for the console, since I see you are using the "C" drive, still try the command. It should still print out the usage if you are not able to read that from any task mgr.

If you see the vram being close to max or if it in fact is used up fully, you're overflowing to RAM.