You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The model will not be unloaded from the VRAM after each generation, and using multiple identical nodes will load the model multiple times, resulting in high VRAM usage.
The screenshot shows the LLM VRAM usage after running the workflow once after a fresh boot and automatically unloading the SD model.
Even using --disable-smart-memory doesn't help.
Even if I use a 4090 graphics card, I still can't bear this consumption.
Is there any way to automatically unload the model from the VRAM after each generation? Or is there any other solution that can reduce the model's video memory usage?
The text was updated successfully, but these errors were encountered:
QL-boy
changed the title
The video memory usage is too high and cannot be released.
The VRAM usage is too high and cannot be released.
Apr 13, 2024
The model will not be unloaded from the VRAM after each generation, and using multiple identical nodes will load the model multiple times, resulting in high VRAM usage.
The screenshot shows the LLM VRAM usage after running the workflow once after a fresh boot and automatically unloading the SD model.
Even using
--disable-smart-memory
doesn't help.Even if I use a 4090 graphics card, I still can't bear this consumption.
Is there any way to automatically unload the model from the VRAM after each generation? Or is there any other solution that can reduce the model's video memory usage?
The text was updated successfully, but these errors were encountered: