feat: latest comfyui; fix: better GPU utilization for SD15 #270
Commits on Sep 23, 2024
-
-
Despite the name, `--novram` still allows the GPU to be used. However, comfyui uses this flag to much more aggressively avoid leaving tensors in VRAM. I am hoping that this will reduce VRAM OOMs and/or shared memory usage (in windows).
-
refactor/fix:
unload_models_from_vrammore oftenWith some recent comfyui changes it appears that the logic prior to this commit was not aggressive enough to avoid OOMs with relying on comfyui's internal decision making alone. This commit causes the worker to unload models from VRAM immediately after an inference result (if it is not about to be used) and right before post processing. Post processing as implemented today almost always overestimates the amount of free VRAM, and tends to cause OOMs or shared memory usage (on window) so more proactively unloading the model should help minimize that problem.
-
-
-
fix: more aggressively unload from system ram
The worker seems to be holding onto too much system RAM on average. I previously relied on comfyui internals to handle this implicitly but recent changes seem to have broken some assumptions I was making. This is an purposely over-zealous attempt to keep system RAM usage down.
-
-
-
-
feat: more configurable memory management
Redefines the broken existing `high_memory_mode` to leverage the recent memory management extension
-
-
-
-
-
-
fix: print to console
PROCESS_ENDEDmessage's infoThis will clarify when the situations such as the shared model manager failing to load or no models being found occur (e.g., when download_models.py isn't)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
fix: detect more deadlocks; less crashes w/ unsresponsive logic
- More fallback logic if there are jobs popped, processes available, but nothing happening. - Resolves certain problems with the unresponsive logic - The case of it ending all jobs after a long period of "No Job" messages from the server followed by successful pops. - Now no longer shuts down in error while processes are restarting
-
-
-
-
feat: time spent w/o jobs logging
Tracks the time spent without any available jobs. This will help worker operators identify potential issues with their configuration. A warning will be logged if the worker spends more than 5 minutes without any jobs, suggesting possible actions to increase job demand.
-
-
-
-
-
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.