Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph. #5365

Closed
kairin opened this issue Oct 25, 2024 · 6 comments
Closed

RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph. #5365

kairin opened this issue Oct 25, 2024 · 6 comments
Labels
User Support A user needs help with something, probably not a bug.

Comments

@kairin
Copy link

kairin commented Oct 25, 2024

Your question

okay so below is the first error that i encountered when trying to use a SUPIR workflow.

the subsequent steps are what i used to get my comfyui running but with other errors...

Logs

ddd  venv
Command 'nvcc' not found, but can be installed with:
sudo apt install nvidia-cuda-toolkit
Fri Oct 25 22:39:20 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.51.01              Driver Version: 565.90         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    On  |   00000000:0B:00.0  On |                  N/A |
|  0%   40C    P2             17W /  320W |     602MiB /  16376MiB |      7%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
|    0   N/A  N/A    619741      C   /python3.12                                 N/A      |
+-----------------------------------------------------------------------------------------+
VIRTUAL_ENV=/home/ks/venv
VIRTUAL_ENV_PROMPT=(venv)
ddd  venv
CODEOWNERS       app               comfyui.prev2.log               fix_torch.py       models           pytest.ini        tests-unit
CONTRIBUTING.md  comfy             comfyui_screenshot.png          folder_paths.py    new_updater.py   requirements.txt  user
LICENSE          comfy_execution   cuda_malloc.py                  input              node_helpers.py  script_examples   utils
README.md        comfy_extras      custom_nodes                    latent_preview.py  nodes.py         server.py         web
__pycache__      comfyui.log       execution.py                    main.py            notebooks        temp
api_server       comfyui.prev.log  extra_model_paths.yaml.example  model_filemanager  output           tests
  on    bash   ks 3.12.3    master ≡                                                                                      0ms 
╭─ 22:39:29 |25 Oct, Friday |in     venv  comf
╰─❯ pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu124
Looking in indexes: https://download.pytorch.org/whl/cu124
Collecting xformers
  Downloading https://download.pytorch.org/whl/cu124/xformers-0.0.28.post2-cp312-cp312-manylinux_2_28_x86_64.whl (20.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.6/20.6 MB 10.5 MB/s eta 0:00:00
Requirement already satisfied: numpy in /home/ks/venv/lib/python3.12/site-packages (from xformers) (1.26.4)
Collecting torch==2.5.0 (from xformers)
  Downloading https://download.pytorch.org/whl/cu124/torch-2.5.0%2Bcu124-cp312-cp312-linux_x86_64.whl (908.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 908.2/908.2 MB 9.9 MB/s eta 0:00:00
Requirement already satisfied: filelock in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (3.16.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (4.12.2)
Requirement already satisfied: networkx in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (3.4.2)
Requirement already satisfied: jinja2 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (3.1.4)
Requirement already satisfied: fsspec in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (2024.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.127)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.5.8)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (11.2.1.3)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (10.3.5.147)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.3.1.170)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (12.4.127)
Requirement already satisfied: triton==3.1.0 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (3.1.0)
Requirement already satisfied: setuptools in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (75.2.0)
Requirement already satisfied: sympy==1.13.1 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.5.0->xformers) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/ks/venv/lib/python3.12/site-packages (from sympy==1.13.1->torch==2.5.0->xformers) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ks/venv/lib/python3.12/site-packages (from jinja2->torch==2.5.0->xformers) (3.0.2)
Installing collected packages: torch, xformers
  Attempting uninstall: torch
    Found existing installation: torch 2.5.1+cu124
    Uninstalling torch-2.5.1+cu124:
      Successfully uninstalled torch-2.5.1+cu124
Successfully installed torch-2.5.0+cu124 xformers-0.0.28.post2
  on    bash   ks 3.12.3    master ≡                                                                             1m 38s 383ms 
╭─ 22:41:02 |25 Oct, Friday |in     venv  comf
╰─❯ pip install upgrade torch
ERROR: Could not find a version that satisfies the requirement upgrade (from versions: none)
ERROR: No matching distribution found for upgrade
  on    bash   ks 3.12.3    master ≡                                                                                    564ms 
╭─ 22:41:19 |25 Oct, Friday |in     venv  comf
╰─❯ python main.py
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-10-25 22:41:38.549009
** Platform: Linux
** Python version: 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]
** Python executable: /home/ks/venv/bin/python
** ComfyUI Path: /home/ks/venv/comf
** Log path: /home/ks/venv/comf/comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.5 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager

Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.5.0+cu124
xformers version: 0.0.28.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync
Using xformers cross attention
[Prompt Server] web root: /home/ks/venv/comf/web
/home/ks/venv/lib/python3.12/site-packages/kornia/feature/lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)
### Loading: ComfyUI-Manager (V2.51.8)
### ComfyUI Revision: 2785 [52810907] | Released on '2024-10-24'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
/home/ks/venv/lib/python3.12/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 1.4.20 (you have 0.0.19). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
[ReActor] - STATUS - Running v0.5.1-b2 in ComfyUI
Torch version: 2.5.0+cu124
Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.5.0+cu124
xformers version: 0.0.28.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync

[rgthree-comfy] Loaded 42 exciting nodes. 🎉

WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `/home/ks/venv/comf/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 218 nodes successfully.

        "Don't let yesterday take up too much of today." - Will Rogers


Import times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/ks/venv/comf/custom_nodes/Skimmed_CFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-AutomaticCFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_JPS-Nodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Image-Saver
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Custom-Scripts
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-KJNodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager
   0.2 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_ExtraModels
   0.4 seconds: /home/ks/venv/comf/custom_nodes/comfyui_face_parsing
   0.8 seconds: /home/ks/venv/comf/custom_nodes/comfyui-reactor-node
   0.8 seconds: /home/ks/venv/comf/custom_nodes/was-node-suite-comfyui
   0.9 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager/extension-node-map.json [DONE]
got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 2448.5241737365723 True
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:31<00:00,  3.18it/s]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
model weight dtype torch.float16, manual cast: None
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Loading weights to:  cpu
Diffusion using fp16
Using XFORMERS
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Attempting to load SDXL model from node inputs
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 4897.0483474731445 True
Loading first clip model from SDXL checkpoint
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Loading second clip model from SDXL checkpoint
Attempting to load SUPIR model: [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Loaded state_dict from [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Encoder using bf16
[Tiled VAE]: input_size: torch.Size([1, 3, 2688, 1536]), tile_size: 512, padding: 32
[Tiled VAE]: split to 6x3 = 18 tiles. Optimal tile size 512x448, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████████████████████████| 1638/1638 [00:06<00:00, 242.84it/s]
[Tiled VAE]: Done in 7.317s, max VRAM alloc 664.499 MB
[Tiled VAE]: input_size: torch.Size([1, 4, 336, 192]), tile_size: 64, padding: 11
[Tiled VAE]: split to 5x3 = 15 tiles. Optimal tile size 64x64, original tile size 64x64
[Tiled VAE]: Executing Decoder Task Queue: 100%|████████████████████████████████████████████████████████| 1845/1845 [00:16<00:00, 113.50it/s]
[Tiled VAE]: Done in 16.819s, max VRAM alloc 1760.142 MB
captions:  [['']]
Batch captioning
Could not load library libcuda.so. Error: libcuda.so: cannot open shared object file: No such file or directory
Could not load library libcuda.so. Error: libcuda.so: cannot open shared object file: No such file or directory
Could not load library libcuda.so. Error: libcuda.so: cannot open shared object file: No such file or directory
!!! Exception during processing !!! cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph.
Traceback (most recent call last):
  File "/home/ks/venv/comf/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/ks/venv/comf/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 637, in condition
    _c, _uc = SUPIR_model.conditioner.get_unconditional_conditioning(cond, uncond)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 190, in get_unconditional_conditioning
    c = self(batch_c)
        ^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 211, in forward
    emb_out = embedder(batch[embedder.input_key])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 503, in forward
    outputs = self.transformer(
              ^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.py", line 997, in forward
    return self.text_model(
           ^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.py", line 901, in forward
    encoder_outputs = self.encoder(
                      ^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.py", line 824, in forward
    layer_outputs = encoder_layer(
                    ^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.py", line 559, in forward
    hidden_states, attn_weights = self.self_attn(
                                  ^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.py", line 491, in forward
    attn_output = torch.nn.functional.scaled_dot_product_attention(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph.

Other

when searching online this is the first item that i found similar to what i was facing:

huggingface/diffusers#9704 (comment)
@kairin kairin added the User Support A user needs help with something, probably not a bug. label Oct 25, 2024
@kairin
Copy link
Author

kairin commented Oct 25, 2024

so the first thing i did was tried to follow a suggestion to go to an older version of torch...

so i did.


ddd  venv
Command 'nvcc' not found, but can be installed with:
sudo apt install nvidia-cuda-toolkit
Fri Oct 25 22:58:42 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.51.01              Driver Version: 565.90         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    On  |   00000000:0B:00.0  On |                  N/A |
|  0%   39C    P2             17W /  320W |    2641MiB /  16376MiB |      7%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A        35      G   /Xwayland                                   N/A      |
|    0   N/A  N/A    620148      C   /python3.12                                 N/A      |
+-----------------------------------------------------------------------------------------+
VIRTUAL_ENV=/home/ks/venv
VIRTUAL_ENV_PROMPT=(venv)
ddd  venv
CODEOWNERS       app               comfyui.prev2.log               fix_torch.py       models           pytest.ini        tests-unit
CONTRIBUTING.md  comfy             comfyui_screenshot.png          folder_paths.py    new_updater.py   requirements.txt  user
LICENSE          comfy_execution   cuda_malloc.py                  input              node_helpers.py  script_examples   utils
README.md        comfy_extras      custom_nodes                    latent_preview.py  nodes.py         server.py         web
__pycache__      comfyui.log       execution.py                    main.py            notebooks        temp
api_server       comfyui.prev.log  extra_model_paths.yaml.example  model_filemanager  output           tests
  on    bash   ks 3.12.3    master ≡                                                                                      0ms 
╭─ 22:58:43 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu124
Looking in indexes: https://download.pytorch.org/whl/cu124
Collecting torch==2.4.1
  Downloading https://download.pytorch.org/whl/cu124/torch-2.4.1%2Bcu124-cp312-cp312-linux_x86_64.whl (797.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 797.1/797.1 MB 10.0 MB/s eta 0:00:00
Collecting torchvision==0.19.1
  Downloading https://download.pytorch.org/whl/cu124/torchvision-0.19.1%2Bcu124-cp312-cp312-linux_x86_64.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 10.2 MB/s eta 0:00:00
Collecting torchaudio==2.4.1
  Downloading https://download.pytorch.org/whl/cu124/torchaudio-2.4.1%2Bcu124-cp312-cp312-linux_x86_64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 6.4 MB/s eta 0:00:00
Requirement already satisfied: filelock in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (3.16.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (4.12.2)
Requirement already satisfied: sympy in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (1.13.1)
Requirement already satisfied: networkx in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (3.4.2)
Requirement already satisfied: jinja2 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (3.1.4)
Requirement already satisfied: fsspec in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (2024.10.0)
Requirement already satisfied: setuptools in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (75.2.0)
Collecting nvidia-cuda-nvrtc-cu12==12.4.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_nvrtc_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (24.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.7/24.7 MB 8.2 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.4.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_runtime_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (883 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 883.4/883.4 kB 9.9 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.4.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_cupti_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (13.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 10.5 MB/s eta 0:00:00
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.1) (9.1.0.70)
Collecting nvidia-cublas-cu12==12.4.2.65 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cublas_cu12-12.4.2.65-py3-none-manylinux2014_x86_64.whl (363.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 363.0/363.0 MB 10.1 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.2.0.44 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cufft_cu12-11.2.0.44-py3-none-manylinux2014_x86_64.whl (211.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 211.5/211.5 MB 10.4 MB/s eta 0:00:00
Collecting nvidia-curand-cu12==10.3.5.119 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_curand_cu12-10.3.5.119-py3-none-manylinux2014_x86_64.whl (56.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.3/56.3 MB 10.1 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.6.0.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cusolver_cu12-11.6.0.99-py3-none-manylinux2014_x86_64.whl (128.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.4/128.4 MB 10.4 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.3.0.142 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cusparse_cu12-12.3.0.142-py3-none-manylinux2014_x86_64.whl (207.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.5/207.5 MB 10.3 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.20.5 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 176.2/176.2 MB 10.3 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.4.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_nvtx_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (99 kB)
Collecting nvidia-nvjitlink-cu12==12.4.99 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/cu124/nvidia_nvjitlink_cu12-12.4.99-py3-none-manylinux2014_x86_64.whl (21.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.1/21.1 MB 10.5 MB/s eta 0:00:00
Collecting triton==3.0.0 (from torch==2.4.1)
  Downloading https://download.pytorch.org/whl/triton-3.0.0-1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (209.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.5/209.5 MB 10.3 MB/s eta 0:00:00
Requirement already satisfied: numpy in /home/ks/venv/lib/python3.12/site-packages (from torchvision==0.19.1) (1.26.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/ks/venv/lib/python3.12/site-packages (from torchvision==0.19.1) (11.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ks/venv/lib/python3.12/site-packages (from jinja2->torch==2.4.1) (3.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/ks/venv/lib/python3.12/site-packages (from sympy->torch==2.4.1) (1.3.0)
Installing collected packages: triton, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusparse-cu12, nvidia-cusolver-cu12, torch, torchvision, torchaudio
  Attempting uninstall: triton
    Found existing installation: triton 3.1.0
    Uninstalling triton-3.1.0:
      Successfully uninstalled triton-3.1.0
  Attempting uninstall: nvidia-nvtx-cu12
    Found existing installation: nvidia-nvtx-cu12 12.4.127
    Uninstalling nvidia-nvtx-cu12-12.4.127:
      Successfully uninstalled nvidia-nvtx-cu12-12.4.127
  Attempting uninstall: nvidia-nvjitlink-cu12
    Found existing installation: nvidia-nvjitlink-cu12 12.4.127
    Uninstalling nvidia-nvjitlink-cu12-12.4.127:
      Successfully uninstalled nvidia-nvjitlink-cu12-12.4.127
  Attempting uninstall: nvidia-nccl-cu12
    Found existing installation: nvidia-nccl-cu12 2.21.5
    Uninstalling nvidia-nccl-cu12-2.21.5:
      Successfully uninstalled nvidia-nccl-cu12-2.21.5
  Attempting uninstall: nvidia-curand-cu12
    Found existing installation: nvidia-curand-cu12 10.3.5.147
    Uninstalling nvidia-curand-cu12-10.3.5.147:
      Successfully uninstalled nvidia-curand-cu12-10.3.5.147
  Attempting uninstall: nvidia-cufft-cu12
    Found existing installation: nvidia-cufft-cu12 11.2.1.3
    Uninstalling nvidia-cufft-cu12-11.2.1.3:
      Successfully uninstalled nvidia-cufft-cu12-11.2.1.3
  Attempting uninstall: nvidia-cuda-runtime-cu12
    Found existing installation: nvidia-cuda-runtime-cu12 12.4.127
    Uninstalling nvidia-cuda-runtime-cu12-12.4.127:
      Successfully uninstalled nvidia-cuda-runtime-cu12-12.4.127
  Attempting uninstall: nvidia-cuda-nvrtc-cu12
    Found existing installation: nvidia-cuda-nvrtc-cu12 12.4.127
    Uninstalling nvidia-cuda-nvrtc-cu12-12.4.127:
      Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.4.127
  Attempting uninstall: nvidia-cuda-cupti-cu12
    Found existing installation: nvidia-cuda-cupti-cu12 12.4.127
    Uninstalling nvidia-cuda-cupti-cu12-12.4.127:
      Successfully uninstalled nvidia-cuda-cupti-cu12-12.4.127
  Attempting uninstall: nvidia-cublas-cu12
    Found existing installation: nvidia-cublas-cu12 12.4.5.8
    Uninstalling nvidia-cublas-cu12-12.4.5.8:
      Successfully uninstalled nvidia-cublas-cu12-12.4.5.8
  Attempting uninstall: nvidia-cusparse-cu12
    Found existing installation: nvidia-cusparse-cu12 12.3.1.170
    Uninstalling nvidia-cusparse-cu12-12.3.1.170:
      Successfully uninstalled nvidia-cusparse-cu12-12.3.1.170
  Attempting uninstall: nvidia-cusolver-cu12
    Found existing installation: nvidia-cusolver-cu12 11.6.1.9
    Uninstalling nvidia-cusolver-cu12-11.6.1.9:
      Successfully uninstalled nvidia-cusolver-cu12-11.6.1.9
  Attempting uninstall: torch
    Found existing installation: torch 2.5.0+cu124
    Uninstalling torch-2.5.0+cu124:
      Successfully uninstalled torch-2.5.0+cu124
  Attempting uninstall: torchvision
    Found existing installation: torchvision 0.20.0
    Uninstalling torchvision-0.20.0:
      Successfully uninstalled torchvision-0.20.0
  Attempting uninstall: torchaudio
    Found existing installation: torchaudio 2.5.0
    Uninstalling torchaudio-2.5.0:
      Successfully uninstalled torchaudio-2.5.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xformers 0.0.28.post2 requires torch==2.5.0, but you have torch 2.4.1+cu124 which is incompatible.
Successfully installed nvidia-cublas-cu12-12.4.2.65 nvidia-cuda-cupti-cu12-12.4.99 nvidia-cuda-nvrtc-cu12-12.4.99 nvidia-cuda-runtime-cu12-12.4.99 nvidia-cufft-cu12-11.2.0.44 nvidia-curand-cu12-10.3.5.119 nvidia-cusolver-cu12-11.6.0.99 nvidia-cusparse-cu12-12.3.0.142 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.4.99 nvidia-nvtx-cu12-12.4.99 torch-2.4.1+cu124 torchaudio-2.4.1+cu124 torchvision-0.19.1+cu124 triton-3.0.0

@kairin
Copy link
Author

kairin commented Oct 25, 2024

Buuuuutttttt.... it causes a different error instead...


Restarting... [Legacy Mode]


[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-10-25 23:04:34.520653
** Platform: Linux
** Python version: 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]
** Python executable: /home/ks/venv/bin/python
** ComfyUI Path: /home/ks/venv/comf
** Log path: /home/ks/venv/comf/comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.6 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager

Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.4.1+cu124
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.5.0+cu124 with CUDA 1204 (you have 2.4.1+cu124)
    Python  3.12.7 (you have 3.12.3)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
xformers version: 0.0.28.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync
Using pytorch cross attention
[Prompt Server] web root: /home/ks/venv/comf/web
/home/ks/venv/lib/python3.12/site-packages/kornia/feature/lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)
### Loading: ComfyUI-Manager (V2.51.8)
### ComfyUI Revision: 2785 [52810907] | Released on '2024-10-24'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
/home/ks/venv/lib/python3.12/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 1.4.20 (you have 0.0.19). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
[ReActor] - STATUS - Running v0.5.1-b2 in ComfyUI
Torch version: 2.4.1+cu124
Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.4.1+cu124
xformers version: 0.0.28.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync

[rgthree-comfy] Loaded 42 fantastic nodes. 🎉

WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `/home/ks/venv/comf/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 218 nodes successfully.

        "Art enables us to find ourselves and lose ourselves at the same time." - Thomas Merton


Import times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/ks/venv/comf/custom_nodes/Skimmed_CFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_JPS-Nodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-AutomaticCFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Image-Saver
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Custom-Scripts
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-KJNodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager
   0.1 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_ExtraModels
   0.3 seconds: /home/ks/venv/comf/custom_nodes/comfyui_face_parsing
   0.8 seconds: /home/ks/venv/comf/custom_nodes/comfyui-reactor-node
   0.8 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR
   1.1 seconds: /home/ks/venv/comf/custom_nodes/was-node-suite-comfyui

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager/extension-node-map.json [DONE]
got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 2448.5241737365723 True
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:38<00:00,  2.59it/s]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Loading weights to:  cpu
Diffusion using fp16
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Attempting to load SDXL model from node inputs
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 4897.0483474731445 True
Loading first clip model from SDXL checkpoint
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Loading second clip model from SDXL checkpoint
Attempting to load SUPIR model: [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Loaded state_dict from [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Encoder using bf16
[Tiled VAE]: input_size: torch.Size([1, 3, 2688, 1536]), tile_size: 512, padding: 32
[Tiled VAE]: split to 6x3 = 18 tiles. Optimal tile size 512x448, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████████████████████████| 1638/1638 [00:06<00:00, 235.35it/s]
[Tiled VAE]: Done in 7.515s, max VRAM alloc 664.499 MB
[Tiled VAE]: input_size: torch.Size([1, 4, 336, 192]), tile_size: 64, padding: 11
[Tiled VAE]: split to 5x3 = 15 tiles. Optimal tile size 64x64, original tile size 64x64
[Tiled VAE]: Executing Decoder Task Queue: 100%|████████████████████████████████████████████████████████| 1845/1845 [00:12<00:00, 142.10it/s]
[Tiled VAE]: Done in 13.520s, max VRAM alloc 1760.142 MB
captions:  [['']]
Batch captioning
Encoder using bf16
[Tiled VAE]: input_size: torch.Size([1, 3, 2688, 1536]), tile_size: 512, padding: 32
[Tiled VAE]: split to 6x3 = 18 tiles. Optimal tile size 512x448, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████████████████████████| 1638/1638 [00:04<00:00, 352.28it/s]
[Tiled VAE]: Done in 5.209s, max VRAM alloc 667.091 MB
sampler_config:  {'target': '.sgm.modules.diffusionmodules.sampling.RestoreDPMPP2MSampler', 'params': {'num_steps': 16, 'restore_cfg': -1, 's_churn': 5, 's_noise': 1.0030000000000001, 'discretization_config': {'target': '.sgm.modules.diffusionmodules.discretizer.LegacyDDPMDiscretization'}, 'guider_config': {'target': '.sgm.modules.diffusionmodules.guiders.LinearCFG', 'params': {'scale': 4.0, 'scale_min': 4.0}}, 'eta': 1.0}}
Using random noise
!!! Exception during processing !!! No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(40, 4032, 1, 64) (torch.float16)
     key         : shape=(40, 4032, 1, 64) (torch.float16)
     value       : shape=(40, 4032, 1, 64) (torch.float16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`[email protected]` is not supported because:
    xFormers wasn't build with CUDA support
`cutlassF` is not supported because:
    xFormers wasn't build with CUDA support
    operator wasn't built - see `python -m xformers.info` for more info
Traceback (most recent call last):
  File "/home/ks/venv/comf/execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)
  File "/home/ks/venv/comf/execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 504, in sample
    _samples = self.sampler(denoiser, noised_z, cond=positive[i], uc=negative[i], x_center=sample.unsqueeze(0), control_scale=control_scale_end,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 657, in __call__
    x, old_denoised = self.sampler_step(
                      ^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 609, in sampler_step
    denoised = self.denoise(x, denoiser, sigma, cond, uc, control_scale=control_scale)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 573, in denoise
    denoised = denoiser(*self.guider.prepare_inputs(x, sigma, cond, uc), control_scale)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 477, in <lambda>
    denoiser = lambda input, sigma, c, control_scale: SUPIR_model.denoiser(SUPIR_model.model, input, sigma, c, control_scale)
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/denoiser.py", line 73, in __call__
    return network(input * c_in, c_noise, cond, control_scale) * c_out + input * c_skip
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/wrappers.py", line 96, in forward
    out = self.diffusion_model(
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 657, in forward
    h = self.project_modules[adapter_idx](control[control_idx], h, control_scale=control_scale)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 150, in forward
    x = self.attn(x, context)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR/sgm/modules/attention.py", line 360, in forward
    out = xformers.ops.memory_efficient_attention(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/__init__.py", line 301, in memory_efficient_attention
    return _memory_efficient_attention(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/__init__.py", line 462, in _memory_efficient_attention
    return _memory_efficient_attention_forward(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/__init__.py", line 481, in _memory_efficient_attention_forward
    op = _dispatch_fw(inp, False)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/dispatch.py", line 135, in _dispatch_fw
    return _run_priority_list(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/dispatch.py", line 76, in _run_priority_list
    raise NotImplementedError(msg)
NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(40, 4032, 1, 64) (torch.float16)
     key         : shape=(40, 4032, 1, 64) (torch.float16)
     value       : shape=(40, 4032, 1, 64) (torch.float16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`[email protected]` is not supported because:
    xFormers wasn't build with CUDA support
`cutlassF` is not supported because:
    xFormers wasn't build with CUDA support
    operator wasn't built - see `python -m xformers.info` for more info

Prompt executed in 142.48 seconds

@kairin
Copy link
Author

kairin commented Oct 25, 2024

so i thought... okay... so something is wrong with the xformers... maybe i should go back an older version...

so i did...


  on    bash   ks 3.12.3    master ≡                                                                             3m 56s 161ms 
╭─ 23:02:43 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers <=0.028
-bash: =0.028: No such file or directory
  on    bash   ks 3.12.3    master ≡                                                                                      5ms 
╭─ 23:09:50 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers =<0.028
-bash: 0.028: No such file or directory
  on    bash   ks 3.12.3    master ≡                                                                                      4ms 
╭─ 23:09:58 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers =<0.0.28
-bash: 0.0.28: No such file or directory
  on    bash   ks 3.12.3    master ≡                                                                                      5ms 
╭─ 23:10:08 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers 0.0.27
Requirement already satisfied: xformers in /home/ks/venv/lib/python3.12/site-packages (0.0.28.post2)
ERROR: Could not find a version that satisfies the requirement 0.0.27 (from versions: none)
ERROR: No matching distribution found for 0.0.27
  on    bash   ks 3.12.3    master ≡                                                                                 8s 536ms 
╭─ 23:10:26 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip --help

Usage:
  pip <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.
  inspect                     Inspect the python environment.
  list                        List installed packages.
  show                        Show information about installed packages.
  check                       Verify installed packages have compatible dependencies.
  config                      Manage local and global configuration.
  search                      Search PyPI for packages.
  cache                       Inspect and manage pip's wheel cache.
  index                       Inspect information available from package indexes.
  wheel                       Build wheels from your requirements.
  hash                        Compute hashes of package archives.
  completion                  A helper command used for command completion.
  debug                       Show information useful for debugging.
  help                        Show help for commands.

General Options:
  -h, --help                  Show help.
  --debug                     Let unhandled exceptions propagate outside the main subroutine, instead of logging them to stderr.
  --isolated                  Run pip in an isolated mode, ignoring environment variables and user configuration.
  --require-virtualenv        Allow pip to only run in a virtual environment; exit with an error otherwise.
  --python <python>           Run pip with the specified Python interpreter.
  -v, --verbose               Give more output. Option is additive, and can be used up to 3 times.
  -V, --version               Show version and exit.
  -q, --quiet                 Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and
                              CRITICAL logging levels).
  --log <path>                Path to a verbose appending log.
  --no-input                  Disable prompting for input.
  --keyring-provider <keyring_provider>
                              Enable the credential lookup via the keyring library if user input is allowed. Specify which mechanism to use
                              [disabled, import, subprocess]. (default: disabled)
  --proxy <proxy>             Specify a proxy in the form scheme://[user:passwd@]proxy.server:port.
  --retries <retries>         Maximum number of retries each connection should attempt (default 5 times).
  --timeout <sec>             Set the socket timeout (default 15 seconds).
  --exists-action <action>    Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
  --trusted-host <hostname>   Mark this host or host:port pair as trusted, even though it does not have valid or any HTTPS.
  --cert <path>               Path to PEM-encoded CA certificate bundle. If provided, overrides the default. See 'SSL Certificate
                              Verification' in pip documentation for more information.
  --client-cert <path>        Path to SSL client certificate, a single file containing the private key and the certificate in PEM format.
  --cache-dir <dir>           Store the cache data in <dir>.
  --no-cache-dir              Disable the cache.
  --disable-pip-version-check
                              Don't periodically check PyPI to determine whether a new version of pip is available for download. Implied
                              with --no-index.
  --no-color                  Suppress colored output.
  --no-python-version-warning
                              Silence deprecation warnings for upcoming unsupported Pythons.
  --use-feature <feature>     Enable new functionality, that may be backward incompatible.
  --use-deprecated <feature>  Enable deprecated functionality, that will be removed in the future.
  on    bash   ks 3.12.3    master ≡                                                                                 7s 949ms 
╭─ 23:10:41 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip uninstall xformers
Found existing installation: xformers 0.0.28.post2
Uninstalling xformers-0.0.28.post2:
  Would remove:
    /home/ks/venv/lib/python3.12/site-packages/xformers-0.0.28.post2.dist-info/*
    /home/ks/venv/lib/python3.12/site-packages/xformers/*
Proceed (Y/n)? y
  Successfully uninstalled xformers-0.0.28.post2
  on    bash   ks 3.12.3    master ≡                                                                                  4s 41ms 
╭─ 23:11:01 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers 0.0.27
Collecting xformers
  Downloading xformers-0.0.28.post2-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (1.0 kB)
ERROR: Could not find a version that satisfies the requirement 0.0.27 (from versions: none)
ERROR: No matching distribution found for 0.0.27
  on    bash   ks 3.12.3    master ≡                                                                                    477ms 
╭─ 23:11:13 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers <0.0.28
-bash: 0.0.28: No such file or directory
  on    bash   ks 3.12.3    master ≡                                                                                      4ms 
╭─ 23:11:38 |  25 Oct, Friday |   in     venv  comf
╰─❯ pip install xformers==0.0.27.post2
Collecting xformers==0.0.27.post2
  Downloading xformers-0.0.27.post2-cp312-cp312-manylinux2014_x86_64.whl.metadata (1.0 kB)
Requirement already satisfied: numpy in /home/ks/venv/lib/python3.12/site-packages (from xformers==0.0.27.post2) (1.26.4)
Collecting torch==2.4.0 (from xformers==0.0.27.post2)
  Downloading torch-2.4.0-cp312-cp312-manylinux1_x86_64.whl.metadata (26 kB)
Requirement already satisfied: filelock in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (3.16.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (4.12.2)
Requirement already satisfied: sympy in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (1.13.1)
Requirement already satisfied: networkx in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (3.4.2)
Requirement already satisfied: jinja2 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (3.1.4)
Requirement already satisfied: fsspec in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (2024.10.0)
Requirement already satisfied: setuptools in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (75.2.0)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (9.1.0.70)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (2.20.5)
Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.4.0->xformers==0.0.27.post2)
  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)
Requirement already satisfied: triton==3.0.0 in /home/ks/venv/lib/python3.12/site-packages (from torch==2.4.0->xformers==0.0.27.post2) (3.0.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/ks/venv/lib/python3.12/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.4.0->xformers==0.0.27.post2) (12.4.99)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ks/venv/lib/python3.12/site-packages (from jinja2->torch==2.4.0->xformers==0.0.27.post2) (3.0.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/ks/venv/lib/python3.12/site-packages (from sympy->torch==2.4.0->xformers==0.0.27.post2) (1.3.0)
Downloading xformers-0.0.27.post2-cp312-cp312-manylinux2014_x86_64.whl (20.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.8/20.8 MB 10.5 MB/s eta 0:00:00
Downloading torch-2.4.0-cp312-cp312-manylinux1_x86_64.whl (797.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 797.2/797.2 MB 10.1 MB/s eta 0:00:00
Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
Installing collected packages: nvidia-nvtx-cu12, nvidia-cusparse-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusolver-cu12, torch, xformers
  Attempting uninstall: nvidia-nvtx-cu12
    Found existing installation: nvidia-nvtx-cu12 12.4.99
    Uninstalling nvidia-nvtx-cu12-12.4.99:
      Successfully uninstalled nvidia-nvtx-cu12-12.4.99
  Attempting uninstall: nvidia-cusparse-cu12
    Found existing installation: nvidia-cusparse-cu12 12.3.0.142
    Uninstalling nvidia-cusparse-cu12-12.3.0.142:
      Successfully uninstalled nvidia-cusparse-cu12-12.3.0.142
  Attempting uninstall: nvidia-curand-cu12
    Found existing installation: nvidia-curand-cu12 10.3.5.119
    Uninstalling nvidia-curand-cu12-10.3.5.119:
      Successfully uninstalled nvidia-curand-cu12-10.3.5.119
  Attempting uninstall: nvidia-cufft-cu12
    Found existing installation: nvidia-cufft-cu12 11.2.0.44
    Uninstalling nvidia-cufft-cu12-11.2.0.44:
      Successfully uninstalled nvidia-cufft-cu12-11.2.0.44
  Attempting uninstall: nvidia-cuda-runtime-cu12
    Found existing installation: nvidia-cuda-runtime-cu12 12.4.99
    Uninstalling nvidia-cuda-runtime-cu12-12.4.99:
      Successfully uninstalled nvidia-cuda-runtime-cu12-12.4.99
  Attempting uninstall: nvidia-cuda-nvrtc-cu12
    Found existing installation: nvidia-cuda-nvrtc-cu12 12.4.99
    Uninstalling nvidia-cuda-nvrtc-cu12-12.4.99:
      Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.4.99
  Attempting uninstall: nvidia-cuda-cupti-cu12
    Found existing installation: nvidia-cuda-cupti-cu12 12.4.99
    Uninstalling nvidia-cuda-cupti-cu12-12.4.99:
      Successfully uninstalled nvidia-cuda-cupti-cu12-12.4.99
  Attempting uninstall: nvidia-cublas-cu12
    Found existing installation: nvidia-cublas-cu12 12.4.2.65
    Uninstalling nvidia-cublas-cu12-12.4.2.65:
      Successfully uninstalled nvidia-cublas-cu12-12.4.2.65
  Attempting uninstall: nvidia-cusolver-cu12
    Found existing installation: nvidia-cusolver-cu12 11.6.0.99
    Uninstalling nvidia-cusolver-cu12-11.6.0.99:
      Successfully uninstalled nvidia-cusolver-cu12-11.6.0.99
  Attempting uninstall: torch
    Found existing installation: torch 2.4.1+cu124
    Uninstalling torch-2.4.1+cu124:
      Successfully uninstalled torch-2.4.1+cu124
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.4.1+cu124 requires torch==2.4.1, but you have torch 2.4.0 which is incompatible.
torchvision 0.19.1+cu124 requires torch==2.4.1, but you have torch 2.4.0 which is incompatible.
Successfully installed nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nvtx-cu12-12.1.105 torch-2.4.0 xformers-0.0.27.post2
  on    bash   ks 3.12.3    master ≡                                                                             1m 33s 481ms 
╭─ 23:14:12 |  25 Oct, Friday |   in     venv  comf
╰─❯

@kairin
Copy link
Author

kairin commented Oct 25, 2024

now i get this annoying error...

but it is related to audio... which means... i can live with it for now.

and my SUPIR workflow completed.


Restarting... [Legacy Mode]


[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2024-10-25 23:14:33.237226
** Platform: Linux
** Python version: 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]
** Python executable: /home/ks/venv/bin/python
** ComfyUI Path: /home/ks/venv/comf
** Log path: /home/ks/venv/comf/comfyui.log

Prestartup times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.5 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager

Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.4.0+cu121
/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/flash.py:211: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_fwd")
/home/ks/venv/lib/python3.12/site-packages/xformers/ops/fmha/flash.py:344: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_bwd")
xformers version: 0.0.27.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync
Using xformers cross attention
[Prompt Server] web root: /home/ks/venv/comf/web
/home/ks/venv/lib/python3.12/site-packages/kornia/feature/lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)
Traceback (most recent call last):
  File "/home/ks/venv/comf/nodes.py", line 2001, in load_custom_node
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/ks/venv/comf/comfy_extras/nodes_audio.py", line 1, in <module>
    import torchaudio
  File "/home/ks/venv/lib/python3.12/site-packages/torchaudio/__init__.py", line 2, in <module>
    from . import _extension  # noqa  # usort: skip
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ks/venv/lib/python3.12/site-packages/torchaudio/_extension/__init__.py", line 42, in <module>
    _check_cuda_version()
  File "/home/ks/venv/lib/python3.12/site-packages/torchaudio/_extension/utils.py", line 175, in _check_cuda_version
    raise RuntimeError(
RuntimeError: Detected that PyTorch and TorchAudio were compiled with different CUDA versions. PyTorch has CUDA version 12.1 whereas TorchAudio has CUDA version 12.4. Please install the TorchAudio version that matches your PyTorch version.

Cannot import /home/ks/venv/comf/comfy_extras/nodes_audio.py module for custom nodes: Detected that PyTorch and TorchAudio were compiled with different CUDA versions. PyTorch has CUDA version 12.1 whereas TorchAudio has CUDA version 12.4. Please install the TorchAudio version that matches your PyTorch version.
### Loading: ComfyUI-Manager (V2.51.8)
### ComfyUI Revision: 2785 [52810907] | Released on '2024-10-24'
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
/home/ks/venv/lib/python3.12/site-packages/albumentations/__init__.py:24: UserWarning: A new version of Albumentations is available: 1.4.20 (you have 0.0.19). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
  check_for_updates()
[ReActor] - STATUS - Running v0.5.1-b2 in ComfyUI
Torch version: 2.4.0+cu121
Total VRAM 16376 MB, total RAM 40071 MB
pytorch version: 2.4.0+cu121
xformers version: 0.0.27.post2
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 SUPER : cudaMallocAsync

[rgthree-comfy] Loaded 42 epic nodes. 🎉

WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `/home/ks/venv/comf/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 218 nodes successfully.

        "Art is the journey of a free soul." - Alev Oguz


Import times for custom nodes:
   0.0 seconds: /home/ks/venv/comf/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/ks/venv/comf/custom_nodes/Skimmed_CFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_JPS-Nodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-AutomaticCFG
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Image-Saver
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Custom-Scripts
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-KJNodes
   0.0 seconds: /home/ks/venv/comf/custom_nodes/rgthree-comfy
   0.0 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager
   0.1 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI_ExtraModels
   0.2 seconds: /home/ks/venv/comf/custom_nodes/comfyui_face_parsing
   0.8 seconds: /home/ks/venv/comf/custom_nodes/ComfyUI-SUPIR
   0.9 seconds: /home/ks/venv/comf/custom_nodes/comfyui-reactor-node
   1.0 seconds: /home/ks/venv/comf/custom_nodes/was-node-suite-comfyui

WARNING: some comfy_extras/ nodes did not import correctly. This may be because they are missing some dependencies.

IMPORT FAILED: nodes_audio.py

This issue might be caused by new missing dependencies added the last time you updated ComfyUI.
Please do a: pip install -r requirements.txt

Starting server

To see the GUI go to: http://127.0.0.1:8188
FETCH DATA from: /home/ks/venv/comf/custom_nodes/ComfyUI-Manager/extension-node-map.json [DONE]
got prompt
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 2448.5241737365723 True
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:40<00:00,  2.46it/s]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
model weight dtype torch.float16, manual cast: None
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
Model maximum sigma: 14.614640235900879 / Model minimum sigma: 0.029167160391807556
Loading weights to:  cpu
Diffusion using fp16
Using XFORMERS
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Attempting to load SDXL model from node inputs
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 4897.0483474731445 True
Loading first clip model from SDXL checkpoint
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Loading second clip model from SDXL checkpoint
Attempting to load SUPIR model: [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Loaded state_dict from [/home/ks/venv/comf/models/checkpoints/SUPIR_pruned/SUPIR-v0Q_fp16.safetensors]
Encoder using bf16
[Tiled VAE]: input_size: torch.Size([1, 3, 2688, 1536]), tile_size: 512, padding: 32
[Tiled VAE]: split to 6x3 = 18 tiles. Optimal tile size 512x448, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████████████████████████| 1638/1638 [00:06<00:00, 245.81it/s]
[Tiled VAE]: Done in 7.213s, max VRAM alloc 664.499 MB
[Tiled VAE]: input_size: torch.Size([1, 4, 336, 192]), tile_size: 64, padding: 11
[Tiled VAE]: split to 5x3 = 15 tiles. Optimal tile size 64x64, original tile size 64x64
[Tiled VAE]: Executing Decoder Task Queue: 100%|████████████████████████████████████████████████████████| 1845/1845 [00:12<00:00, 146.60it/s]
[Tiled VAE]: Done in 13.116s, max VRAM alloc 1760.142 MB
captions:  [['']]
Batch captioning
Encoder using bf16
[Tiled VAE]: input_size: torch.Size([1, 3, 2688, 1536]), tile_size: 512, padding: 32
[Tiled VAE]: split to 6x3 = 18 tiles. Optimal tile size 512x448, original tile size 512x512
[Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████████████████████████| 1638/1638 [00:04<00:00, 362.45it/s]
[Tiled VAE]: Done in 5.071s, max VRAM alloc 667.091 MB
sampler_config:  {'target': '.sgm.modules.diffusionmodules.sampling.RestoreDPMPP2MSampler', 'params': {'num_steps': 16, 'restore_cfg': -1, 's_churn': 5, 's_noise': 1.0030000000000001, 'discretization_config': {'target': '.sgm.modules.diffusionmodules.discretizer.LegacyDDPMDiscretization'}, 'guider_config': {'target': '.sgm.modules.diffusionmodules.guiders.LinearCFG', 'params': {'scale': 4.0, 'scale_min': 4.0}}, 'eta': 1.0}}
Using random noise
Sampled  1  of  1
Decoder using bf16
SUPIR decoder using torch.bfloat16
[Tiled VAE]: input_size: torch.Size([1, 4, 336, 192]), tile_size: 64, padding: 11
[Tiled VAE]: split to 5x3 = 15 tiles. Optimal tile size 64x64, original tile size 64x64
[Tiled VAE]: Executing Decoder Task Queue: 100%|████████████████████████████████████████████████████████| 1845/1845 [00:10<00:00, 169.13it/s]
[Tiled VAE]: Done in 11.450s, max VRAM alloc 5398.604 MB
Prompt executed in 178.14 seconds

@kairin

This comment was marked as outdated.

Sign in to view
@kairin
Copy link
Author

kairin commented Oct 27, 2024 •
edited
Loading

so i might as well update here since managed to solve it unintentionally because i felt my generations is too slow.

i was able to use back the workflows from above and also able to shave off 100 seconds from my current workflow.

#4923

@kairin kairin closed this as completed Oct 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
User Support A user needs help with something, probably not a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kairin