Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mac m2: NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook emplo #25

Open
i18nsite opened this issue Jan 28, 2024 · 3 comments
Open

mac m2: NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook emplo #25

i18nsite opened this issue Jan 28, 2024 · 3 comments

Comments

@i18nsite
Copy link

i18nsite commented Jan 28, 2024 •
edited
Loading

image 
2024-01-28 16:34:25,167 - ControlNet - INFO - ControlNet model control_sd15_inpaint_depth_hand_fp16 [09456e54] loaded.
2024-01-28 16:34:25,196 - ControlNet - INFO - Using preprocessor: depth_hand_refiner
2024-01-28 16:34:25,197 - ControlNet - INFO - preprocessor resolution = 744
set os.environ[OMP_NUM_THREADS] to 4
街照-744-1200.flow
{'state-ext-adetailer-txt2img_0-textarea_1': 'paintings, sketches, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, u
gly, mutation, mangled, gross proportion, out of focus, mutilated hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-ext-adetailer-txt2img-toggled': 'true', 'state-ext-control-net-txt2img_0-model': 'ip-adapt
er-faceid-plusv2_sdxl [187cb962]', 'state-txt2img_seed': '3777285790', 'state-ext-adetailer-txt2img_1-use-separate-noise-multiplier-2nd': 'true', 'state-ext-dynamic-prompts-txt2img-magic-prompt': 'true', 'state-ext-adetailer-txt2img_1-use-separate-vae-2nd': 'true', 'state-ext-adetailer-t
xt2img_0-use-separate-checkpoint': 'true', 'state-ext-control-net-txt2img-tab': 'ControlNet Unit 0 ', 'state-txt2img_sampling': 'DPM++ SDE Karras', 'state-ext-adetailer-txt2img_1-use-separate-cfg-scale-2nd': 'true', 'state-txt2img_batch_count': '1', 'state-ext-adetailer-txt2img_0-use-sep
arate-sampler': 'true', 'state-setting_sd_model_model': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-dynamic-prompts-txt2img-combinatorial-generation': 'false', 'state-ext-adetailer-txt2img-enable-adetailer': 'true', 'state-ext-adetailer-txt2img_1-textarea_2': 'B
est quality, masterpiece, ultra high res, photorealistic, raw photo,\n1gril,look at viewer,\ngirl face, sexy lip,perfect eyes,\nlong hair,high bangs,\nyangmi,\n <lora:beautiful_girl_ver2:0.3>,\n <lora:YangmiV81:0.6>, \n<lora:AIGirlXL_V1:0.3>,\n <lora:RMSDXL_Enhance:1>,\n', 'state-txt2img
_batch_size': '1', 'state-tab': 'txt2img', 'state-txt2img_#script_list': 'None', 'state-ext-control-net-txt2img_0-control-weight': '0.5', 'state-ext-dynamic-prompts-txt2img-toggled': 'false', 'state-setting_sd_model_checkpoints': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]',
 'state-ext-adetailer-txt2img_1-use-separate-checkpoint-2nd': 'true', 'state-ext-adetailer-txt2img_1-use-separate-clip-skip-2nd': 'false', 'state-txt2img_cfg_scale': '1.5', 'state-txt2img_height': '1200', 'state-ext-adetailer-txt2img_1-use-separate-sampler-2nd': 'true', 'state-txt2img_st
eps': '7', 'state-ext-adetailer-txt2img_1-adetailer-steps-2nd': '15', 'state-ext-adetailer-txt2img_0-use-separate-vae': 'true', 'state-ext-adetailer-txt2img_0-inpaint-denoising-strength': '0.53', 'state-ext-control-net-txt2img_0-checkpoints': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', '
state-ext-adetailer-txt2img_1-adetailer-model-2nd': 'hand_yolov8n.pt', 'state-ext-adetailer-txt2img_1-use-separate-steps-2nd': 'true', 'state-setting_sd_vae': 'Automatic', 'state-ext-adetailer-txt2img_1-inpaint-denoising-strength-2nd': '0.5', 'state-txt2img_neg_prompt': 'paintings, sketc
hes, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, ugly, mutation, mangled, gross proportion, out of focus, mutila
ted hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-setting_sd_model_models': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-adetailer-txt2img_0-use-separate-steps': 'true', 'state-
ext-control-net-txt2img_0-preprocessor': 'ip-adapter_face_id_plus', 'state-ext-adetailer-txt2img_1-controlnet-model-2nd': 'control_sd15_inpaint_depth_hand_fp16 [09456e54]', 'state-ext-adetailer-txt2img_0-textarea_0': 'nice hands, perfect hands, girl hands,detailed hand,Hand close-up\n',
'state-ext-adetailer-txt2img_1-adetailer-checkpoint-2nd': 'dreamshaper_8.safetensors', 'state-txt2img_width': '744', 'state-ext-control-net-txt2img_0-models': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', 'state-ext-adetailer-txt2img_1-controlnet-module-2nd': 'depth_midas', 'state-ext-adet
ailer-txt2img_1-textarea_3': 'paintings, sketches, (worst quality), (low quality), (normal quality), lowres, 2 heads, elongated body, 2 faces, cropped image, out of frame, draft, deformed hands, signature, watermark, twisted fingers, double image, long neck, extra limb, ugly, mutation, m
angled, gross proportion, out of focus, mutilated hands, missing arms, mutated hands, missing legs, mutated legs, disconnected limbs, bad anatomy, cloned face, cloned eyes,', 'state-ext-control-net-txt2img_0-checkpoint': 'ip-adapter-faceid-plusv2_sdxl [187cb962]', 'state-ext-adetailer-tx
t2img-tab': '2nd ', 'state-setting_sd_model_checkpoint': 'dreamshaperXL_sfwTurboDpmppSDE.safetensors [3a22504d8a]', 'state-ext-adetailer-txt2img_1-adetailer-sampler-2nd': 'Restart', 'state-ext-control-net-txt2img-toggled': 'true', 'state-ext-control-net-txt2img_0-enable': 'true', 'state-
txt2img_prompt': 'Best quality, masterpiece, ultra high res, photorealistic, raw photo,\nwave hand, nice hand,arm up, detailed hand,\n full shot , Volume light,\n1gril, sexy pose,\n{glasses|coloured pupil|Watery|black|blue} eyes,\n{dimple|smile|sad|shy|anger|weep} face,\nblack hair,\ngir
l face, sexy lip,perfect eyes,\nlong hair,high bangs,\nyangmi,\n <lora:beautiful_girl_ver2:0.3>,\n <lora:YangmiV81:0.6>, \n<lora:AIGirlXL_V1:0.3>,\n <lora:RMSDXL_Enhance:1>,\nlook at viewer,\n{police uniform|Cycling Jersey and Shorts|Flight Attendant Uniform| Multi Pockets Jumpsuit Mecha
nic Uniform|Doctor‘s White Coat|Tennis Dress|Maid outfit| Golf Polo Shirt and Pants|Construction Worker’s Safety Vest|wedding dress}\n\n', 'state-ext-adetailer-txt2img_1-adetailer-cfg-scale-2nd': '2'}
using_loras = ['beautiful_girl_ver2', 'YangmiV81', 'AIGirlXL_V1', 'RMSDXL_Enhance', 'beautiful_girl_ver2', 'YangmiV81', 'AIGirlXL_V1', 'RMSDXL_Enhance']
/Users/z/git/stable-diffusion-webui/models/Lora/beautiful_girl_ver2.safetensors
Use Memory Optimized SHA256
/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/scipy/sparse/_index.py:102: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
  self._set_intXint(row, col, x.flat[0])
*** Error running process: /Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
    Traceback (most recent call last):
      File "/Users/z/git/stable-diffusion-webui/modules/scripts.py", line 718, in process
        script.process(p, *script_args)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 1138, in process
        self.controlnet_hack(p)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 1123, in controlnet_hack
        self.controlnet_main_entry(p)
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 930, in controlnet_main_entry
        detected_map, is_image = self.preprocessor[unit.module](
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 80, in decorated_func
        return cached_func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 64, in cached_func
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 858, in run_model
        self.load_model()
      File "/Users/z/git/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 844, in load_model
        self.model = MeshGraphormerDetector.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/hand_refiner/__init__.py", line 17, in from_pretrained
        pipeline = MeshGraphormerMediapipe(args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/hand_refiner/pipeline.py", line 76, in __init__
        mesh_sampler = Mesh(device=args.device)
                       ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/mesh_graphormer/modeling/_mano.py", line 124, in __init__
        self._U = [u.to(device) for u in self._U]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/z/git/stable-diffusion-webui/venv/lib/python3.11/site-packages/mesh_graphormer/modeling/_mano.py", line 124, in <listcomp>
        self._U = [u.to(device) for u in self._U]
                   ^^^^^^^^^^^^
    NotImplementedError: Could not run 'aten::empty.memory_format' with arguments from the 'SparseMPS' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook emplo
yee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::empty.memory_format' is only available for these backends: [CPU, MPS, Meta, QuantizedCPU, QuantizedMeta, MkldnnCPU, SparseCPU, SparseMeta, SparseCsrCPU, BackendSelect, Python, FuncTorchD
ynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1,
AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

    CPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterCPU.cpp:31034 [kernel]
    MPS: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterMPS.cpp:22748 [kernel]
    Meta: registered at /dev/null:241 [kernel]
    QuantizedCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterQuantizedCPU.cpp:929 [kernel]
    QuantizedMeta: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterQuantizedMeta.cpp:105 [kernel]
    MkldnnCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterMkldnnCPU.cpp:507 [kernel]
    SparseCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseCPU.cpp:1379 [kernel]
    SparseMeta: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseMeta.cpp:249 [kernel]
    SparseCsrCPU: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterSparseCsrCPU.cpp:1128 [kernel]
    BackendSelect: registered at /Users/runner/work/pytorch/pytorch/pytorch/build/aten/src/ATen/RegisterBackendSelect.cpp:726 [kernel]
    Python: registered at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/core/PythonFallbackKernel.cpp:144 [backend fallback]
    FuncTorchDynamicLayerBackMode: registered at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/functorch/DynamicLayer.cpp:491 [backend fallback]
@kavinbj
Copy link

kavinbj commented Jan 31, 2024

same in mac m1pro, maybe not support MPS

@misato2099
Copy link

Same problem here. Can Macbook install the hand refiner directly instead of using WEBUI contrlnet?

@michaeldlutlee1
Copy link

same error in hand refiner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@kavinbj @misato2099 @i18nsite @michaeldlutlee1