Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem generating 3D after point cloud #12

Open
throttlekitty opened this issue Apr 10, 2023 · 2 comments
Open

Problem generating 3D after point cloud #12

throttlekitty opened this issue Apr 10, 2023 · 2 comments

Comments

@throttlekitty
Copy link

throttlekitty commented Apr 10, 2023

Running local on Win10 I'm getting the following error when generating 3D after creating the point cloud. This happens at some point during the lora process. I did initially have some problems getting PyTorch3D set up, and I did have to create the conda environment in 3.9, this repo suggests 3.8, in case that's a problem here. I see that ultimately it fails to find this particular image, as it doesn't exist.

...
Collapsing Lin Lora in out_proj
Collapsing Lin Lora in k_proj
Collapsing Lin Lora in v_proj
Collapsing Lin Lora in q_proj
Collapsing Lin Lora in out_proj
Reshaping encoder.mid.attn_1.q.weight for SD format
Reshaping encoder.mid.attn_1.k.weight for SD format
Reshaping encoder.mid.attn_1.v.weight for SD format
Reshaping encoder.mid.attn_1.proj_out.weight for SD format
Reshaping decoder.mid.attn_1.q.weight for SD format
Reshaping decoder.mid.attn_1.k.weight for SD format
Reshaping decoder.mid.attn_1.v.weight for SD format
Reshaping decoder.mid.attn_1.proj_out.weight for SD format
Added Token: ['<0>']
a fuzzy <0>
side view of a fuzzy <0>:   2%|█                                                   | 199/10000 [00:45<37:02,  4.41it/s]
Traceback (most recent call last):
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\routes.py", line 393, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\blocks.py", line 1108, in process_api
    result = await self.call_function(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\blocks.py", line 929, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\utils.py", line 490, in async_iteration
    return next(iterator)
  File "C:\stable-diffusion\3DFuse\gradio_app.py", line 47, in gen_3d
    yield from model.run_gradio(points,exp_instance_dir)
  File "C:\stable-diffusion\3DFuse\run_3DFuse.py", line 147, in run_gradio
    yield from fuse_3d(**cfgs, poser=poser,model=model,vox=vox,exp_instance_dir=exp_instance_dir, points=points, is_gradio=True)
  File "C:\stable-diffusion\3DFuse\run_3DFuse.py", line 248, in fuse_3d
    vis_routine(metric, y, depth,p,depth_map[0])
  File "C:\stable-diffusion\3DFuse\run_3DFuse.py", line 345, in vis_routine
    metric.put_artifact("img", ".png",prompt, lambda fn: imwrite(fn, im))
  File "C:\stable-diffusion\3DFuse\my\utils\event.py", line 110, in put_artifact
    save_func(fname)
  File "C:\stable-diffusion\3DFuse\run_3DFuse.py", line 345, in <lambda>
    metric.put_artifact("img", ".png",prompt, lambda fn: imwrite(fn, im))
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\core\functions.py", line 303, in imwrite
    writer = get_writer(uri, format, "i", **kwargs)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\core\functions.py", line 231, in get_writer
    return format.get_writer(request)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\core\format.py", line 185, in get_writer
    return self.Writer(self, request)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\core\format.py", line 221, in __init__
    self._open(**self.request.kwargs.copy())
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\plugins\pillow.py", line 356, in _open
    PillowFormat.Writer._open(self)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\plugins\pillow.py", line 195, in _open
    self._fp = self.request.get_file()
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\imageio\core\request.py", line 331, in get_file
    self._file = open(self.filename, "wb")
OSError: [Errno 22] Invalid argument: 'C:\\stable-diffusion\\3DFuse\\results\\cat_0000\\3d\\img\\step_199_backside_view_of_a_fuzzy_<0>.png'
@j0seo
Copy link
Collaborator

j0seo commented Apr 11, 2023

Hi, @throttlekitty Thanks for your interest in our work. It seems an error occurs in the part where intermediate results are saved as images in that environment. I cannot currently reproduce the Windows environment, so could you try to erase the lines starting with "metric.put_artifact" in the 'run_3DFuse.py' file and see if it works well?

@throttlekitty
Copy link
Author

That ran for a lot longer than before and leads to a similar error with different path formatting now.:

...
Reshaping decoder.mid.attn_1.v.weight for SD format
Reshaping decoder.mid.attn_1.proj_out.weight for SD format
Added Token: ['<0>']
a comfortable <0>
  0%|                                                                                        | 0/10000 [00:00<?, ?it/s]C:\apps\anaconda3\envs\3DFuse\lib\site-packages\torch\nn\functional.py:4227: UserWarning:

Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.

100%|████████████████████████████████████████████████████████████████████████████████| 100/100 [00:13<00:00,  7.67it/s]
Traceback (most recent call last):███████████████████████████████████████████████████| 100/100 [00:13<00:00,  7.68it/s]
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\routes.py", line 393, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\blocks.py", line 1111, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\blocks.py", line 1036, in postprocess_data
    prediction_value = postprocess_update_dict(
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\blocks.py", line 432, in postprocess_update_dict
    prediction_value["value"] = block.postprocess(prediction_value["value"])
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\components.py", line 2152, in postprocess
    y = self.make_temp_copy_if_needed(y)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\components.py", line 245, in make_temp_copy_if_needed
    temp_dir = self.hash_file(file_path)
  File "C:\apps\anaconda3\envs\3DFuse\lib\site-packages\gradio\components.py", line 217, in hash_file
    with open(file_path, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: './results\\bed_0000\\3d/result_10000/video/step_100_.mp4'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants