Pretrained nnunet in monai label + 3D slicer #1277

sofisappia · 2023-02-06T09:48:10Z

sofisappia
Feb 6, 2023

Hi! I am trying to use the monai label plugin in 3D slicer in combination with a pretrained nnunet model for a semantic segmentation task. I am trying to build a custom app, following these steps to adjust the radiology app. I see that the dynunet architecture was developed based on nnunet, and that it is used as building block of deepedit. I am taking inspiration from deepedit and segmentation scripts in the radiology app. However, I am a bit lost as to what adjustments I should make and was wondering someone has already worked on something similar and has some guidelines to share.
Thanks!

SachidanandAlle · 2023-02-06T09:49:36Z

SachidanandAlle
Feb 6, 2023
Maintainer

Hi! I am trying to use the monai label plugin in 3D slicer in combination with a pretrained nnunet model for a semantic segmentation task. I am trying to build a custom app, following these steps to adjust the radiology app. I see that the dynunet architecture was developed based on nnunet, and that it is used as building block of deepedit. I am taking inspiration from deepedit and segmentation scripts in the radiology app. However, I am a bit lost as to what adjustments I should make and was wondering someone has already worked on something similar and has some guidelines to share.
Thanks!

Please share your GitHub code.. it's easier to help or suggest based on that

2 replies

sofisappia Feb 7, 2023
Author

Thank @SachidanandAlle for your response. I added the code on this github repository. The scripts that I adapted for my model are called dynunet_placenta.py.

I am now trying to run inference with it through the plugin on 3D slicer, but I get a CUDA out of memory error. I had previously had this with a different model for training, and solved it using the PersistentDataset. How can I solve this for inference?

Any comments/suggestions on the code I adjusted?

Thanks!

SachidanandAlle Feb 7, 2023
Maintainer

I am now trying to run inference with it through the plugin on 3D slicer, but I get a CUDA out of memory error.
What GPU you have. You can share nvidia-smi output.
Also share the logs for the server (from the time you make the infer request). What is the size of your input image?

sofisappia · 2023-02-07T23:27:52Z

sofisappia
Feb 7, 2023
Author

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1460 C ...nailabel_clone\python.exe N/A |
+-----------------------------------------------------------------------------+

Size of input images: num_frames x 562 x 744; where num_frames is approximately 100, and varies per volume.

Server logs from the time of infer request:
[2023-02-07 18:05:41,536] [1460] [MainThread] [INFO] (monailabel.endpoints.infer:160) - Infer Request: {'model': 'dynunet_placenta', 'image': '20161018-002_2_0000', 'device': 'cuda', 'largest_cc': True, 'result_extension': '.nrrd', 'result_dtype': 'uint8', 'client_id': 'user-xyz'}
[2023-02-07 18:05:41,538] [1460] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:258) - Infer Request (final): {'device': 'cuda', 'largest_cc': True, 'model': 'dynunet_placenta', 'image': 'C:\Users\Sofia Sappia\Documents\monai_test\datasets\Task431_Placenta\20161018-002_2_0000.nii.gz', 'result_extension': '.nrrd', 'result_dtype': 'uint8', 'client_id': 'user-xyz', 'description': 'A pre-trained model for volumetric (3D) segmentation over 3D Images'}
[2023-02-07 18:05:41,539] [1460] [MainThread] [INFO] (monailabel.interfaces.utils.transform:63) - PRE - Run Transform(s)
[2023-02-07 18:05:41,540] [1460] [MainThread] [INFO] (monailabel.interfaces.utils.transform:64) - PRE - Input Keys: ['device', 'largest_cc', 'model', 'image', 'result_extension', 'result_dtype', 'client_id', 'description', 'image_path']
[2023-02-07 18:05:44,139] [1460] [MainThread] [INFO] (monailabel.interfaces.utils.transform:109) - PRE - Transform (LoadImaged): Time: 2.5978; image: (744, 562, 115)(torch.float32)
[2023-02-07 18:05:44,970] [1460] [MainThread] [INFO] (monailabel.interfaces.utils.transform:109) - PRE - Transform (EnsureTyped): Time: 0.8299; image: (744, 562, 115)(torch.float32)
[2023-02-07 18:05:44,971] [1460] [MainThread] [INFO] (monailabel.interfaces.utils.transform:109) - PRE - Transform (EnsureChannelFirstd): Time: 0.0; image: (1, 744, 562, 115)(torch.float32)
[2023-02-07 18:05:48,710] [1460] [MainThread] [ERROR] (uvicorn.error:369) - Exception in ASGI application
Traceback (most recent call last):
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 366, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 75, in call
return await self.app(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\fastapi\applications.py", line 269, in call
await super().call(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\applications.py", line 124, in call
await self.middleware_stack(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\middleware\errors.py", line 184, in call
raise exc
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\middleware\errors.py", line 162, in call
await self.app(scope, receive, _send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\middleware\cors.py", line 84, in call
await self.app(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\exceptions.py", line 93, in call
raise exc
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\exceptions.py", line 82, in call
await self.app(scope, receive, sender)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in call
raise e
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\routing.py", line 670, in call
await route.handle(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\routing.py", line 266, in handle
await self.app(scope, receive, send)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\starlette\routing.py", line 65, in app
response = await func(request)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\fastapi\routing.py", line 227, in app
raw_response = await run_endpoint_function(
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\fastapi\routing.py", line 160, in run_endpoint_function
return await dependant.call(**values)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\endpoints\infer.py", line 179, in api_run_inference
return run_inference(background_tasks, model, image, session_id, params, file, label, output)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\endpoints\infer.py", line 161, in run_inference
result = instance.infer(request)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\interfaces\app.py", line 299, in infer
result_file_name, result_json = task(request)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\tasks\infer\basic_infer.py", line 267, in call
data = self.run_pre_transforms(data, pre_transforms)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\tasks\infer\basic_infer.py", line 343, in run_pre_transforms
return run_transforms(data, transforms, log_prefix="PRE", use_compose=False)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monailabel\interfaces\utils\transform.py", line 93, in run_transforms
data = t(data)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\transforms\spatial\dictionary.py", line 416, in call
d[key] = self.spacing_transform(
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\utils\deprecate_utils.py", line 221, in _wrapper
return func(*args, **kwargs)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\transforms\spatial\array.py", line 603, in call
data_array = self.sp_resample(
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\utils\deprecate_utils.py", line 221, in _wrapper
return func(*args, **kwargs)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\transforms\spatial\array.py", line 315, in call
img = affine_xform(img.unsqueeze(0), theta=xform, spatial_size=spatial_size).squeeze(0)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\monai\networks\layers\spatial_transforms.py", line 575, in forward
grid = nn.functional.affine_grid(theta=theta[:, :sr], size=list(dst_size), align_corners=self.align_corners)
File "C:\Users\Sofia Sappia\miniconda3\envs\monailabel_clone\lib\site-packages\torch\nn\functional.py", line 4332, in affine_grid
return torch.affine_grid_generator(theta, size, align_corners)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 33.50 GiB (GPU 0; 4.00 GiB total capacity; 723.11 MiB already allocated; 1.70 GiB free; 746.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

3 replies

SachidanandAlle Feb 7, 2023
Maintainer

4GB is small against (744, 562, 115) image.. atleast 8GB is recommended

sarbabi Mar 13, 2023

Were you able to use nnUNet in MONAI Label?

dominiklangner23 Mar 23, 2023

Hi, I would also be interested to know if the implementation of nnUNet in MONAI Label worked out

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pretrained nnunet in monai label + 3D slicer #1277

{{title}}

Replies: 2 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Pretrained nnunet in monai label + 3D slicer #1277

sofisappia Feb 6, 2023

Replies: 2 comments · 5 replies

SachidanandAlle Feb 6, 2023 Maintainer

sofisappia Feb 7, 2023 Author

SachidanandAlle Feb 7, 2023 Maintainer

sofisappia Feb 7, 2023 Author

SachidanandAlle Feb 7, 2023 Maintainer

sarbabi Mar 13, 2023

dominiklangner23 Mar 23, 2023

sofisappia
Feb 6, 2023

Replies: 2 comments 5 replies

SachidanandAlle
Feb 6, 2023
Maintainer

sofisappia Feb 7, 2023
Author

SachidanandAlle Feb 7, 2023
Maintainer

sofisappia
Feb 7, 2023
Author

SachidanandAlle Feb 7, 2023
Maintainer