Nvahmadi/3dregistration learn2reg

[nvahmadi]

Description

Added MONAI tutorial for paired lung-CT 3D registration with keypoint matching (Learn2Reg 2023, NLST task), in collaboration with @brudfors. This is a modified version of the of the training code for the Learn2Reg-NLST submission by @brudfors (currently rank 14/111) in the challenge (see leaderboard).

Contributions beyond current tutorial

Compared to the current tutorial (paired_lung_ct.ipynb), this tutorial uses data from the current NLST task of the Learn2Reg 2023 challenge. The solution was developed in close alignment with the challenge organizer Mattias Heinrich (@mattiaspaul on GitHub) and collaborators, who will create a template test submission based on this solution. Together, these tutorials will serve as a MONAI-native guide on how to participate this year and in upoming iterations of Learn2Reg .

Beyond challenge participation, the tutorial introduces a new datatype to MONAI (pointclouds) and demonstrates how easy it is to write custom transforms for them (I/O and linear transforms). It also shows how to write a custom multi-target loss function for registration that uses loss components from MONAI/PyTorch/custom-code. This composite loss can be flexibly used in both unsupervised (image-based) registration scenarios, as well as supervised registration tasks (i.e. with auxiliary keypoints and/or labelmaps).

Checks

• [ X ] Avoid including large-size files in the PR.
• [ X ] Clean up long text outputs from code cells in the notebook.
• [ X ] For security purposes, please check the contents and remove any sensitive info such as user names and private key.
• [ X ] Ensure (1) hyperlinks and markdown anchors are working (2) use relative paths for tutorial repo files (3) put figure and graphs in the ./figure folder
• [ X ] Notebook runs automatically ./runner.sh -t <path to .ipynb file>. Note: The execution takes very long (approx. 50 mins, output of runner.sh below:

Executing: 100%|█████████████████████████████████████████████████████████████████████| 37/37 [49:42<00:00, 80.61s/cell]
/home/ahmad/.local/lib/python3.8/site-packages/papermill/iorw.py:153: UserWarning: the file is not specified with any extension : -
  warnings.warn(

real    49m46.070s
user    10m25.173s
sys     4m0.110s

Testing finished. 0 of 1 executed tests passed!

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[wyli]

thanks @nvahmadi @brudfors, this looks great, this PR is a good reminder that we should consider adding the point transform and collate fn to the core (Project-MONAI/MONAI#4027) in the future.

also please have a look at the minor style issues:

• https://github.com/Project-MONAI/tutorials/actions/runs/6124451978/job/16624498264?pr=1513

NotImplementedError                       Traceback (most recent call last)
Input In [8], in <cell line: 52>()
     50 ax.scatter(moving_points[:, 0], moving_points[:, 1], moving_points[:, 2], s=10.0, marker="o", color="orange")
     51 ax.view_init(-10, 80)
---> 52 ax.set_aspect("equal")
     53 plt.title("Pointcloud visualizations")
     54 plt.show()

File /opt/conda/lib/python3.8/site-packages/mpl_toolkits/mplot3d/axes3d.py:311, in Axes3D.set_aspect(self, aspect, adjustable, anchor, share)
    258 """
    259 Set the aspect ratios.
    260 
   (...)
    308 mpl_toolkits.mplot3d.axes3d.Axes3D.set_box_aspect
    309 """
    310 if aspect != 'auto':
--> 311     raise NotImplementedError(
    312         "Axes3D currently only supports the aspect argument "
    313         f"'auto'. You passed in {aspect!r}."
    314     )
    315 super().set_aspect(
    316     aspect, adjustable=adjustable, anchor=anchor, share=share)

NotImplementedError: Axes3D currently only supports the aspect argument 'auto'. You passed in 'equal'.

• https://github.com/Project-MONAI/tutorials/actions/runs/6124451974/job/16624494374?pr=1513

stdin:83:2: N812 lowercase 'torch.nn.functional' imported as non lowercase 'F'
import torch.nn.functional as F
 ^
stdin:88:1: F401 'monai.apps.download_url' imported but unused
from monai.apps import download_url, download_and_extract
^
stdin:481:121: E501 line too long (124 > 120 characters)
...

[brudfors]

@wyli @nvahmadi Thanks for the feedback! Happy to help with Core integration if need be.

[nvahmadi]

Thanks @wyli - I just pushed changes to fix the minor style issues. A few notes:

---> 52 ax.set_aspect("equal")-->NotImplementedError`

The build setup uses a slightly older version of matplotlib (v3.5.1), the feature axes3d.set_aspect('equal') was added in v3.6.0 (current mpl stable version is 3.7.x). I still changed the keyword equal to auto, as required by v3.5.1.

stdin:83:2: N812

import torch.nn.functional as F is an official PyTorch convention, better not to change it even if styleguide recommends that

stdin:88:1: F401

monai.apps.download_url removed

stdin:481:121:

Added line break / multiline string

[wyli]

thanks @nvahmadi, let me try to modify the config to ignore N812, there are a few other style issues, I'll try to fix them.

[wyli]

thanks, it looks good to me, making the papermill test skipped as it took too much time

[nvahmadi]

Awesome, thanks @wyli for the review, the fixes and for approving! Great to see this added!