Extending MONAI from medical to biomedical

[jxchen01]

Hello All, When I look at the MONAI contribution guideline, I see that MONAI can only take contributions that are "medical-application specific". What if it is related to other microscopy images problem for biological research? How does everyone think about extending "medical-specific" to "biomedical-specific"? The underlying AI methods for "medical imaging" and "biomedical imaging" share a lot of similarity, but still have considerable difference. For example, the noise model in common light microscopes is very different from MR or CT. Another example is that it is common to see very 3D microscopy images have much more anisotropic dimensions than medical data. Also, the IO packages are different for bio data and for medical data.

My group has been using MONAI for quite a while, but mostly using the models and loss functions. Re-building something new for the microscopy imaging community is too expensive and mostly re-invent the wheels. I can see that there is a huge potential to easily extend MONAI to support not only medical-specific applications, but general biomedical-specific applications. This will create a huge value for the microscopy / bio imaging community.

As soon as there is a green light, my group would definitely wants to contribute to this bio imaging effort for MONAI.

Best,
Jianxu Chen

[Nic-Ma]

Hi @jxchen01 ,

Thanks very much for your great feedback and experiments with MONAI.
We are also extending our use cases in the medical DL domain, like detection, multi-modal, pathology, etc.
For example: https://github.com/Project-MONAI/tutorials/tree/master/pathology

Thanks.

[Nic-Ma]

@wyli @ericspod , I think maybe we can have a deeper discussion with @jxchen01 for the feature requests first?
And of course, welcome your contributions to project MONAI!

Thanks in advance.

[jxchen01]

dear @Nic-Ma , the extension you mentioned above is still more or less "medial". One example of AI for bioimaging is like https://www.nature.com/articles/s41592-018-0111-2. I just don't want to make any "out-of-scope" contribution, which cannot be merged anyway. I would be happy to discussion in depth, either here, or on MONAI slack, or even by video call.

[Nic-Ma]

CC @ericspod @wyli .

Thanks.

[wyli]

Thanks, I think that's very reasonable and interesting. We'll discuss this with the team and get back to you soon (Because of the Easter holidays we expect some delays here).

[ericspod]

Seconded, we're definitely open to other application areas from medical-allied fields that would find MONAI useful. Recent work has been done on whole slide imaging for example which represents some new extensions to support that sort of data.

[aylward]

It has been a long-standing, word-choice, unresolved debate regarding whether the term "medical" or the term "biomedical" is more all-encompassing or if they are disjoint sets or overlapping sets or application specific rather than modality specific terms or ... So, one option is to clarify what we mean by "medical", rather than saying we're moving into a "new area" or "adjacent field" when discussing biomedical imaging.

[bhashemian]

Hi @jxchen01,thank you very much for your message and your interest in MONAI. We are already extending MONAI to some biomedical imaging applications like digital pathology and microscopy but would love to hear more about your applications.

[jxchen01]

I made a demo of how I extend the image reading function in TorchIO to support bioimage data: https://github.com/MMV-Lab/demo_box/blob/main/extending_torchio_for_bioimage.ipynb

I hope this can be helpful for evaluating how difficult it could be to extend MONAI LoadImage function to support bio-formats.

[jxchen01]

As requested from MONAI dev team, I attach the key points of what I presented in the dev meeting:

Five major potential directions:

1. I/O support for bio-formats with aicsimageio. See example above

2. Add more "build blocks" for bioimaging AI:

• pre-processing: e.g., adjusted NormalizeIntensity, see [1]
• augmentation: e.g., crappification, see [2]
• post-processing: e.g., clustering for instance segmentation, see [3]
• models: e.g., cellpose [4], anisotropic UNet [5], etc.

3. Microscopy datasets, Applications and Tutorials: e.g., see [6]

4. Visualization and annotation (MONAI Label) for bioimages:

• nbvv (comparable to itkwidgets)
• Napari (comparable to 3DSlicer)

5. interoperability: MONAI deploy for bioimage analysis workflows (e.g., BIAFlow or Galaxy project)

References:
[1] Ounkomol, Chawin et al. “Label-free prediction of three-dimensional fluorescence images from transmitted light microscopy.” Nature methods 15 (2018): 917 - 920.
[2] Fang, Linjing et al. “Deep Learning-Based Point-Scanning Super-Resolution Imaging.” Nature methods 18 (2021): 406 - 416.
[3] Lalit, Manan et al. “Embedding-based Instance Segmentation in Microscopy.” MIDL (2021).
[4] Stringer, Carsen et al. “Cellpose: a generalist algorithm for cellular segmentation.” Nature Methods 18 (2020): 100-106.
[5] Chen, Jianxu et al. “The Allen Cell and Structure Segmenter: a new open source toolkit for segmenting 3D intracellular structures in fluorescence microscopy images.” bioRxiv (2020).
[6] Edlund, Christoffer et al. “LIVECell—A large-scale dataset for label-free live cell segmentation.” Nature Methods 18 (2021): 1038 - 1045.