Custom annotation of open heart surgery videos - any best practices in MONAILabel?

[bilalcodehub]

Hi All,

Firstly, I'm new to MONAILabel so apologies for the naive questions.

I want to use monai endoscopic application for labelling open heart surgery videos. I think this is the closest one can get in monailabel for this task. Is that correct?

I want to utilise built-in tooltracking functionality and pre-trained models that come with the endoscopic app. There are eighteen surgical tools used in the procedure. I was able to copy key video frames into the datasets folder.
Can someone guide me what is the best strategy to do this kind of labelling in monailabel? Where to define new custom labels and how to repurpose tool tracking models? Shall I manually annotate first and then retrain models or use other models like DeepGrow or DeepEdit?

Any link to the relevant materials for achieving this task will be much appreciated.

Many thanks

Bilal

[SachidanandAlle]

I believe you have already explored MONAILabel + CVAT. As part of endoscopy example:

• we only illustrated how you can create dataset out of video frames (flattened outside)
• segmentation model - tooltracking
• classification model - inbody/outbody
• interactive model - deepedit

These models are trained on very few samples (public data). But they are good enough to refer for each of the kind. You should be able to refer them and train your own model.

Sometimes to train your own model pythonic way, you don't need entire setup. You can run the sample-app locally without starting the server. For example: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/endoscopy/main.py#L304-L331

[bilalcodehub]

@SachidanandAlle: thanks for translating MONAILabel-specific terminologies with the common ML tasks.

Does the classification work for multi-label problems having more than two classes?

Is it possible to integrate the code written with fastai into monailabel since it is also built on top of the pytorch?