How can we use our trained model as pretrained model in MONAILabel (3D Slicer)

[Suchi97]

Hi Team,

We are trying to use our trained model to predict the labels in MONAI Label via 3D Slicer. Our trained model is trained on the labels background and spleen and we are trying to use this trained model as pretrained model and then predict the spleen using MONAI Label via 3D silcer. So, this model having a mean dice of 0.95. Since this model having input channels as 1 and output channels as 2, we changed the input from 1 to 3 (background, spleen, image) and output channels to 2 (background and spleen) to match the dimensions. When I use this trained model, the prediction result looks different. Spleen organ is not correctly predicted.

So, how can we use our trained model as the pretrained model in MONAI Label which gives the same result when predicted outside the MONAI Label.

Can you help me to understand how the pretrained model used in MONAI Label is trained. Like, is the model is trained using MONAI Label via 3D Slicer or any other methods.

Any suggestions is appreciated.
Thanks in advance

[diazandr3s]

Hi @Suchi97,

This is a good question. Which App are you using? How did you get the pre-trained model?

Importing pre-trained models obtained from other pipelines is something MONAI Label can be used for.

My suggestion is to use the Segmentation App. Other Apps such as DeepGrow and DeepEdit have more sophisticated training loops that involved click simulation. Input tensors of these Apps have several channels as you may know: image, foreground, and background clicks.

As the Segmentation App is a non-interactive model, the input tensor is only the image.

The pre-trained model should be located in a folder called model inside the folder App and the .pt file should be named pretrained

You should also make sure that the network architecture is the same as the pre-trained model https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/segmentation/main.py#L48

Please let us know how that goes.

[SachidanandAlle]

Look for latest example (more easier for anyone to add their own models and use them)
https://github.com/Project-MONAI/MONAILabel/tree/main/sample-apps/radiology#how-to-add-new-model

You can install monailabel from github or latest docker or https://pypi.org/project/monailabel/0.4.0rc1/

and you start monailabel as monailabel start_server -a radiology -s images -c models <your_model>

[Suchi97]

Thanks @SachidanandAlle for this new information. I will try this and let u know the results.

[diazandr3s]

Superb! Glad to hear this, @Suchi97 :)

[drchen123]

Hi @Suchi97,

This is a good question. Which App are you using? How did you get the pre-trained model?

Importing pre-trained models obtained from other pipelines is something MONAI Label can be used for.

My suggestion is to use the Segmentation App. Other Apps such as DeepGrow and DeepEdit have more sophisticated training loops that involved click simulation. Input tensors of these Apps have several channels as you may know: image, foreground, and background clicks.

As the Segmentation App is a non-interactive model, the input tensor is only the image.

The pre-trained model should be located in a folder called model inside the folder App and the .pt file should be named pretrained

You should also make sure that the network architecture is the same as the pre-trained model https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/segmentation/main.py#L48

Please let us know how that goes.

Dear sir
I am a vascular surgeon, i have a high volume of CTA data, and i need to train my own model eventually for auto-segmentation of the aorta for different disease (abdominal aortic aneurysm, aortic dissection, intramural hematoma), i also want to public these trained models, so other clinicians major in these diseases could use it.
The methods you provide are of little difficult for me, could you provide a step-to-step tutorial to create my own model based on existing files in radiology?
Thanks, and look forward to hear from you.

[diazandr3s]

Hi @drchen123,

We're glad you consider MONAI Label for your task. Let's chat more about this via email or a videocall.
Please send me an email ([email protected]) so we further discuss this.

[Suchi97]

Hi Team,

We trained a model having the labels background, spleen and liver. For the training purpose, we used BTCV dataset.

We are trying to use this model as a pretrained model in MONAI Label server. Here, I used the Segmentation app.

To do this, we followed the below steps:

Changes done in MONAI_Label_Multilabel/MONAILabel/sample-apps/segmentation/main.py

1. Updated the labels dictionary to self.label_names = { "spleen": 1, "liver": 2, "background": 0, }
2. Changed the spatial size to which we used for training
3. Copied our trained model to MONAILabel/sample-apps/segmentation/model directory.
4. Changed the pretrained model name to our trained model name.

Changes done in MONAILabel/sample-apps/segmentation/lib/infer.py

1. Updated the pre-transforms.

              LoadImaged(keys=["image"]), 
              EnsureChannelFirstd(keys=["image"]),
              ScaleIntensityRanged(keys=["image"], a_min=-175, a_max=250, b_min=0, b_max=1.0, clip=True),
              EnsureTyped(keys=["image"],device=data.get("device") if data else None)

These are the pretransforms which I used for training purpose.

When I inferred a sample image of BTCV dataset outside MONAI Label, we got the below result

From the above image, we can see that the predicted results are almost good. After training we got a mean dice of 0.93.

When we used the same model in 3D Slcier, the below results are obtained.

From the above image, we can see that the results are not matched correctly.

Can you please help me to solve this

Any suggestions is appreciated.
Thanks in advance :)

[diazandr3s]

Hi @Suchi97,

Thanks for your comment. It is nice to see the results you're getting.
I think the problem comes from the inverse transforms. I need to update that bit in the segmentation app.
Please add these lines in the infer.py file in the Segmentation App. That should solve the problem.

Please let us know how that goes.

Thanks again!!

[Suchi97]

Thanks @diazandr3s for your information.
Yes we are using BTCV dataset.
We will try this and let you know the results

[Suchi97]

@diazandr3s ,

Using the dataset (BTCV_reoriented), I inferred the image img0002.nii.gz using MONAI Label Segmentation app and without MONAI Label

The below results are obtained without using MONAI Label

The below results are obtained when using MONAI Label Segmentation App

From the above image, we can some progress. But the labels are not mapped correctly.

Can you please look into this issue.
Thanks

[diazandr3s]

Hi @Suchi97,

If you're using the BTCV_reoriented dataset, why the image is still upside down? I see the axial view isn't properly oriented.
I checked myself and got the following result:

Are you using an orientation transform? These are the ones I used.

It seems there is an issue with a transformation.

BTW, after checking in more detail, now I think you don't need to use the method I recommended you before:

This is because there isn't transform to invert. Please remove those lines :)

I'd suggest you check why the images aren't properly oriented. Or maybe there is a transform that is not being applied in inference.

Keep us posted.

[Suchi97]

Thanks @diazandr3s for your reply. It was really helpful.

We could able to solve the issue. When we checked with the image img0031.nii.gz, we got the following results

The image below shows the inference without using MONAI Label

The image below shows the inference using the segmentation app in MONAI Label

Now, we could able to use our own trained model in MONAI Label (multilabel). [The model is not that good and the mean dice obtained for the above model is 0.89]

Can you please help us to understand the reason behind the reorientation of BTCV images and how to reorient the images.
If we come across any other images like that, it will be helpful.

Thanks in advance.

[SachidanandAlle]

to answer your question on re-orientation.. you have to invert if you are doing it as part of pre-transform.

For example.. for deepedit, you have pretransform: Orientationd(keys="image", axcodes="RAS")
https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/infers/deepedit.py#L73

so you need to revert it back.. through invert/post transform

[SachidanandAlle]

I suggest to refer latest radiology app and there segmentation model..
https://github.com/Project-MONAI/MONAILabel/tree/main/sample-apps/radiology#segmentation

Similarly you can add your 3-label model into radiology app..
More details on how to add a new model:
https://github.com/Project-MONAI/MONAILabel/tree/main/sample-apps/radiology#how-to-add-new-model

few things are simplified new latest main branch (which shall be released in 0.4 version)

[diazandr3s]

Thanks for the update, @Suchi97.
You could use the BTCV dataset as is to train DeepEdit or Segmentation App.
The reason we've decided to reorient it first is because clinicians are used to see the images in this orientation:

If you don't mind having the images upside down, you don't need to reorient them first.

As @SachidanandAlle commented, all radiology apps were merged into single App.

Hope this helps.

[Suchi97]

Thanks @SachidanandAlle and @diazandr3s for your reply.

We could able to understood how to use our own trained model in MONAI Label.
We will also try with the new MONAI Label version also.

Thanks again :)

[DAC-ACD]

Hello! I have a similar questions.

Some information about my requests: The task is to make the automated segmentation, but I can't see my target organ in default labels (rectum and colon). So, I decided to train custom model for my needs, based on Your deepedit model.

I'd like to train deepedit model for several times with different volumes. As I understand, after every training session server downloading trained weights:
"...\radiology\model\deepedit_dynunet.pt"
But also I see some more created weights:
"...\radiology\model\deepedit_dynunet\train_01\checkpoint_final.pt"
"...\radiology\model\deepedit_dynunet\train_01\model.pt"

So, the first question is: which one is needed to continue training my new model?
The second question is: if I continue training after going to the "next sample", I continue training of previous model or I creating a new one?
The third question is: if the answer of the second question "continue training of previous model", how can I start next session with previously trained model?

What I already understood:

1. I need to enable pre-trained model:
   if strtobool(self.conf.get("use_pretrained_model", "true")): url = f"{self.conf.get('pretrained_path', self.PRE_TRAINED_PATH)}/deepedit_{network}_multilabel.pt" download_file(url, self.path[0]) -> "false" in deepedit.py
2. correct self.labels in deepedit.py

Also, that would be great, if You give me some more recommendations to simplify my work!

[diazandr3s]

Hello! I have a similar questions.

Some information about my requests: The task is to make the automated segmentation, but I can't see my target organ in default labels (rectum and colon). So, I decided to train custom model for my needs, based on Your deepedit model.

I'd like to train deepedit model for several times with different volumes. As I understand, after every training session server downloading trained weights: "...\radiology\model\deepedit_dynunet.pt" But also I see some more created weights: "...\radiology\model\deepedit_dynunet\train_01\checkpoint_final.pt" "...\radiology\model\deepedit_dynunet\train_01\model.pt"

So, the first question is: which one is needed to continue training my new model? The second question is: if I continue training after going to the "next sample", I continue training of previous model or I creating a new one? The third question is: if the answer of the second question "continue training of previous model", how can I start next session with previously trained model?

What I already understood:

1. I need to enable pre-trained model:
   `if strtobool(self.conf.get("use_pretrained_model", "true")): url = f"{self.conf.get('pretrained_path', self.PRE_TRAINED_PATH)}/deepedit_{network}_multilabel.pt" download_file(url, self.path[0])`  ->  "false" in deepedit.py

2. correct `self.labels` in deepedit.py

Also, that would be great, if You give me some more recommendations to simplify my work!

Hi @DAC-ACD,

So, the first question is: which one is needed to continue training my new model?

This one "...\radiology\model\deepedit_dynunet.pt"

The second question is: if I continue training after going to the "next sample", I continue training of previous model or I creating a new one?

By default, it'll use the previous as starting point. It means it'll re-train the previous model. Unless you disable the flag pretrained in Slicer.

The third question is: if the answer of the second question "continue training of previous model", how can I start next session with previously trained model?

If this file "...\radiology\model\deepedit_dynunet.pt"exists and the pretrained flag is enabled, you'll use that model.

This file gets updated automatically if during training there is a model performing better.