Accuracy Metric #1715
-
|
Hi everyone! I have a quick question about the Accuracy metric GUI in the MONAI Label extension on 3D Slicer. Based on previous discussions, there's an automatic 20% split from training for validation. If, for example, I've manually labeled 30 images using segmentation.py (out of a total of unlabeled 300 images) and trained on each one, what does the accuracy metric of 60% in the GUI signify? Would it mean it is using that trained and labeled image for validation? Thanks! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
Hi @jesamir, That's right. By default, MONAI Label uses 20% of the training set to get the dice score you see in the GUI. You could redefine this method in the trainers file to use a different validation set or change the percentage Bear in mind that you'll need an annotated dataset to validate/test your model. Dice score is a measure of the similarity between the prediction and ground truth. Hope this helps, |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much! For validation and testing the model, do I press on the Submit Label GUI only while with training, press on the Submit Label GUI and the Train? A follow-up question: if I do end up pressing the Submit Label GUI while also pressing train, what would then the accuracy represent? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @jesamir, I realise the terms testing and validation here may cause confusion. MONAI Label uses 20% of the annotated dataset to get a Dice score so user get a sense of how the model is doing. It is up to the user and the use case how they interpret this score. When you hit Submit Label using the GUI, MONAI Label adds that annotation to the annotated set.
If you hit Submit Label and then Train, MONAI Label adds the annotation to the labelled dataset and then splits 80/20 the dataset to train or retrain the model. The score you see is always the one obtained from previous epoch of training. Hope this helps, |
Beta Was this translation helpful? Give feedback.
-
|
This makes sense now! Thank you so much for your help! |
Beta Was this translation helpful? Give feedback.
Hi @jesamir,
I realise the terms testing and validation here may cause confusion. MONAI Label uses 20% of the annotated dataset to get a Dice score so user get a sense of how the model is doing. It is up to the user and the use case how they interpret this score.
When you hit Submit Label using the GUI, MONAI Label adds that annotation to the annotated set.
When you hit Train, MONAI Lable splits the annotated dataset into 80/20 for training an getting the Dice score.
If you hit Submit Label and then Train, MONAI Label adds the annotation to the labelled d…