Binary segmentation #28

ThierryJudge · 2021-11-11T21:52:02Z

Add support for binary segmentation.
Changes were made to the Dice loss, segmentation system and Camus dataset.

Other minor changes in this PR (sorry for the random changes)

Add image logging to segmentation and classification system.
Remove weights_only and add weights param to avoid two parameters for loading only the weights.
Fix logging of Comet Tags
Add fix_registry_name for loading comet models with illegal characters.

…=classification +module=mlp"

…mp/refactor/hydra

log hparams

add arguments for system instantiate add missing params to camus config

fix camus labels

…s.txt

…ital into tmp/hydra/yaml

…de module parameters

* Add softmax and mcdropout system * Add evaluation pipeline (Seg metrics, calibration overlap)

Remove resume/train/ckpt_path errors from runner Change default name to remove error.

Update networks for 1 class output

Co-authored-by: Nathan Painchaud <[email protected]>

…ng weights.

…ture/binary-segmentation

Co-authored-by: Nathan Painchaud <[email protected]>

…ra-dev

…ture/binary-segmentation

nathanpainchaud

I didn't have to review all the modifications for now, but I commented on slight docstrings edits and on what required the most rework IMO (the image logging debug/dev code).

nathanpainchaud · 2021-12-03T21:27:40Z

vital/data/camus/dataset.py

+                if len(self.labels) == 2:  # For binary segmentation, make foreground class 1.
+                    proc_gts_tensor[proc_gts_tensor != 0] = 1
+
+                if len(self.labels) == 2:  # For binary segmentation, make foreground class 1.
+                    gts[gts != 0] = 1


Both ifs should be merged.

nathanpainchaud · 2021-12-03T21:47:16Z

vital/metrics/train/functional.py

@@ -35,32 +36,58 @@ def tversky_score(
            Available reduction methods:
            - ``'elementwise_mean'``: takes the mean (default)
            - ``'none'``: no reduction will be applied
+        apply_activation: when True, softmax or sigmoid is applied to input.


I'd reword the doc slightly:

Suggested change

apply_activation: when True, softmax or sigmoid is applied to input.

apply_activation: If ``True``, applies softmax or sigmoid to the input before computing the score. Otherwise, assumes the input is already normalized.

nathanpainchaud · 2021-12-03T21:47:29Z

vital/metrics/train/functional.py

@@ -25,8 +26,8 @@ def tversky_score(
          https://lars76.github.io/2018/09/27/loss-functions-for-segmentation.html

    Args:
-        input: (N, C, H, W), Raw, unnormalized scores for each class.
-        target: (N, H, W), Groundtruth labels, where each value is 0 <= targets[i] <= C-1.
+        input: (N, C, H, W), Raw, unnormalized (or normalized apply_activation == False) scores for each class.


Missing if + formatting:

Suggested change

input: (N, C, H, W), Raw, unnormalized (or normalized apply_activation == False) scores for each class.

input: (N, C, H, W), Raw, unnormalized (or normalized if ``apply_activation == False``) scores for each class.

nathanpainchaud · 2021-12-03T21:49:17Z

vital/metrics/train/functional.py

 ) -> Tensor:
    """Computes the loss definition of the dice coefficient.

    Args:
-        input: (N, C, H, W), Raw, unnormalized scores for each class.
+        input: (N, C, H, W), Raw, unnormalized (or normalized apply_activation == False) scores for each class.


Same comment as for tversky_score.

nathanpainchaud · 2021-12-03T21:49:21Z

vital/metrics/train/functional.py

@@ -83,12 +111,20 @@ def differentiable_dice_score(
            Available reduction methods:
            - ``'elementwise_mean'``: takes the mean (default)
            - ``'none'``: no reduction will be applied
+        apply_activation: when True, softmax or sigmoid is applied to input.


Same comment as for tversky_score.

nathanpainchaud · 2021-12-03T21:49:57Z

vital/metrics/train/metric.py

@@ -23,19 +24,21 @@ def __init__(
                Available reduction methods:
                - ``'elementwise_mean'``: takes the mean (default)
                - ``'none'``: no reduction will be applied
+            apply_activation: when True, softmax or sigmoid is applied to input.


Same comment as for the functional version.

nathanpainchaud · 2021-12-03T21:50:05Z

vital/metrics/train/metric.py


    def forward(self, input: Tensor, target: Tensor) -> Tensor:
        """Actual metric calculation.

        Args:
-            input: (N, C, H, W), Raw, unnormalized scores for each class.
+            input: (N, C, H, W), Raw, unnormalized (or normalized apply_activation == False) if scores for each class.


Same comment as for the functional version.

nathanpainchaud · 2021-12-03T21:54:58Z

vital/utils/serialization.py

+    """Changes registry name to replace characters that COMET ML does not accept in the registry name.
+
+    Note: Comet ML rules (Might need to be updated)
+        - All special characters are replaced with dash (ex. camus_unet -> camus-unet, camus!unet -> camus-unet)
+        - All non-letter and non-number characters at the end of the name are deleted.
+        - All letters are lower-case.
+
+    Args:
+        registry_name: Name of the registered model to update.
+
+    Returns:
+        Updated registry name
+    """


Just a few edits:

Suggested change

"""Changes registry name to replace characters that COMET ML does not accept in the registry name.

Note: Comet ML rules (Might need to be updated)

- All special characters are replaced with dash (ex. camus_unet -> camus-unet, camus!unet -> camus-unet)

- All non-letter and non-number characters at the end of the name are deleted.

- All letters are lower-case.

Args:

registry_name: Name of the registered model to update.

Returns:

Updated registry name

"""

"""Changes registry names to replace characters that Comet ML does not accept.

Notes: Comet ML rules (Might need to be updated)

- All special characters are replaced with dash (ex. my_model -> my-model, my!model -> my-model)

- All non-letter and non-number characters at the end of the name are deleted.

- All letters are lower-case.

Args:

registry_name: Name of the registered model to fix.

Returns:

Fixed registry name.

"""

nathanpainchaud · 2021-12-03T21:56:05Z

vital/utils/serialization.py

+    registry_name = re.sub("[^a-zA-Z0-9]+", "-", registry_name)  # Replace all special characters with `-`
+    registry_name = re.sub(r"^\W+|\W+$", "", registry_name)  # Remove all special characters at end of name


I don't use Python's regex often enough to remember how they work off the top of my head, so I'll trust you on that one 😉

nathanpainchaud · 2021-12-03T21:59:32Z

vital/systems/classification.py

+        if self.is_val_step and batch_idx == 0:
+            self.log_images(
+                title="Sample",
+                num_images=5,
+                axes_content={"Image": x.cpu().squeeze().numpy()},
+                info=[f"(Gt: {y[i].item()} - Pred: {y_hat.argmax(dim=1)[i].item()})" for i in range(5)],
+            )


I'm in a bit of a rush to make a well thought-out suggestion, but to me this seems like debug/dev code, that we shouldn't ship as part of the model. I assume it would be possible to bundle both this bit of code and the log_images added to VitalSystem into a callback. And I would very much suggest to do it that way.

nathanpainchaud and others added 30 commits May 14, 2021 22:40

Refactored DataManagerMixin as LightningDataModule

c31658c

working on hydra to replace argparse

d32982f

working training for mnist with "python runner.py +data=mnist +system…

e4eaba0

…=classification +module=mlp"

Merge branch 'dev' of https://github.com/nathanpainchaud/vital into t…

4223448

…mp/refactor/hydra

working training for acdc

e42e6d5

Add callbacks

fe4ddd3

change callbacks from list to dict

31b58ec

add check_cfg method

8d17de6

add check for default root dir

dc0b652

implement yaml configs

214df88

change config dir name,

5df3bcf

log hparams

callbacks and config

d60a349

fix callbacks and loggers

159d300

add arguments for system instantiate add missing params to camus config

Fix pre-commit

f2a444b

merge dev

19512a7

add hydra to requirements.txt

a36608b

fix camus labels

allow more configurable callbacks

301619e

fix loading from checkpoint path, add hydra and dotenv to requirement…

f2097c8

…s.txt

Merge branch 'tmp/hydra/yaml' of https://github.com/nathanpainchaud/v…

47a7b64

…ital into tmp/hydra/yaml

allow model saving if not checkpoint if used

a289bf8

fix loggers

28eab4a

Move module configs in system folder to allow system config to overri…

242f2cb

…de module parameters

Work in progress:

bb921b2

* Add softmax and mcdropout system * Add evaluation pipeline (Seg metrics, calibration overlap)

change unet dropout parameter name to be uniform with enet

2919c22

Add resolvers for num_workers and gpus

8c6c63c

add workers=True to seed_everything in runner.py

491a5f3

Merge dev.

df6c67f

Remove argparse methods.

b970267

Minor fixes and PR comment fixes

1941861

Remove test_step from computation.

bde2127

Remove resume/train/ckpt_path errors from runner Change default name to remove error.

ThierryJudge and others added 23 commits August 3, 2021 12:24

Add softmax parameter to Dice

012cc46

Update segmentation metric for dice for binary segmentation

eff50ce

Update networks for 1 class output

Add image loggind to segmentation.py and classification system.

40e008a

Almost functioning binary dice

312e734

minor improvements

d1ff81d

Refactor Hydra to use yaml configuration (#22)

53851e1

Co-authored-by: Nathan Painchaud <[email protected]>

minor change

91e7e3a

Work in progress, fix comet TAGS

19d6d00

Merge hydra dev into feature/binary seg

27b94d5

change weights_only to weights to avoid passing 2 argument when loadi…

7f5cdf8

…ng weights.

Fix dropout for Unet

b0cc1b1

WIP learning sample confidence

5afd341

Add dropout attribute

e4e5d9c

Add fix registry name for serialization.py

9d8931e

Fix binary dice with pred.squeeze() and clean up.

9c6aa0c

Fix typos in docstring

d4a5b9d

Merge dev into hydra-dev

dd5b2ef

Typo fix for MLP

37a96c0

Merge branch 'hydra-dev' of github.com:nathanpainchaud/vital into fea…

be1b920

…ture/binary-segmentation

Refactor Hydra to use yaml configuration (#22)

c7872c7

Co-authored-by: Nathan Painchaud <[email protected]>

Typo fix for MLP

2280ff8

Merge branch 'hydra-dev' of github.com:nathanpainchaud/vital into hyd…

0067543

…ra-dev

Merge branch 'hydra-dev' of github.com:nathanpainchaud/vital into fea…

5a0b318

…ture/binary-segmentation

ThierryJudge requested a review from nathanpainchaud November 11, 2021 21:52

ThierryJudge changed the title ~~Feature/binary segmentation~~ Binary segmentation Nov 11, 2021

Remove comment from unet.

0509ae7

nathanpainchaud assigned ThierryJudge Nov 11, 2021

nathanpainchaud added the enhancement New feature or request label Nov 11, 2021

nathanpainchaud requested changes Dec 3, 2021

View reviewed changes

Base automatically changed from hydra-dev to dev April 15, 2022 12:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Binary segmentation #28

Binary segmentation #28

ThierryJudge commented Nov 11, 2021

nathanpainchaud left a comment

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

nathanpainchaud Dec 3, 2021

	apply_activation: when True, softmax or sigmoid is applied to input.
	apply_activation: If ``True``, applies softmax or sigmoid to the input before computing the score. Otherwise, assumes the input is already normalized.

	input: (N, C, H, W), Raw, unnormalized (or normalized apply_activation == False) scores for each class.
	input: (N, C, H, W), Raw, unnormalized (or normalized if ``apply_activation == False``) scores for each class.

		registry_name = re.sub("[^a-zA-Z0-9]+", "-", registry_name) # Replace all special characters with `-`
		registry_name = re.sub(r"^\W+\|\W+$", "", registry_name) # Remove all special characters at end of name

Binary segmentation #28

Are you sure you want to change the base?

Binary segmentation #28

Conversation

ThierryJudge commented Nov 11, 2021

nathanpainchaud left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment