2024-02-08 Update

mi2rl · Feb 7, 2024 · 9dece08 · 9dece08
1 parent cbae24c
commit 9dece08
Show file tree

Hide file tree

Showing 362 changed files with 23,099 additions and 13,598 deletions.
diff --git a/.gitignore b/.gitignore
@@ -112,3 +112,12 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
+# 제외 목록들
+checkpoints/
+Notebook/
+predictions/
+백업용/
+train.sh
+test.sh
+
diff --git a/Blind_Reader_Study/Blind_Reader_Study_OMR.xlsx b/Blind_Reader_Study/Blind_Reader_Study_OMR.xlsx
diff --git a/Blind_Reader_Study/set_1/AA.png b/Blind_Reader_Study/set_1/AA.png
diff --git a/Blind_Reader_Study/set_1/BB.png b/Blind_Reader_Study/set_1/BB.png
diff --git a/Blind_Reader_Study/set_1/CC.png b/Blind_Reader_Study/set_1/CC.png
diff --git a/Blind_Reader_Study/set_1/DD.png b/Blind_Reader_Study/set_1/DD.png
diff --git a/Blind_Reader_Study/set_1/EE.png b/Blind_Reader_Study/set_1/EE.png
diff --git a/Blind_Reader_Study/set_1/FF.png b/Blind_Reader_Study/set_1/FF.png
diff --git a/Blind_Reader_Study/set_1/GG.png b/Blind_Reader_Study/set_1/GG.png
diff --git a/Blind_Reader_Study/set_1/HH.png b/Blind_Reader_Study/set_1/HH.png
diff --git a/Blind_Reader_Study/set_1/LDCT.png b/Blind_Reader_Study/set_1/LDCT.png
diff --git a/Blind_Reader_Study/set_1/NDCT.png b/Blind_Reader_Study/set_1/NDCT.png
diff --git a/Blind_Reader_Study/set_10/AA.png b/Blind_Reader_Study/set_10/AA.png
diff --git a/Blind_Reader_Study/set_10/BB.png b/Blind_Reader_Study/set_10/BB.png
diff --git a/Blind_Reader_Study/set_10/CC.png b/Blind_Reader_Study/set_10/CC.png
diff --git a/Blind_Reader_Study/set_10/DD.png b/Blind_Reader_Study/set_10/DD.png
diff --git a/Blind_Reader_Study/set_10/EE.png b/Blind_Reader_Study/set_10/EE.png
diff --git a/Blind_Reader_Study/set_10/FF.png b/Blind_Reader_Study/set_10/FF.png
diff --git a/Blind_Reader_Study/set_10/GG.png b/Blind_Reader_Study/set_10/GG.png
diff --git a/Blind_Reader_Study/set_10/HH.png b/Blind_Reader_Study/set_10/HH.png
diff --git a/Blind_Reader_Study/set_10/LDCT.png b/Blind_Reader_Study/set_10/LDCT.png
diff --git a/Blind_Reader_Study/set_10/NDCT.png b/Blind_Reader_Study/set_10/NDCT.png
diff --git a/Blind_Reader_Study/set_2/AA.png b/Blind_Reader_Study/set_2/AA.png
diff --git a/Blind_Reader_Study/set_2/BB.png b/Blind_Reader_Study/set_2/BB.png
diff --git a/Blind_Reader_Study/set_2/CC.png b/Blind_Reader_Study/set_2/CC.png
diff --git a/Blind_Reader_Study/set_2/DD.png b/Blind_Reader_Study/set_2/DD.png
diff --git a/Blind_Reader_Study/set_2/EE.png b/Blind_Reader_Study/set_2/EE.png
diff --git a/Blind_Reader_Study/set_2/FF.png b/Blind_Reader_Study/set_2/FF.png
diff --git a/Blind_Reader_Study/set_2/GG.png b/Blind_Reader_Study/set_2/GG.png
diff --git a/Blind_Reader_Study/set_2/HH.png b/Blind_Reader_Study/set_2/HH.png
diff --git a/Blind_Reader_Study/set_2/LDCT.png b/Blind_Reader_Study/set_2/LDCT.png
diff --git a/Blind_Reader_Study/set_2/NDCT.png b/Blind_Reader_Study/set_2/NDCT.png
diff --git a/Blind_Reader_Study/set_3/AA.png b/Blind_Reader_Study/set_3/AA.png
diff --git a/Blind_Reader_Study/set_3/BB.png b/Blind_Reader_Study/set_3/BB.png
diff --git a/Blind_Reader_Study/set_3/CC.png b/Blind_Reader_Study/set_3/CC.png
diff --git a/Blind_Reader_Study/set_3/DD.png b/Blind_Reader_Study/set_3/DD.png
diff --git a/Blind_Reader_Study/set_3/EE.png b/Blind_Reader_Study/set_3/EE.png
diff --git a/Blind_Reader_Study/set_3/FF.png b/Blind_Reader_Study/set_3/FF.png
diff --git a/Blind_Reader_Study/set_3/GG.png b/Blind_Reader_Study/set_3/GG.png
diff --git a/Blind_Reader_Study/set_3/HH.png b/Blind_Reader_Study/set_3/HH.png
diff --git a/Blind_Reader_Study/set_3/LDCT.png b/Blind_Reader_Study/set_3/LDCT.png
diff --git a/Blind_Reader_Study/set_3/NDCT.png b/Blind_Reader_Study/set_3/NDCT.png
diff --git a/Blind_Reader_Study/set_4/AA.png b/Blind_Reader_Study/set_4/AA.png
diff --git a/Blind_Reader_Study/set_4/BB.png b/Blind_Reader_Study/set_4/BB.png
diff --git a/Blind_Reader_Study/set_4/CC.png b/Blind_Reader_Study/set_4/CC.png
diff --git a/Blind_Reader_Study/set_4/DD.png b/Blind_Reader_Study/set_4/DD.png
diff --git a/Blind_Reader_Study/set_4/EE.png b/Blind_Reader_Study/set_4/EE.png
diff --git a/Blind_Reader_Study/set_4/FF.png b/Blind_Reader_Study/set_4/FF.png
diff --git a/Blind_Reader_Study/set_4/GG.png b/Blind_Reader_Study/set_4/GG.png
diff --git a/Blind_Reader_Study/set_4/HH.png b/Blind_Reader_Study/set_4/HH.png
diff --git a/Blind_Reader_Study/set_4/LDCT.png b/Blind_Reader_Study/set_4/LDCT.png
diff --git a/Blind_Reader_Study/set_4/NDCT.png b/Blind_Reader_Study/set_4/NDCT.png
diff --git a/Blind_Reader_Study/set_5/AA.png b/Blind_Reader_Study/set_5/AA.png
diff --git a/Blind_Reader_Study/set_5/BB.png b/Blind_Reader_Study/set_5/BB.png
diff --git a/Blind_Reader_Study/set_5/CC.png b/Blind_Reader_Study/set_5/CC.png
diff --git a/Blind_Reader_Study/set_5/DD.png b/Blind_Reader_Study/set_5/DD.png
diff --git a/Blind_Reader_Study/set_5/EE.png b/Blind_Reader_Study/set_5/EE.png
diff --git a/Blind_Reader_Study/set_5/FF.png b/Blind_Reader_Study/set_5/FF.png
diff --git a/Blind_Reader_Study/set_5/GG.png b/Blind_Reader_Study/set_5/GG.png
diff --git a/Blind_Reader_Study/set_5/HH.png b/Blind_Reader_Study/set_5/HH.png
diff --git a/Blind_Reader_Study/set_5/LDCT.png b/Blind_Reader_Study/set_5/LDCT.png
diff --git a/Blind_Reader_Study/set_5/NDCT.png b/Blind_Reader_Study/set_5/NDCT.png
diff --git a/Blind_Reader_Study/set_6/AA.png b/Blind_Reader_Study/set_6/AA.png
diff --git a/Blind_Reader_Study/set_6/BB.png b/Blind_Reader_Study/set_6/BB.png
diff --git a/Blind_Reader_Study/set_6/CC.png b/Blind_Reader_Study/set_6/CC.png
diff --git a/Blind_Reader_Study/set_6/DD.png b/Blind_Reader_Study/set_6/DD.png
diff --git a/Blind_Reader_Study/set_6/EE.png b/Blind_Reader_Study/set_6/EE.png
diff --git a/Blind_Reader_Study/set_6/FF.png b/Blind_Reader_Study/set_6/FF.png
diff --git a/Blind_Reader_Study/set_6/GG.png b/Blind_Reader_Study/set_6/GG.png
diff --git a/Blind_Reader_Study/set_6/HH.png b/Blind_Reader_Study/set_6/HH.png
diff --git a/Blind_Reader_Study/set_6/LDCT.png b/Blind_Reader_Study/set_6/LDCT.png
diff --git a/Blind_Reader_Study/set_6/NDCT.png b/Blind_Reader_Study/set_6/NDCT.png
diff --git a/Blind_Reader_Study/set_7/AA.png b/Blind_Reader_Study/set_7/AA.png
diff --git a/Blind_Reader_Study/set_7/BB.png b/Blind_Reader_Study/set_7/BB.png
diff --git a/Blind_Reader_Study/set_7/CC.png b/Blind_Reader_Study/set_7/CC.png
diff --git a/Blind_Reader_Study/set_7/DD.png b/Blind_Reader_Study/set_7/DD.png
diff --git a/Blind_Reader_Study/set_7/EE.png b/Blind_Reader_Study/set_7/EE.png
diff --git a/Blind_Reader_Study/set_7/FF.png b/Blind_Reader_Study/set_7/FF.png
diff --git a/Blind_Reader_Study/set_7/GG.png b/Blind_Reader_Study/set_7/GG.png
diff --git a/Blind_Reader_Study/set_7/HH.png b/Blind_Reader_Study/set_7/HH.png
diff --git a/Blind_Reader_Study/set_7/LDCT.png b/Blind_Reader_Study/set_7/LDCT.png
diff --git a/Blind_Reader_Study/set_7/NDCT.png b/Blind_Reader_Study/set_7/NDCT.png
diff --git a/Blind_Reader_Study/set_8/AA.png b/Blind_Reader_Study/set_8/AA.png
diff --git a/Blind_Reader_Study/set_8/BB.png b/Blind_Reader_Study/set_8/BB.png
diff --git a/Blind_Reader_Study/set_8/CC.png b/Blind_Reader_Study/set_8/CC.png
diff --git a/Blind_Reader_Study/set_8/DD.png b/Blind_Reader_Study/set_8/DD.png
diff --git a/Blind_Reader_Study/set_8/EE.png b/Blind_Reader_Study/set_8/EE.png
diff --git a/Blind_Reader_Study/set_8/FF.png b/Blind_Reader_Study/set_8/FF.png
diff --git a/Blind_Reader_Study/set_8/GG.png b/Blind_Reader_Study/set_8/GG.png
diff --git a/Blind_Reader_Study/set_8/HH.png b/Blind_Reader_Study/set_8/HH.png
diff --git a/Blind_Reader_Study/set_8/LDCT.png b/Blind_Reader_Study/set_8/LDCT.png
diff --git a/Blind_Reader_Study/set_8/NDCT.png b/Blind_Reader_Study/set_8/NDCT.png
diff --git a/Blind_Reader_Study/set_9/AA.png b/Blind_Reader_Study/set_9/AA.png
diff --git a/Blind_Reader_Study/set_9/BB.png b/Blind_Reader_Study/set_9/BB.png
diff --git a/Blind_Reader_Study/set_9/CC.png b/Blind_Reader_Study/set_9/CC.png
diff --git a/Blind_Reader_Study/set_9/DD.png b/Blind_Reader_Study/set_9/DD.png
diff --git a/Blind_Reader_Study/set_9/EE.png b/Blind_Reader_Study/set_9/EE.png
diff --git a/Blind_Reader_Study/set_9/FF.png b/Blind_Reader_Study/set_9/FF.png
diff --git a/Blind_Reader_Study/set_9/GG.png b/Blind_Reader_Study/set_9/GG.png
diff --git a/Blind_Reader_Study/set_9/HH.png b/Blind_Reader_Study/set_9/HH.png
diff --git a/Blind_Reader_Study/set_9/LDCT.png b/Blind_Reader_Study/set_9/LDCT.png
diff --git a/Blind_Reader_Study/set_9/NDCT.png b/Blind_Reader_Study/set_9/NDCT.png
diff --git a/README.md b/README.md
@@ -5,36 +5,47 @@ We proposed a supervised multi-task aiding representation transfer learning netw
 
 
 ## 💡 Highlights
-<!-- + Develop a robust feature extractor of brain hemorrhage in head & neck NCCT through three kinds of -->
-<!-- multi-task representation learning. -->
+<!--
+• Develop a discriminator utilizing multi-task learning (MTL), which leverages three simultaneous tasks—restoration, image-level decision, and pixel-level decision—to transfer contextual, global, and local feedback between the real normal-dose and synthesized images to the generator.
+-->
+<!--
+• Propose two regulations to improve the representation capabilities of the discriminator: restoration consistency (RC), which compares the discriminator's outputs from the input data with the corresponding restoration data generated by our MTL discriminator for consistency, and non-difference suppression (NDS), which excludes areas that cause confusion in discriminator decisions.
+-->
+<!--
+• Design a novel generator that consists of residual fast Fourier transform with convolution (Res-FFT-Conv) blocks [13] that fuse frequency-spatial dual-domain representations. The proposed generator effectively captures rich information by simultaneously utilizing spatial (or local), spectral (or global), and residual connections. To the best of our knowledge, this represents an inaugural effort in employing the Res-FFT-Conv block within the generator for LDCT denoising, which demonstrates the versatility of the block.
+-->
+<!--
+• Evaluate our network with extensive experiments, including an ablation study and visual scoring using two distinct datasets of brain and abdominal CT images. Six metrics based on pixel- and feature-spaces were used, and the results indicated superior performances in both quantitative and qualitative measures compared to those of state-of-the-art denoising techniques.
+-->
 
 
 
-<!-- <p align="center"><img width="100%" src="figures/graphical_abstract.png" /></p> -->
+<!-- <p align="center"><img width="100%" src="figures/Graphical_Abstract.png" /></p> -->
 
 
 ## Paper
 This repository provides the official implementation of training SMART-Net as well as the usage of the pre-trained MTD-Net in the following paper:
 
-<!-- <b>Improved performance and robustness of multi-task representation learning with consistency loss between pretexts for intracranial hemorrhage identification in head CT</b> <br/>
-[Sunggu Kyung](https://github.com/babbu3682)<sup>1</sup>, Keewon Shin, Hyunsu Jeong, Ki Duk Kim, Jooyoung Park, Kyungjin Cho, Jeong Hyun Lee, Gil-Sun Hong, and Namkug Kim <br/>
+Sunggu Kyung, Jongjun Won, Seongyong Pak, Sunwoo Kim, Sangyoon Lee, Kanggil Park, Gil-Sun Hong, and Namkug Kim
+
+<!-- <b>Generative Adversarial Network with Robust Discriminator Through Multi-Task Learning for Low-Dose CT Denoising</b> <br/>
+[Sunggu Kyung](https://github.com/babbu3682)<sup>1</sup>, Jongjun Won, Seongyong Pak, Sunwoo Kim, Sangyoon Lee, Kanggil Park, Gil-Sun Hong, and Namkug Kim <br/>
 [MI2RL LAB](https://www.mi2rl.co/) <br/>
-<b>(Under revision...)</b> Medical Image Analysis (MedIA) <br/>
-<!-- [paper](https://arxiv.org/pdf/2004.07882.pdf) | [code](https://github.com/babbu3682/SMART-Net) | [graphical abstract](https://ars.els-cdn.com/content/image/1-s2.0-S1361841520302048-fx1_lrg.jpg) -->
-[code](https://github.com/babbu3682/SMART-Net) -->
+<b>(Under revision...)</b> IEEE Transactions on Medical Imaging (TMI) <br/>
+
 
 
 ## Requirements
 + Linux
++ CUDA 11.6
 + Python 3.8.5
-+ PyTorch 1.8.0
-
++ Pytorch 1.13.1
 
-## 📦 MTD-Net Framework
+## 📦 MTD-GAN Framework
 ### 1. Clone the repository and install dependencies
 ```bash
-$ git clone https://github.com/babbu3682/MTD-Net.git
-$ cd MTD-Net/
+$ git clone https://github.com/babbu3682/MTD-GAN.git
+$ cd MTD-GAN/
 $ pip install -r requirements.txt
 ```
 
@@ -43,97 +54,90 @@ $ pip install -r requirements.txt
 #### Note: We do not use this data as a train, it is just for code publishing examples.
 
 <!-- Download the data from [this repository](https://zenodo.org/record/4625321/files/TransVW_data.zip?download=1).  -->
-You can use your own data using the [dicom2nifti](https://github.com/icometrix/dicom2nifti) for converting from dicom to nii.
 
-- The processed hemorrhage directory structure
+- The processed dataset directory structure
 ```
-datasets/samples/
+datasets/MAYO
     train
-        |--  sample1_hemo_img.nii.gz
-        |--  sample1_hemo_mask.nii
-        |--  sample2_normal_img.nii.gz
-        |--  sample2_normal_mask.nii        
-                .
-                .
-                .
+        |--  full_3mm
+        |--  quarter_3mm
+            |--  L067
+            |--  L096
+            |--  L109
+            |--  L143
+                    .
+                    .
+                    .
     valid
-        |--  sample9_hemo_img.nii.gz
-        |--  sample9_hemo_mask.nii
-        |--  sample10_normal_img.nii.gz
-        |--  sample10_normal_mask.nii
-                .
-                .
-                .
+        |--  full_3mm
+        |--  quarter_3mm
+            |--  L333
+                    .
+                    .
+                    .
     test
-        |--  sample20_hemo_img.nii.gz
-        |--  sample20_hemo_mask.nii
-        |--  sample21_normal_img.nii.gz
-        |--  sample21_normal_mask.nii
-                .
-                .
-                .   
+        |--  full_3mm
+        |--  quarter_3mm
+            |--  L506
+                    .
+                    .
+                    .
 ```
 
 ### 3. Upstream
 
-#### 📋 Available List
-- [x] Up_SMART_Net
-- [x] Up_SMART_Net_Dual_CLS_SEG
-- [x] Up_SMART_Net_Dual_CLS_REC
-- [x] Up_SMART_Net_Dual_SEG_REC
-- [x] Up_SMART_Net_Single_CLS
-- [x] Up_SMART_Net_Single_SEG
-- [x] Up_SMART_Net_Single_REC
+#### 📋 To Do List
+- [x] publish previous works' weights
+- [x] publish our works' weights
 
 
-**+ train**: We conducted upstream training with three multi-task including classificatiom, segmentation and reconstruction.
+**+ train**:
 ```bash
-python train.py \
---data-folder-dir '/workspace/sunggu/4.Dose_img2img/datasets/[sinogram]Brain_3mm_DCM' \
---model-name 'Markovian_Patch_GAN' \
+CUDA_VISIBLE_DEVICES=2 python -W ignore train.py \
+--dataset 'mayo' \
+--dataset-type-train 'window_patch' \
+--dataset-type-valid 'window' \
+--batch-size 20 \
+--train-num-workers 16 \
+--valid-num-workers 16 \
+--model 'MTD_GAN_Method' \
+--loss 'L1 Loss' \
+--method 'pcgrad' \
+--optimizer 'adamw' \
+--scheduler 'poly_lr' \
+--epochs 500 \
+--warmup-epochs 10 \
 --lr 1e-4 \
 --min-lr 1e-6 \
---batch-size 100 \
---epochs 500 \
---num_workers 8 \
---pin-mem \
---criterion 'L1 Loss' \
---criterion_mode 'not balance' \
---multiple_GT "False" \
---windowing "True" \
---patch_training "True" \
 --multi-gpu-mode 'Single' \
---cuda-visible-devices '3' \
---print-freq 1 \
---save-checkpoint-every 2 \
---checkpoint-dir '/workspace/sunggu/4.Dose_img2img/model/[Previous]Markovian_Patch_GAN' \
---png-save-dir '/workspace/sunggu/4.Dose_img2img/Predictions/Train/[Previous]Markovian_Patch_GAN/'
+--device 'cuda' \
+--print-freq 10 \
+--save-checkpoint-every 1 \
+--checkpoint-dir '/workspace/sunggu/4.Dose_img2img/MTD_GAN/checkpoints/abdomen/MTD_GAN' \
+--save-dir '/workspace/sunggu/4.Dose_img2img/MTD_GAN/predictions/train/abdomen/MTD_GAN' \
+--memo 'abdomen, 500 epoch, node 14'
 ```
-<!-- **+ test**: We conducted upstream training with three multi-task including classificatiom, segmentation and reconstruction.
+
+<!-- **+ test**:
 ```bash
-python test.py \
---data-folder-dir '/workspace/sunggu/1.Hemorrhage/SMART-Net/datasets/samples' \
---test-dataset-name 'Custom' \
---slice-wise-manner "True" \
---model-name 'Up_SMART_Net' \
---num-workers 4 \
---pin-mem \
---training-stream 'Upstream' \
---multi-gpu-mode 'Single' \
---cuda-visible-devices '2' \
---print-freq 1 \
---output-dir '/workspace/sunggu/1.Hemorrhage/SMART-Net/checkpoints/up_test' \
---resume '/workspace/sunggu/1.Hemorrhage/SMART-Net/checkpoints/up_test/epoch_0_checkpoint.pth'
+# CUDA_VISIBLE_DEVICES=2 python -W ignore test.py \
+# --dataset 'mayo_test' \
+# --dataset-type-test 'window' \
+# --test-batch-size 1 \
+# --test-num-workers 16 \
+# --model 'MTD_GAN' \
+# --loss 'L1 Loss' \
+# --multi-gpu-mode 'Single' \
+# --device 'cuda' \
+# --print-freq 10 \
+# --checkpoint-dir '/workspace/sunggu/4.Dose_img2img/MTD_GAN/checkpoints/abdomen/MTD_GAN' \
+# --save-dir '/workspace/sunggu/4.Dose_img2img/MTD_GAN/predictions/test/abdomen/MTD_GAN' \
+# --resume "/workspace/sunggu/4.Dose_img2img/MTD_GAN/checkpoints/abdomen/MTD_GAN/epoch_77777_checkpoint.pth" \
+# --memo 'abdomen, node 14' \
+# --epoch 77777
 ``` -->
 
 
-## Fail training methods
-### [SACNN](https://github.com/mmlipku/sacnn), [SA-SNMP](https://github.com/reach2sbera/ldct_nonlocal)
-```
-⏳ The official code was performed, but the results were very poor, and we requested from the authors, but it was unresponsive.
-```
-
-
 ## Excuse
 For personal information security reasons of medical data in Korea, our data cannot be disclosed.
 
@@ -145,9 +149,10 @@ If you use this code for your research, please cite our papers:
 ```
 
 ## 🤝 Acknowledgement
-We build SMART-Net framework by referring to the released code at [qubvel/segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch) and [Project-MONAI/MONAI](https://github.com/Project-MONAI/MONAI). 
+We build MTD-GAN framework by referring to the released code at [qubvel/segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch) and [Project-MONAI/MONAI](https://github.com/Project-MONAI/MONAI). 
 This is a patent-pending technology.
 
+We acknowledge the open-source libraries, including the [Diffuser](https://github.com/huggingface/diffusers) and [MONAI Generative Models](https://github.com/Project-MONAI/GenerativeModels), which enabled valuable comparisons in this study, and we extend our thanks to the pioneering authors (e.g., [RED-CNN](https://github.com/SSinyu/RED-CNN), [EDCNN](https://github.com/workingcoder/EDCNN), [CTformer](https://github.com/wdayang/CTformer), [Restormer](https://github.com/swz30/Restormer), [WGAN_VGG](https://github.com/hyeongyuy/CT-WGAN_VGG_tensorflow), [MAP-NN](https://github.com/hmshan/MAP-NN), [DUGAN](https://github.com/Hzzone/DU-GAN)).
 
 ### 🛡️ License <a name="license"></a>
-Project is distributed under [MIT License](https://github.com/babbu3682/SMART-Net/blob/main/LICENSE)
+Project is distributed under [MIT License](https://github.com/babbu3682/MTD-GAN/blob/main/LICENSE)
diff --git a/arch/CTformer/LICENSE b/arch/CTformer/LICENSE
diff --git a/arch/CTformer/data_aug.py b/arch/CTformer/data_aug.py