Table of cells

README.md

@@ -59,8 +59,8 @@ python evaluation.py --inputPath test256_LNENonly --configPath configs/eval_Ki67
 ```
 
 ## Step 4: Optimize the cell detection threshold
-- The post-processing pipeline applied after UNET uses dea seuila to establish which are the "true cells". We propose to optimize these thresholds for marker-positive and marker-negative cells using the `eval_opt_thresholds.py` script.
-- The command below is used to run the script for channel 0 associated with cells detected as marker positive:
+- The post-processing pipeline applied after UNET uses two thresholds to establish which are the "true cells". We propose to optimize these thresholds for marker-positive and marker-negative cells using the `eval_opt_thresholds.py` script.
+- The following command is used to run the script for channel 0, which is associated with cells detected as marker positive:
 ```
 python eval_opt_thresholds.py --inputPath test256 --configPath configs/eval_Ki67_LNEN.json --dataname OptTh0_pos_cells.csv --minth 75 --minth 95 --channel 0
 ```
@@ -85,9 +85,40 @@ python infer.py --inputPath KI67_Tiles_256_256_40x/ --configPath configs/eval_Ki
     - `jpg` annotated image if the `--visualization` argument is specified 
 - The `outputPath` folder will have the same organization as the `inputPath` folder.
 
+## Step 6: Calculate spatial statistics:
+### Step 6.1: Create a table of detected cells within the tumour area
+-In order to create the graph needed to compute the spatial statistics, we first need to create a table of detected cells within the tumour area, with their xy coordinates and class.
+- For this we assume that the tumour has been previously segmented as described in [TumorSegmentationCFlowAD](https://github.com/IARCbioinfo/TumorSegmentationCFlowAD) github repository.
+- Command line:
+```
+python table_of_cells_after_segmentation.py --inputdir ~/LNENWork/Ki67InferencePathonet --patient_id TNE1983 --segmentation_dir TumorSegmentation_Ki67
+```
+- The `segmentation_dir` should follow the following architecture:
+    - `segmentation_dir`
+        - `patient_id`
+            - `prediction_tumor_normal_TNE1983.csv`
+- The table  `prediction_tumor_normal_TNE1983.csv` contains the following information:
+
+|file_path|PredTumorNomal                                                                                      |
+|---------|----------------------------------------------------------------------------------------------------|
+|KI67_Tiling_256_256_40x/TNE1983.svs/accept/TNE1983.svs_11777_28161.jpg|Tumor                                                                                               |
+|KI67_Tiling_256_256_40x/TNE1983.svs/accept/TNE1983.svs_16385_7169.jpg|Tumor                                                                                               |
+|KI67_Tiling_256_256_40x/TNE1983.svs/accept/TNE1983.svs_21505_10241.jpg|Tumor                                                                                               |
+|KI67_Tiling_256_256_40x/TNE1983.svs/accept/TNE1983.svs_18945_21505.jpg|Normal                      
+
+- The output table is stored in  `inputdir/patient_id/patient_id_cells_detected_segmented.csv` and will contains the following information:
+
+|x  |y                                                                                                   |label            |
+|---|----------------------------------------------------------------------------------------------------|-----------------|
+|17659.5|6669.5                                                                                              |1                |
+|17637.785720825195|6902.785720825195                                                                                   |1                |
+|17616.0|6663.0                                                                                              |2                |
+
+
+*Note: In this table, the label 1 corresponds to a positive cell and 2 to a negative cell.
 ## TO DO LIST
 
-+ :construction: Add PHH3 configs and weights
 + :construction: Add NetworkX construction
++ :construction: Table of cells
 + :construction: Add Spatial statistics
 + :construction: Add presentation WSI

SpatialStatsGraph/table_of_cells_after_segmentation.py

@@ -0,0 +1,69 @@
+import json
+import pandas as pd
+import os
+import argparse
+parser = argparse.ArgumentParser('table_of_cells')
+parser.add_argument('--inputdir', '-i', required=True, help="Folder where json files generated by infer.py are stored")
+parser.add_argument('--patient_id', '-s', required=True, help="Patient ID for which the segmentation table will be created")
+parser.add_argument('--segmentation_dir',  default=   '/home/mathiane/LNENWork/CFlow_mathiane/CFlow/TumorSegmentation_Ki67_Baseline_2809_1004/segmentation_maps_3_thresholds' , required=False)
+
+# Get parameters
+args = parser.parse_args()
+inputdir = args.inputdir
+segmentation_dir = args.segmentation_dir
+patient_id = args.patient_id
+# Name of the output table
+df_name =  patient_id + '_cells_detected_segmented.csv'
+print('patient_id  ', patient_id)
+
+print("Get tiles ID within the tumoral area")
+# Get segmentation table
+segmentation_correct = False
+for tneid in os.listdir(segmentation_dir):
+    if tneid.find(patient_id) != -1:  
+        for ele in os.listdir(os.path.join(segmentation_dir, tneid)):
+            if ele.find("csv") != -1:
+                df_seg = pd.read_csv(os.path.join(segmentation_dir, tneid, ele))
+                segmentation_correct = True
+
+# Take into account only the tumoral tiles
+df_seg_tumor = df_seg[df_seg['PredTumorNomal'] == 'Tumor'] 
+print(df_seg_tumor.head())
+if segmentation_correct:
+
+    print("Get the infrence json filename of the tumoral tiles")
+    tiles_to_include = []
+    for i in range(df_seg_tumor.shape[0]):
+        # Get name of json files generated by infer.py
+        tile_id  =  df_seg_tumor.iloc[i,1].split('/')[-1][:-3] + 'json'
+        tiles_to_include.append(tile_id)
+
+    print("Create the coordianate table")
+    x_list = []
+    y_list = []
+    label_list = []
+    print(os.path.join(inputdir,args.patient_id, 'accept'))
+
+    for file in os.listdir(os.path.join(inputdir,args.patient_id, 'accept')):
+        # Read inference files if the tiles belongs to the tumor area
+        if file.find('json') != -1 and file in tiles_to_include:
+            # Get tile coords
+            xul = int(file.split('_')[1])
+            yul = int(file.split('_')[2].split('.')[0])
+            with open(os.path.join(inputdir, args.patient_id ,'accept', file)) as json_file:
+                cells = json.load(json_file)
+            # Get all cells predicted in the tiles
+            for cell in cells:
+                    x_list.append(cell['x']+ xul )
+                    y_list.append(cell['y'] + yul)
+                    label_list.append(cell['label_id'])
+
+    # Table summarizing the coordinated and the class of the cells detected by Pathonet in the tumor area
+    df = pd.DataFrame()
+    df['x'] = x_list
+    df['y'] = y_list
+    df['label'] = label_list
+    print("Write the table")
+    df.to_csv(os.path.join(inputdir,args.patient_id, df_name))
+else:
+    print("Segmentation table not found")