App is expecting key.txt to be CSV

[soichih]

Jasleen is running into this error (on parc-stats-v1.1 branch)

Traceback (most recent call last):
  File "/export/prod/5f984bdc82260651b50a20aa/5fb2c3345079858265dace12/createColorLUT.py", line 46, in <module>
    main()
  File "/export/prod/5f984bdc82260651b50a20aa/5fb2c3345079858265dace12/createColorLUT.py", line 43, in main
    createColorLUT(keyfile,outfile)
  File "/export/prod/5f984bdc82260651b50a20aa/5fb2c3345079858265dace12/createColorLUT.py", line 13, in createColorLUT
    df = df[[2,3]]
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/frame.py", line 2806, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer
    keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/indexing.py", line 1640, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([2, 3], dtype='int64')] are in the [columns]"

The input key.txt is not CSV, but stored in this format (generated by app-roiGenerator)

lh.hv4 -> 1
lh.lO1 -> 2
lh.lO2 -> 3
lh.tO1 -> 4
lh.tO2 -> 5
lh.v1 -> 6
lh.v2 -> 7
lh.v3 -> 8
lh.v3a -> 9
lh.v3b -> 10
lh.vO1 -> 11
lh.vO2 -> 12
rh.hv4 -> 13
rh.lO1 -> 14
rh.lO2 -> 15
rh.tO1 -> 16
rh.tO2 -> 17
rh.v1 -> 18
rh.v2 -> 19
rh.v3 -> 20
rh.v3a -> 21
rh.v3b -> 22
rh.vO1 -> 23
rh.vO2 -> 24

key.txt is really supposed to look like this

0 -> 0 == Unknown
1 -> 1 == Left-Cerebral-Exterior
2 -> 2 == Left-Cerebral-White-Matter
3 -> 3 == Left-Cerebral-Cortex
4 -> 4 == Left-Lateral-Ventricle
5 -> 5 == Left-Inf-Lat-Vent

As I reported in brainlife/app-roiGenerator#3 , key.txt should be deprecated by label.json. It would be ideal if app-roiGenerator outputs label.json, and have app-freesurfer-stats reads label.json - and maybe fall back to reading key.txt if it doesn't exist?

[bacaron]

Hey @soichih,

Thanks for adding the issues.

I agree, key.txt should be deprecated by label.json. I just haven't gotten around to updating the apps since our discussion last week. I'll work on that this week.

In terms of the expectations of csv, that's not an issue. Pandas is really good at reading files of all different types. The issue is I'm hard indexing the data I needed in that createColorLUT.py function. Just so happens that was built off the normal key.txt file and not the one from the specific app causing the error now.

Thanks,
Brad