[ENH] Allow the T1-Linear pipeline to go over (subject, session) couples that do not have data

[AliceJoubert]

TO DISCUSS : change of a logic for clinica_file_reader but also clinica_list_of_files_reader (DWI pipelines)

Currently clinica_file_reader returns a list of files and a list errors messages/strings. As such, complicated to retrieve what sub/ses is problematic + most of the time these are not used since function called like so files, _ = clinica_file_reader(...). Otherwise, used to stop the processing when a subject/session is missing files for ex. : a bit critical.

-> list of messages replaced by list of tuples with the actual problematic sessions
-> no more breaking when subject/session does not have files, they are removed from the lists of sub/sess
-> raise replaced by cprint. User informed that some files won't be processed instead so removed parameter/part
-> clinica_list_of_files impacted

Currently clinica_list_of_files_reader uses clinica_files_reader to retrieve files by batches. Does not consider a list of files if there is any errors in it instead of removing the error.

-> not considering "errors" (file not found)

TO DO :

• verify what happens when empty subject/sess list after cleaning
• correct typing
• Change function descriptions
• Do for all pipelines
• Docs to change ?

From :

(clinEnv) alice.joubert@ICM-COLLI-MP013 clinica % clinica run t1-linear BIDS CAPS

2024-09-19 15:21:13,301:WARNING:More than one run were found for subject sub-OAS31158 and session ses-M024 : 

- /Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-01_T1w.nii.gz
- /Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-02_T1w.nii.gz

Clinica will proceed with the latest run available, that is 

-/Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-02_T1w.nii.gz.

2024-09-19 15:21:13,302:ERROR:Clinica faced error(s) while trying to read files in your BIDS directory.
Clinica encountered 2 problem(s) while getting T1w MRI:
        * (sub-OAS31158 | ses-M006): No file found
        * (sub-OAS31158 | ses-M078): No file found
 
-- end --

to :

(clinEnv) alice.joubert@ICM-COLLI-MP013 clinica % clinica run t1-linear BIDS CAPS

2024-09-19 15:24:45,966:WARNING:More than one run were found for subject sub-OAS31158 and session ses-M024 : 

- /Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-01_T1w.nii.gz
- /Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-02_T1w.nii.gz

Clinica will proceed with the latest run available, that is 

-/Users/alice.joubert/clinicaQC/data/test_jump_subjects/bids/sub-OAS31158/ses-M024/anat/sub-OAS31158_ses-M024_run-02_T1w.nii.gz.

2024-09-19 15:24:45,967:WARNING:Clinica encountered 2 problem(s) while getting T1w MRI:
        * (sub-OAS31158 | ses-M006)
        * (sub-OAS31158 | ses-M078)
Clinica could not identify which file to use (missing or too many) for these sessions. They will not be processed.

2024-09-19 15:24:45,967:INFO:The pipeline will be run on the following 3 image(s):
2024-09-19 15:24:45,967:INFO:   sub-OAS31158 | ses-M000, ses-M024, ses-M084,
2024-09-19 15:24:45,967:INFO:The pipeline will last approximately 6 minutes per image.

-- ... --

[NicolasGensollen]

Thanks @AliceJoubert !
I had a quick look and I think the proposed solution looks good.
We can discuss this at some point if you have some questions in mind that I missed here.

[AliceJoubert]

Thnaks for the review @NicolasGensollen ! My biggest question was related to the function clinica_list_of_file_reader since the modification I made changes the file it looks at. Then, if you agree with the changes, should I also touch up other pipelines in this PR since they use the same function ?

[NicolasGensollen]

Thanks @AliceJoubert !
I made a quick pass with small suggestions, but will need to test it a bit more.

[NicolasGensollen]

Thanks @AliceJoubert !
The code LGTM and I've run some of the non regression tests without errors.
Were you able to run some tests on your end ?
Are you still working on the PR or is it ready ?

[AliceJoubert]

Thanks for the review @NicolasGensollen ! I am not working on it anymore technically but I did not test more than t1-linear and non-regression tests. Maybe it is still lacking some tests with other pipelines...

[NicolasGensollen]

Thanks for the review @NicolasGensollen ! I am not working on it anymore technically but I did not test more than t1-linear and non-regression tests. Maybe it is still lacking some tests with other pipelines...

I did a few others as well without issues.
Let's merge and see what the CI will tell us.