Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RF] Remove the recon workflows from QSIPrep #802

Merged
merged 36 commits into from
Aug 14, 2024
Merged

Conversation

mattcieslak
Copy link
Collaborator

@mattcieslak mattcieslak commented Aug 9, 2024

Closes #697.

It's difficult to maintain the recon workflows and preprocessing workflows in one package. We're creating a separate package that takes QSIPrep (or UKBB) preprocessed data and runs the recon workflows on it. The new QSIRecon repo can be found here.

For users this means that the following options will no longer be available from qsiprep:

  • --recon-only
  • --recon-spec
  • --recon-input
  • --recon-input-pipeline
  • --freesurfer-input
  • --skip-odf-reports
  • --interactive-reports-only

And because preprocessing doesn't directly use freesurfer, we don't need

  • --fs-license-file

Also, QSIPrep will no longer append qsiprep to the output directory.

@tsalo tsalo added the breaking-change Issues/PRs that change results or interfaces. label Aug 12, 2024
@tsalo
Copy link
Member

tsalo commented Aug 12, 2024

When I disabled test_main, I got the weirdest error:

Cmdline:
	dsi_studio --action=rec --method=4 --align_acpc=0 --check_btable=0 --dti_no_high_b=1 --source=/src/qsiprep/.circleci/work/dscsdsi/qsiprep_0_22_wf/sub_tester_wf/dwi_preproc_acq_HASC55AP_wf/pre_hmc_wf/merge_and_denoise_wf/dwi_qc_wf/raw_gqi/sub-tester_acq-HASC55AP_dwi_merged.src.gz --num_fiber=3 --thread_count=8 --other_output=all --record_odf=1 --r2_weighted=0 --param0=1.2500 --thread_count=8
Stdout:
	DSI Studio version: Hou "侯" Aug  2 2024
	┌──�[1;34m📟command line�[0m
	├──�[0;32msource�[0m=/src/qsiprep/.circleci/work/dscsdsi/qsiprep_0_22_wf/sub_tester_wf/dwi_preproc_acq_HASC55AP_wf/pre_hmc_wf/merge_and_denoise_wf/dwi_qc_wf/raw_gqi/sub-tester_acq-HASC55AP_dwi_merged.src.gz
	├──�[0;32maction�[0m=rec
	├──�[0;32mloop�[0m=/src/qsiprep/.circleci/work/dscsdsi/qsiprep_0_22_wf/sub_tester_wf/dwi_preproc_acq_HASC55AP_wf/pre_hmc_wf/merge_and_denoise_wf/dwi_qc_wf/raw_gqi/sub-tester_acq-HASC55AP_dwi_merged.src.gz
	├──┬──�[1;34m📟run rec�[0m
	│  ├──┬──�[1;35m📂open SRC file sub-tester_acq-HASC55AP_dwi_merged.src.gz�[0m
	│  │  ├──generating mask
	│  │  └──⏱49ms
	│  ├──┬──�[1;34m📟reconstruction parameters�[0m
	│  │  ├──�[0;32mmethod�[0m=4
	│  │  ├──�[0;32modf_resolving�[0m=0
	│  │  ├──�[0;32mrecord_odf�[0m=1
	│  │  ├──�[0;32mdti_no_high_b�[0m=1
	│  │  ├──�[0;32mother_output�[0m=all
	│  │  ├──�[0;32mr2_weighted�[0m=0
	│  │  ├──�[0;32mthread_count�[0m=8
	│  │  ├──�[0;32mparam0�[0m=1.2500
	│  │  ├──�[0;32mparam1�[0m=3000
	│  │  ├──�[0;32mparam2�[0m=0.05
	│  │  ├──�[0;33mtemplate 0�[0m: "ICBM152_adult.QA.nii"
	│  │  ├──�[0;33mtemplate 1�[0m: "C57BL6_mouse.QA.nii"
	│  │  ├──�[0;33mtemplate 2�[0m: "dHCP_neonate.QA.nii"
	│  │  ├──�[0;33mtemplate 3�[0m: "INDI_rhesus.QA.nii"
	│  │  ├──�[0;33mtemplate 4�[0m: "Pitt_marmoset.QA.nii"
	│  │  ├──�[0;33mtemplate 5�[0m: "WHS_SD_rat.QA.nii"
	│  │  ├──�[0;32mtemplate�[0m=0
	│  │  └──⏱0ms
	│  ├──�[0;32mmask�[0m=1
	│  ├──┬──�[1;34m📟pre-processing steps�[0m
	│  │  ├──�[0;32mmotion_correction�[0m=0
	│  │  ├──�[0;32mcheck_btable�[0m=0
	│  │  ├──�[0;32malign_acpc�[0m=0
	│  │  └──⏱0ms
	│  ├──┬──�[1;34m📟pre-reconstruction�[0m
	│  │  ├──11ReadDWIData
	│  │  ├──10Dwi2Tensor
	│  │  ├──9GQI_Recon
	│  │  ├──9RDI_Recon
	│  │  ├──11SaveMetrics
	│  │  ├──9OutputODF
	│  │  └──⏱46ms
	│  ├──┬──�[1;34m📟GQI�[0m
	│  │  └──⏱158ms
	│  ├──┬──�[1;34m📟post-reconstruction�[0m
	│  │  ├──11ReadDWIData
	│  │  ├──10Dwi2Tensor
	│  │  ├──9GQI_Recon
	│  │  ├──9RDI_Recon
	│  │  ├──11SaveMetrics
	│  │  ├──9OutputODF
	│  │  ├──┬──�[1;34m📟odf�[0m
	│  │  │  └──⏱1.25s
	│  │  └──⏱1.362s
	│  ├──�[1;35m💾saving /src/qsiprep/.circleci/work/dscsdsi/qsiprep_0_22_wf/sub_tester_wf/dwi_preproc_acq_HASC55AP_wf/pre_hmc_wf/merge_and_denoise_wf/dwi_qc_wf/raw_gqi/sub-tester_acq-HASC55AP_dwi_merged.src.gz.odf.gqi.1.25.fib.gz�[0m
	│  └──⏱1.618s
	└──⏱1.62s
	�[1;31m❗--num_fiber is not used/recognized. Did you mean --stay_open ?�[0m
	�[1;31m❗--thread_count is not used/recognized. Did you mean --other_image ?�[0m
Stderr:

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 400, in run
	    outputs = self.aggregate_outputs(runtime)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 429, in aggregate_outputs
	    predicted_outputs = self._list_outputs()  # Predictions from _list_outputs
	  File "/src/qsiprep/qsiprep/interfaces/dsi_studio.py", line 235, in _list_outputs
	    config.loggers.interface.info("current dir", os.getcwd())
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1477, in info
	    self._log(INFO, msg, args, **kwargs)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1624, in _log
	    self.handle(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1634, in handle
	    self.callHandlers(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1696, in callHandlers
	    hdlr.handle(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 968, in handle
	    self.emit(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/_pytest/logging.py", line 388, in emit
	    super().emit(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1108, in emit
	    self.handleError(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 1100, in emit
	    msg = self.format(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 943, in format
	    return fmt.format(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/_pytest/logging.py", line 141, in format
	    return super().format(record)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 678, in format
	    record.message = record.getMessage()
	  File "/opt/conda/envs/qsiprep/lib/python3.10/logging/__init__.py", line 368, in getMessage
	    msg = msg % self.args
	TypeError: not all arguments converted during string formatting

@mattcieslak
Copy link
Collaborator Author

This is from a bad call to the logger in interfaces/dsi_studio.py. There were a couple other instances in there that I fixed, but it switched from a print (that allowed multiple arguments) to a logger.info, where it is expecting the second and later arguments to be used in formatting the first

@tsalo
Copy link
Member

tsalo commented Aug 12, 2024

It's still really weird that the way I run the tests affects it. I'll have to dig into that in a follow-up PR.

@mattcieslak
Copy link
Collaborator Author

I was wondering about that too. qsiprep will print the error but exit with a 0 status

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 59.30233% with 35 lines in your changes missing coverage. Please review.

Project coverage is 28.59%. Comparing base (7506edb) to head (302f030).

Files Patch % Lines
qsiprep/workflows/anatomical/surface.py 0.00% 6 Missing ⚠️
qsiprep/workflows/anatomical/volume.py 14.28% 6 Missing ⚠️
qsiprep/cli/workflow.py 20.00% 4 Missing ⚠️
qsiprep/utils/misc.py 0.00% 4 Missing ⚠️
qsiprep/workflows/base.py 0.00% 3 Missing ⚠️
qsiprep/cli/run.py 71.42% 2 Missing ⚠️
qsiprep/interfaces/bids.py 92.00% 2 Missing ⚠️
qsiprep/workflows/dwi/finalize.py 0.00% 2 Missing ⚠️
qsiprep/workflows/fieldmap/unwarp.py 0.00% 2 Missing ⚠️
qsiprep/interfaces/dipy.py 0.00% 1 Missing ⚠️
... and 3 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #802      +/-   ##
==========================================
- Coverage   29.86%   28.59%   -1.27%     
==========================================
  Files          97       69      -28     
  Lines       14627    10218    -4409     
  Branches     1892     1331     -561     
==========================================
- Hits         4368     2922    -1446     
+ Misses      10126     7195    -2931     
+ Partials      133      101      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mattcieslak mattcieslak merged commit 6decb38 into master Aug 14, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Issues/PRs that change results or interfaces.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restructure CLI to bring in line with nipreps
3 participants