From a1b088f5ab0425caecbece353a0a680674e80ba6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C5=82gorzata=20Wierzba?= Date: Sat, 23 Nov 2024 16:56:14 +0100 Subject: [PATCH 1/4] doc: configure special remote with autoenable=true This is needed when one works with a clone and wants the clone to have the special remote readily available. --- examples/fmriprep-docker/README.md | 3 ++- examples/fmriprep-singularity/README.md | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/examples/fmriprep-docker/README.md b/examples/fmriprep-docker/README.md index 72a5893..d32c2ba 100644 --- a/examples/fmriprep-docker/README.md +++ b/examples/fmriprep-docker/README.md @@ -63,7 +63,8 @@ Configure the dataset in which you want to collect the results of the (re)comput Add a `datalad-remake` special remote: ```bash -> git annex initremote datalad-remake type=external externaltype=datalad-remake encryption=none allow-untrusted-execution=true +> git annex initremote datalad-remake type=external externaltype=datalad-remake \ +encryption=none allow-untrusted-execution=true autoenable=true ``` ### Add template diff --git a/examples/fmriprep-singularity/README.md b/examples/fmriprep-singularity/README.md index f11faa5..0c73bca 100644 --- a/examples/fmriprep-singularity/README.md +++ b/examples/fmriprep-singularity/README.md @@ -66,7 +66,8 @@ Configure the dataset in which you want to collect the results of the (re)comput Add a `datalad-remake` special remote: ```bash -> git annex initremote datalad-remake type=external externaltype=datalad-remake encryption=none allow-untrusted-execution=true +> git annex initremote datalad-remake type=external externaltype=datalad-remake \ +encryption=none allow-untrusted-execution=true autoenable=true ``` ### Add template From 505ddf9319e55042d10bb61cba83aac5828d2434 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C5=82gorzata=20Wierzba?= Date: Sat, 23 Nov 2024 17:00:52 +0100 Subject: [PATCH 2/4] doc: add resampling use case --- examples/fmriprep-resample/README.md | 108 +++++++++++++++++++ examples/fmriprep-resample/fmriprep-resample | 24 +++++ examples/fmriprep-resample/input.txt | 18 ++++ examples/fmriprep-resample/output.txt | 2 + examples/fmriprep-resample/parameter.txt | 4 + 5 files changed, 156 insertions(+) create mode 100644 examples/fmriprep-resample/README.md create mode 100644 examples/fmriprep-resample/fmriprep-resample create mode 100644 examples/fmriprep-resample/input.txt create mode 100644 examples/fmriprep-resample/output.txt create mode 100644 examples/fmriprep-resample/parameter.txt diff --git a/examples/fmriprep-resample/README.md b/examples/fmriprep-resample/README.md new file mode 100644 index 0000000..9d099e7 --- /dev/null +++ b/examples/fmriprep-resample/README.md @@ -0,0 +1,108 @@ +# Use case: resample a BOLD image onto a target space + +This example demonstrates how to run a simple Python [script](https://hub.datalad.org/mslw/fmriprep-resampling) to reproduce fMRIPrep's preprocessed BOLD image (projected onto a target space) from the raw image, ancillary fMRIPrep derivatives, and related metadata. All dependencies for the script are provided by an fMRIPrep singularity container. The singularity container used in this example is `bids-fmriprep--24.1.0` and comes from the [ReproNim containers collection](https://github.com/ReproNim/containers). + +The example comprises the following files: +- `fmriprep-resample` template +- `input.txt` input specification +- `output.txt` output specification +- `parameter.txt` parameters + +## Requirements + +This example requires Singularity. + +Please note, that there is no need to install fMRIPrep. The singularity container will be automatically retrieved from the ReproNim containers collection. + +## How to install + +Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). + +## How to use + +It is assumed that you have a local copy of the `datalad-remake` project in your `$HOME` directory. If this not the case, adjust the path below: + +``` +EXAMPLE=$HOME/datalad-remake/examples/fmriprep-resample +``` + +### Clone example dataset + +To run the example, you'll need a raw BIDS dataset that has been minimally preprocessed with fMRIPrep. For a complete list of data dependencies, please refer to [this](https://github.com/datalad/datalad-remake/blob/main/examples/fmriprep-resample/input.txt) specification. + +For convenience, a ready-made dataset containing all inputs required for running the example can be obtained like so: + +```bash +> cd $HOME +> datalad clone https://hub.datalad.org/example my-project +> cd my-project +> datalad get -n data/ds001734 +> datalad get -n derivatives/ds001734 +``` + +The dataset is organized in a modular way. It contains raw BIDS data (`data/ds001734`), as well as fMRIPrep derivative data (`derivatives/ds001734`). Also, it includes the `resample.py` script, as well as the software container that will be needed for running the script (`code/containers`). + +The resulting dataset structure is as follows: + +``` +my-project +├── code +│ ├── containers +│ ├── make +│ └── resample.py +├── data +│ └── ds001734 +└── derivatives + └── ds001734 +``` + +### Configure (or enable) special remote + +Configure the dataset in which you want to collect the results of the (re)computation, in this case `derivatives/ds001734` subdataset. + +```bash +> cd $HOME/my-project/derivatives/ds001734 +``` + +If you're working with the ready-made example dataset, you are good to go. The `datalad-remake` special remote is already enabled. You can verify that by running: + +```bash +> datalad siblings +``` + +If you're working with your own dataset, make sure to configure a `datalad-remake` special remote: + +```bash +> git annex initremote datalad-remake type=external externaltype=datalad-remake \ +encryption=none allow-untrusted-execution=true autoenable=true +``` + +### Add template + +Place the `fmriprep-resample` template in the `.datalad/make/methods` of the root dataset: + +```bash +> cd $HOME/my-project +> cp $EXAMPLE/fmriprep-resample .datalad/make/methods/fmriprep-resample +> datalad save -m "Add a make method" +``` + +Place the `input.txt`, `output.txt` and `parameter.txt` files in the root dataset. These files do not have to be tracked in git history, so no `datalad save` is required at this point. + +```bash +> mkdir -p code/make/fmriprep-resample +> cp $EXAMPLE/*.txt ./code/make/fmriprep-resample/ +``` + +### Execute (re)computation + +To test the example, run: + +```bash +> cd $HOME/my-project +> datalad make \ +-I code/make/fmriprep-resample/input.txt \ +-O code/make/fmriprep-resample/output.txt \ +-P code/make/fmriprep-resample/parameter.txt \ +--allow-untrusted-execution fmriprep-resample +``` diff --git a/examples/fmriprep-resample/fmriprep-resample b/examples/fmriprep-resample/fmriprep-resample new file mode 100644 index 0000000..ecfcced --- /dev/null +++ b/examples/fmriprep-resample/fmriprep-resample @@ -0,0 +1,24 @@ +# This example template runs a Python script to reproduce fMRIPrep's +# preprocessed BOLD image (projected onto a target space) from the raw image, +# ancillary fMRIPrep derivatives, and related metadata. All dependencies for +# the script are provided by an fMRIPrep singularity container. +# +# The template takes the following inputs: +# - container: the path to fMRIPrep singularity container image +# - target_file: resampled file (fMRIPrep derivative) to reproduce +# - raw_dataset: raw BIDS dataset location +# - out_dir: base output path + +parameters = ['container', 'target_file', 'raw_dataset', 'out_dir'] + +# Note: `{root_directory}` resolves to the directory of the dataset in which the +# computation was started with `datalad make`. + +command = [ + 'singularity', + 'exec', '{container}', + 'python', 'code/resample.py', + '{target_file}', + '{root_directory}/{raw_dataset}', + '{root_directory}/{out_dir}' +] diff --git a/examples/fmriprep-resample/input.txt b/examples/fmriprep-resample/input.txt new file mode 100644 index 0000000..ab5ab69 --- /dev/null +++ b/examples/fmriprep-resample/input.txt @@ -0,0 +1,18 @@ +# Paths are relative to the dataset in which `datalad make` was executed +derivatives/ds001734/dataset_description.json +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.json +derivatives/ds001734/sub-001/fmap/sub-001_fmapid-auto00000_desc-preproc_fieldmap.json +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_desc-confounds_timeseries.tsv +data/ds001734/task-MGT_bold.json +data/ds001734/sub-001/func/sub-001_task-MGT_run-01_bold.nii.gz +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_from-orig_to-boldref_mode-image_desc-hmc_xfm.txt +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_from-boldref_to-auto00000_mode-image_xfm.txt +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_from-boldref_to-T1w_mode-image_desc-coreg_xfm.txt +derivatives/ds001734/sub-001/anat/sub-001_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5 +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_space-MNI152NLin2009cAsym_boldref.nii.gz +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_desc-hmc_boldref.nii.gz +derivatives/ds001734/sub-001/fmap/sub-001_fmapid-auto00000_desc-magnitude_fieldmap.nii.gz +derivatives/ds001734/sub-001/fmap/sub-001_fmapid-auto00000_desc-coeff_fieldmap.nii.gz +code/containers/images/bids/bids-fmriprep--24.1.0.sing +code/resample.py \ No newline at end of file diff --git a/examples/fmriprep-resample/output.txt b/examples/fmriprep-resample/output.txt new file mode 100644 index 0000000..cc743b4 --- /dev/null +++ b/examples/fmriprep-resample/output.txt @@ -0,0 +1,2 @@ +# Paths are relative to the dataset in which `datalad make` was executed +derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz \ No newline at end of file diff --git a/examples/fmriprep-resample/parameter.txt b/examples/fmriprep-resample/parameter.txt new file mode 100644 index 0000000..59dd034 --- /dev/null +++ b/examples/fmriprep-resample/parameter.txt @@ -0,0 +1,4 @@ +container=code/containers/images/bids/bids-fmriprep--24.1.0.sing +target_file=derivatives/ds001734/sub-001/func/sub-001_task-MGT_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz +raw_dataset=data/ds001734 +out_dir=derivatives/ds001734 From f45274a254ba9d2c47f9a45ba37df344b6382687 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C5=82gorzata=20Wierzba?= Date: Wed, 29 Jan 2025 21:51:17 +0100 Subject: [PATCH 3/4] doc: remove instructions to initiate the datalad-remake remote --- examples/fmriprep-docker/README.md | 15 --------------- examples/fmriprep-resample/README.md | 21 --------------------- examples/fmriprep-singularity/README.md | 15 --------------- 3 files changed, 51 deletions(-) diff --git a/examples/fmriprep-docker/README.md b/examples/fmriprep-docker/README.md index d32c2ba..eb97ee1 100644 --- a/examples/fmriprep-docker/README.md +++ b/examples/fmriprep-docker/README.md @@ -52,21 +52,6 @@ my-project └── ds001734 ``` -### Configure special remote - -Configure the dataset in which you want to collect the results of the (re)computation, in this case `derivatives/ds001734` subdataset. - -```bash -> cd $HOME/my-project/derivatives/ds001734 -``` - -Add a `datalad-remake` special remote: - -```bash -> git annex initremote datalad-remake type=external externaltype=datalad-remake \ -encryption=none allow-untrusted-execution=true autoenable=true -``` - ### Add template Place the `fmriprep-docker` template in the `.datalad/make/methods` of the root dataset: diff --git a/examples/fmriprep-resample/README.md b/examples/fmriprep-resample/README.md index 9d099e7..9842147 100644 --- a/examples/fmriprep-resample/README.md +++ b/examples/fmriprep-resample/README.md @@ -56,27 +56,6 @@ my-project └── ds001734 ``` -### Configure (or enable) special remote - -Configure the dataset in which you want to collect the results of the (re)computation, in this case `derivatives/ds001734` subdataset. - -```bash -> cd $HOME/my-project/derivatives/ds001734 -``` - -If you're working with the ready-made example dataset, you are good to go. The `datalad-remake` special remote is already enabled. You can verify that by running: - -```bash -> datalad siblings -``` - -If you're working with your own dataset, make sure to configure a `datalad-remake` special remote: - -```bash -> git annex initremote datalad-remake type=external externaltype=datalad-remake \ -encryption=none allow-untrusted-execution=true autoenable=true -``` - ### Add template Place the `fmriprep-resample` template in the `.datalad/make/methods` of the root dataset: diff --git a/examples/fmriprep-singularity/README.md b/examples/fmriprep-singularity/README.md index 0c73bca..f2a7a5c 100644 --- a/examples/fmriprep-singularity/README.md +++ b/examples/fmriprep-singularity/README.md @@ -55,21 +55,6 @@ my-project └── ds001734 ``` -### Configure special remote - -Configure the dataset in which you want to collect the results of the (re)computation, in this case `derivatives/ds001734` subdataset. - -```bash -> cd $HOME/my-project/derivatives/ds001734 -``` - -Add a `datalad-remake` special remote: - -```bash -> git annex initremote datalad-remake type=external externaltype=datalad-remake \ -encryption=none allow-untrusted-execution=true autoenable=true -``` - ### Add template Place the `fmriprep-singularity` template in the `.datalad/make/methods` of the root dataset: From 847f21391122aa693084f5f2f53f9d4618dc627d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C5=82gorzata=20Wierzba?= Date: Thu, 30 Jan 2025 10:14:11 +0100 Subject: [PATCH 4/4] doc: mention trusted execution in example READMEs --- examples/fmriprep-docker/README.md | 10 +++++++++- examples/fmriprep-resample/README.md | 10 +++++++++- examples/fmriprep-singularity/README.md | 10 +++++++++- 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/examples/fmriprep-docker/README.md b/examples/fmriprep-docker/README.md index eb97ee1..44c7b46 100644 --- a/examples/fmriprep-docker/README.md +++ b/examples/fmriprep-docker/README.md @@ -18,7 +18,7 @@ It is assumed that the license file is located in `/tmp`. Make sure to copy it t ## How to install -Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). +Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). Make sure that you have a valid GPG key and that you have successfully configured Git for commit signing, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#requirements). ## How to use @@ -70,6 +70,14 @@ Place the `input.txt`, `output.txt` and `parameter.txt` files in the root datase > cp $EXAMPLE/*.txt ./code/make/fmriprep-docker/ ``` +### Configure trusted keys + +Configure trusted keys, by executing the command below. Replace `` with a GPG key that you have used for signing commits. For more details, please go [here](https://github.com/datalad/datalad-remake#trusted-keys). + +```bash +> git config --global --add datalad.make.trusted-keys +``` + ### Execute (re)computation To test the example, run: diff --git a/examples/fmriprep-resample/README.md b/examples/fmriprep-resample/README.md index 9842147..f7f9ffe 100644 --- a/examples/fmriprep-resample/README.md +++ b/examples/fmriprep-resample/README.md @@ -16,7 +16,7 @@ Please note, that there is no need to install fMRIPrep. The singularity containe ## How to install -Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). +Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). Make sure that you have a valid GPG key and that you have successfully configured Git for commit signing, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#requirements). ## How to use @@ -73,6 +73,14 @@ Place the `input.txt`, `output.txt` and `parameter.txt` files in the root datase > cp $EXAMPLE/*.txt ./code/make/fmriprep-resample/ ``` +### Configure trusted keys + +Configure trusted keys, by executing the command below. Replace `` with a GPG key that you have used for signing commits. For more details, please go [here](https://github.com/datalad/datalad-remake#trusted-keys). + +```bash +> git config --global --add datalad.make.trusted-keys +``` + ### Execute (re)computation To test the example, run: diff --git a/examples/fmriprep-singularity/README.md b/examples/fmriprep-singularity/README.md index f2a7a5c..a57fff5 100644 --- a/examples/fmriprep-singularity/README.md +++ b/examples/fmriprep-singularity/README.md @@ -18,7 +18,7 @@ It is assumed that the license file is located in `/tmp`. Make sure to copy it t ## How to install -Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). +Install `datalad-remake` extension, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#installation). Make sure that you have a valid GPG key and that you have successfully configured Git for commit signing, as described [here](https://github.com/datalad/datalad-remake/tree/main?tab=readme-ov-file#requirements). ## How to use @@ -73,6 +73,14 @@ Place the `input.txt`, `output.txt` and `parameter.txt` files in the root datase > cp $EXAMPLE/*.txt ./code/make/fmriprep-singularity/ ``` +### Configure trusted keys + +Configure trusted keys, by executing the command below. Replace `` with a GPG key that you have used for signing commits. For more details, please go [here](https://github.com/datalad/datalad-remake#trusted-keys). + +```bash +> git config --global --add datalad.make.trusted-keys +``` + ### Execute (re)computation To test the example, run: