Does dandi organize work for BIDS datasets?

[kabilar]

Hi team,

From what I understand about dandi organize, it is meant to reorganize a dandiset directory based on the metadata in the NWB file (Reference: docstring).

Does dandi organize also work for BIDS datasets? Was receiving the error below for a BIDS dataset:

(dandi) % pwd
/Users/kabilar/Desktop/000724
(dandi) % ls
dandiset.yaml
(dandi) % dandi organize ../example 
2023-12-14 19:54:33,470 [ WARNING] A newer version (0.58.2) of dandi/dandi-cli is available. You are using 0.58.1
2023-12-14 19:54:34,598 [    INFO] Loading metadata from 0 files
[Parallel(n_jobs=-1)]: Using backend LokyBackend with 10 concurrent workers.
2023-12-14 19:54:34,604 [    INFO] Logs saved in /Users/kabilar/Library/Logs/dandi-cli/20231215015431Z-38483.log
Error: symlink: src should be string, bytes or os.PathLike, not NoneType

If not, I can add a separate section on uploading data for the BIDS format which will not include the dandi organize step. Current upload steps:

dandi download https://dandiarchive.org/dandiset/<dataset_id>/draft
cd <dataset_id>
dandi organize <source_folder> -f dry
dandi organize <source_folder>
dandi validate .
dandi upload

Thank you.

[satra]

yes, organize only works on nwb files/collections.

[kabilar]

Thank you, @satra.

[yarikoptic]

there is a script @bendichter has prototyped (this gist?) and we should generalize it to be supported by dandi organize: filed dandi/dandi-cli#1378 . We do mention various BIDS converters on https://www.dandiarchive.org/handbook/13_upload/#data-uploadmanagement-workflow and the say (though only after listing commands) that

Note that the organize steps should not be used if you are preparing a BIDS dataset with the NWB files.

So we might want to make it clearer, but in general: "information is there"

[satra]

however, i believe @kabilar is asking for microscopy and MRI data, not neurophysiology data. and organizing these data into bids is non-trivial for the generators at the moment (most are not dicom based and often are derivatives).

[kabilar]

Thank you, @yarikoptic @satra. That is a good catch. I did miss that note. I should have specified more, but as @satra mentioned I am looking to upload BIDS datasets with microscopy and MRI data, and their derivatives.

I will see if I can update the docs to handle the three cases of data that we have discussed: NWB, NWB+BIDS, BIDS (e.g. NIfTI, OME-Zarr, derivatives). I was stumped for a bit with the error message above and I had a user also reach out to me about the same error, so hoping that further clarification in the docs will help others.

Perhaps we will also need to update our front-end as the instructions show up as below?