Skip to content

Commit

Permalink
Remove analysis module (#166)
Browse files Browse the repository at this point in the history
* remove analysis module

* move scripts from /examples to /scripts

* clean up requirements

* pytest ignote scripts/

* move acquisition_settings to examples

* Update README.md

* update paper link

* add note on biahub installation
  • Loading branch information
ieivanov authored Dec 17, 2024
1 parent faa33db commit 253b4a3
Show file tree
Hide file tree
Showing 73 changed files with 15 additions and 6,720 deletions.
116 changes: 8 additions & 108 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ shrimPy (pronounced: ʃrɪm-pai) is a pythonic framework for high-throughput sma

The acquisition engine synchronizes data collection using hardware triggering and carries out smart microscopy tasks such as autofocus and autoexposure.

The acquired multidimensional raw datasets are processed by the reconstruction engine to generate registered multimodal data that can be used for analysis. Raw data are first converted to the [OME-Zarr](https://ngff.openmicroscopy.org/) format using [iohub](https://github.com/czbiohub-sf/iohub) to facilitate parallel processing and metadata management. Discrete data volumes then undergo deskewing of fluorescence channels, reconstruction of phase and orientation (using [recOrder](https://github.com/mehta-lab/recOrder)), registration and virtual staining (using [VisCy](https://github.com/mehta-lab/viscy)).
The acquired multidimensional raw datasets are processed with the [biahub](https://github.com/czbiohub-sf/biahub) library to generate registered multimodal data that can be used for analysis. Raw data are first converted to the [OME-Zarr](https://ngff.openmicroscopy.org/) format using [iohub](https://github.com/czbiohub-sf/iohub) to facilitate parallel processing and metadata management. Discrete data volumes then undergo deskewing of fluorescence channels, reconstruction of phase and orientation (using [recOrder](https://github.com/mehta-lab/recOrder)), registration and virtual staining (using [VisCy](https://github.com/mehta-lab/viscy)).

This version of the code still uses the legacy name `mantis`, which overlaps with the name of the microscope which is used to acquire data. In a future release we will transition the codebase to the name `shrimPy`.

Expand All @@ -26,8 +26,12 @@ conda activate mantis
pip install .
```

3. Optionally, install the [biahub](https://github.com/czbiohub-sf/biahub) image analysis library.

`biahub` is currently used when characterizing the microscope point spread function, and will be used for real-time image processing in the future.

## Setting up the mantis microscope
The mantis microscope implements simultaneous label-free and light-sheet imaging as described in [Ivanov et al.](https://www.biorxiv.org/content/10.1101/2023.12.19.572435v1) The two imaging modalities are acquired on two independent arms of the microscope running separate instances of [Micro-Manager](https://micro-manager.org/) and [pycromanager](https://pycro-manager.readthedocs.io/). shrimPy was developed to enable robust long-term imaging with mantis and efficient analysis of resulting TB-scale datasets.
The mantis microscope implements simultaneous label-free and light-sheet imaging as described in [Ivanov et al.](https://doi.org/10.1093/pnasnexus/pgae323) The two imaging modalities are acquired on two independent arms of the microscope running separate instances of [Micro-Manager](https://micro-manager.org/) and [pycromanager](https://pycro-manager.readthedocs.io/). shrimPy and [biahub](https://github.com/czbiohub-sf/biahub) were developed to enable robust long-term imaging with mantis and efficient analysis of the resulting TB-scale datasets on a high-performance compute cluster.

The [Setup Guide](docs/setup_guide.md) outlines how the mantis microscope is configured.

Expand All @@ -47,7 +51,7 @@ Data are acquired using `mantis run-acquisition`, and a list of arguments can be
mantis run-acquisition --help
```

The mantis acquisition is configured using a YAML file. An example of a configuration file can be found [here](mantis/acquisition/settings/example_acquisition_settings.yaml).
The mantis acquisition is configured using a YAML file. An example of a configuration file can be found [here](examples/acquisition_settings/example_acquisition_settings.yaml).

This is an example of a command which will start an acquisition on the mantis microscope:

Expand All @@ -68,111 +72,7 @@ mantis run-acquisition \

## Data reconstruction

Data reconstruction also uses a command line interface. All reconstruction calls take an input `-i` and an output `-o`, and most reconstruction calls use configuration files passed via a `-c` option.

A typical set of CLI calls to go from raw data to registered volumes looks like:

```sh
# CONVERT TO ZARR
iohub convert \
-i ./acq_name/acq_name_labelfree_1 \
-o ./acq_name_labelfree.zarr \
iohub convert \
-i ./acq_name/acq_name_lightsheet_1 \
-o ./acq_name_lightsheet.zarr

# DECONVOLVE FLUORESCENCE
# estimate PSF parameters
mantis estimate-psf \
-i ./beads.zarr \
-c ./psf_params.yml \
-o ./psf.zarr
# deconvolve data
mantis deconvolve \
-i ./acq_name_lightsheet.zarr \
-c ./deconvolve_params.yml \
--psf-dirpath ./psf.zarr
-o ./acq_name_lightsheet_deconvolved.zarr

# DESKEW FLUORESCENCE
# estimate deskew parameters
mantis estimate-deskew \
-i ./acq_name_lightsheet.zarr/0/0/0 \
-o ./deskew.yml
# apply deskew parameters
mantis deskew \
-i ./acq_name_lightsheet.zarr/*/*/* \
-c ./deskew_params.yml \
-o ./acq_name_lightsheet_deskewed.zarr

# RECONSTRUCT PHASE/BIREFRINGENCE
recorder reconstruct \
-i ./acq_name_labelfree.zarr/*/*/* \
-c ./recon.yml \
-o ./acq_name_labelfree_reconstructed.zarr

# STABILIZE
# estimate stabilization parameters
mantis estimate-stabilization \
-i ./acq_name_labelfree.zarr/*/*/* \
-o ./stabilization.yml \
--stabilize-xy \
--stabilize-z
# stabilize data
mantis stabilize \
-i ./acq_name_labelfree.zarr/*/*/* \
-c ./stabilization.yml \
-o ./acq_name_labelfree_stabilized.zarr/*/*/*

# REGISTER
# estimate registration parameters
mantis estimate-registration \
-s ./acq_name_labelfree_reconstructed.zarr/0/0/0 \
-t ./acq_name_lightsheet_deskewed.zarr/0/0/0 \
-o ./register.yml
# optimize registration parameters
mantis optimize-registration \
-s ./acq_name_labelfree_reconstructed.zarr/0/0/0 \
-t ./acq_name_lightsheet_deskewed.zarr/0/0/0 \
-c ./register.yml \
-o ./register_optimized.yml
# register data
mantis register \
-s ./acq_name_labelfree_reconstructed.zarr/*/*/* \
-t ./acq_name_lightsheet_deskewed.zarr/*/*/* \
-c ./register_optimized.yml \
-o ./acq_name_registered.zarr

# CONCATENATE CHANNELS
mantis concatenate \
-c ./concatenate.yml \
-o ./acq_name_concatenated.zarr

# STITCH
# estimate stitching parameters
mantis estimate-stitching \
-i ./acq_name.zarr/*/*/* \
-o ./stitching.yml \
--channel DAPI
--percent-overlap 0.05
# stitch fields of view
mantis stitch \
-i ./acq_name.zarr/*/*/* \
-c ./stitching.yml \
-o ./acq_name_stitched.zarr/*/*/*
```

## Additional utilities

We have also developed the following CLI utilities to help with data acquisition and reconstruction

```sh
# Estimate fluorescence bleaching
mantis estimate-bleaching --help

# Generate an HTML report with PSF measurements
mantis characterize-psf --help
```
Data reconstruction is accomplished with the [biahub](https://github.com/czbiohub-sf/biahub) library. Visit the link for the latest information on our reconstruction workflows.

## Data and metadata format

Expand Down
13 changes: 0 additions & 13 deletions examples/pvcam_11bit_acq.py

This file was deleted.

84 changes: 0 additions & 84 deletions examples/pvcam_jvm_debugging.py

This file was deleted.

1 change: 0 additions & 1 deletion examples/run_logging.py

This file was deleted.

Loading

0 comments on commit 253b4a3

Please sign in to comment.