This is an open source repository of the Computational Miniature Mesoscope (CM2) project in the Computational Imaging Systems Lab (CISL) at Boston University. We aim to develop next-generation “wearable” computational fluorescence microscope that achieves a centimeter-scale field-of-view (FOV) and micron-scale resolution with single-shot 3D imaging capability. In this repository, we provide: 1) the hardware design of the CM2 device including 3D printable CAD files of the LED, microlens array housing, and 3D printable freeform illuminators; 2) the Zemax models of the CM2 system including the ZMX files, spectra data and coating profile; 3) the reconstruction algorithms: ADMM-based 3D deconvolution algorithm with calibrated 3D PSFs and sample measurements, and pre-trained CM2Net for fast, near-isotropic 3D reconstructions.
If you find this project useful in your research, please consider citing our paper:
Fluorescence microscopes are indispensable to biology and neuroscience. The need for recording in freely behaving animals has further driven the development in miniaturized microscopes (miniscopes). However, conventional microscopes/miniscopes are inherently constrained by their limited space-bandwidth product, shallow depth of field (DOF), and inability to resolve three-dimensional (3D) distributed emitters. Here, we present a Computational Miniature Mesoscope (CM2) that overcomes these bottlenecks and enables single-shot 3D imaging across a 7-mm field of view and mm-scale DOF, achieving 7-μm lateral resolution and better than 200-μm axial resolution. The CM2 features a compact lightweight design that integrates a microlens array for imaging and a light-emitting diode array for excitation. Its expanded imaging capability is enabled by computational imaging that augments the optics by algorithms. We experimentally validate the mesoscopic imaging capability on 3D fluorescent samples. We further quantify the effects of scattering and background fluorescence on phantom experiments.
Computational Miniature Mesoscope (CM2) is a recently developed computational imaging system that enables single-shot 3D imaging across a wide field-of-view (FOV) using a compact optical platform. In this work, we present CM2 V2 - an advanced CM2 system that integrates novel hardware improvements and a new deep learning reconstruction algorithm. The platform features a 3D-printed freeform LED collimator that achieves ~80% excitation efficiency - a ~3x improvement over our V1 design, and a hybrid emission filter design that improves the measurement contrast by >5x. The new computational pipeline includes an accurate and computationally efficient 3D linear shift-variant (LSV) forward model and a novel multi-module CM2Net deep learning model. As compared to the model-based deconvolution in our V1 system, CM2Net achieves ~8x better axial localization and ~1400x faster reconstruction speed. In addition, CM2Net consistently achieves high detection performance and superior axial localization across a wide FOV at a variety of conditions. Trained entirely on our 3D-LSV simulator generated training data set, CM2Net generalizes well to real experiments. We experimentally demonstrate that CM2Net achieves accurate 3D reconstruction of fluorescent emitters across a ~7-mm FOV and 800-μm depth, and provides ~6-μm lateral and ~25-μm axial resolution. We anticipate that this simple and low-cost computational miniature imaging system will be impactful to many large-scale 3D fluorescence imaging applications.
The directory 'CAD_models' contains the CAD files of the CM2's V1 and V2 systems, including the LED housing, microlens array housing, LED base plate and the freefrom collimator. All CAD models are 3D printable on lab table-top 3D printers. The subdirectory 'assembly' further provides an assembly of the CM2 systems shown as below (note that the sensor is not to-scale).
The part list of all optical and electronic components used in the CM2 prototypes can be found here.
To use the Zemax models, after cloning this repository, copy the coating file "cm2_coating_profiles_ver2.DAT" to the directory "Zemax\Coatings", copy the spectra files "gfp_emission.spcd" and "led_spectrum_interp.spcd" to the directory "Zemax\Objects\Sources\Spectrum Files", copy the CAD files (end with '.stl') to the directory "Zemax\Objects\CAD Files", and then open "CM2_V1_opensource.zos" or "CM2_V2_opensource.zos" in Zemax to view the CM2 design and ray tracing results. Pre-rendered ray tracing data can be downloaded here.
The script "cm2_related_code.m" in the "Algorithm" folder provides a demo of CM2 3D reconstruction pipeline on a simulated measurement using down-sampled PSFs. A full-scale experimental measurement is also provided under the "Algorithm" direcory, which requires large system memory to run. The GIF file below shows the flying-through of a reconstructed 3D object (a fluorescent fiber sample) from an experimental measurement.
The CM2Net is implemented in Python 3.7 with TensorFlow 2.3.
We provide an example of how to use the trained CM2net for fast and robust 3D reconstruction on CM2 measurement. The script “cm2_models.py” and "pred_cm2net.py" in the "Algorithm/cm2net" folder provides the deep learning model and the inference code. The pretrained model, sample measurement and ground truth volume can be found in cm2net data, which requires large system memory to run. The GIF file below shows the reconstructed 3D volume for the sample measurement.
For further information, please feel free to contact us: Guorong Hu ([email protected]), Qianwan Yang ([email protected]), and Prof. Lei Tian ([email protected]).
This project is licensed under the terms of the MIT license. see the LICENSE file for details