3D Visualizer Workflow
We developed an open-source web-based multidimensional 3D visualizer to visualize the 3D nature of imaging obtained from confocal laser scanning and focused-ion beam electron microscopy modalities on any web browser. In our 3D visualizer platform, we have implemented three distinct visualizers: 1). 3D (three-dimensional view), 2). Tri-Planar (cross-sectional view), and 3). MPR (orthogonal viewer). The 3D viewer is further separated into two viewers: i). Volume and ii). Surface rendering.
Figure 1: Three-channel CLSM image is rendered into a volume using a volume rendering algorithm.
- As we all know, there is no data file format standardization in the microscope field, as there is in the medical industry, and our previous study discovered that there are more than dozens of file formats in the microscope field, and designing a visualizer for each file format is not viable. As a result, to reduce interoperability issues between scanners while also reducing proprietary issues within their file formats, we previously developed a conversion pipeline [1] that efficiently converts confocal laser scanning and focused ion beam electron microscope imaging modalities into DICOM file standards. Therefore, in a conclusion, our visualizer intakes a series of DICOM files as input.
Note: It is now possible to view images with file extensions such as .dcm, .tif, .tiff, .nii, .bmp, .nrrd, .mha, .jpg, .png, and others.
Figure 2: Shows the process of file conversion between raw and DICOM.
- Below you will find the steps about how to use it:
-
- At the very first step, the user needs to convert their raw data files into any abovementioned file formats if they are not already. There are open-source packages available to accomplish this, such as the OME-Bioformats library, which can read microscope imaging raw data files, and the Pydicom library, which can read/write DICOM files or the user can user other libraries or packages.
-
- Secondly, before uploading files/folders into the system, users must determine how or which viewer they wish to use to visualize their data. Three viewers are provided, each with its unique input intake functions and visualization features. When using the 3D viewer, users may upload several files by manually entering numbers in the select the number of inputs field or spinning the wheel, but in the case of tri-planar and MPR viewers, users are allowed to upload only one file/folder at a time.
-
First, choose the 3D option (if the 3D option is selected then the other two tri-planar and MPR options will be disabled).
-
In the second step, select volume rendering instead of surface since we will be visualizing our data in 3D volume space in this example.
-
In the third step, load the image files. For example, if you have a four-channel CLSM image, first you need to split it into four color channels and save each channel in a separate folder. Likewise, you will have four folders with four channel files and later load them one by one. It does not matter whether the images are a sequence of images or already 3D data.
-
By following the above steps, now we will visualize a four-channel CLSM image in a volume rendering viewer.
Figure 3: Four-channel CLSM image is rendered into a volume using a volume rendering algorithm.
Likewise, the user can follow the same above-described steps to visualise their data in the surface, tri-planar, and MPR viewer.
-
- Volume rendering viewer consists of four features: Image visibility on/off, opacity slider, sample distance slider, and colour.
- Surface rendering viewer consists of four features: Image visibility on/off, opacity slider, Iso-value slider, and colour.
- Tri-Planar viewer consists of the following features: (XY, XZ, and YZ) slice slider, colour level/window slider, and mouse selector (rotate, pan, zoom).
- Multi-planar reconstruction consists of the following features:
- Region of interest clipping features: This feature will work with both volume and surface rendering viewers.
- Z-voxel dimension scaling factor features: This feature is still in the experiment phase, and for now it will only work with one stack of the image. However, the output might not be satisfying.
-
Figure 4: Two-channel CLSM image is rendered into a volume using a volume rendering algorithm.
Figure 5: Two-channel CLSM image is rendered into a surface using a surface rendering algorithm.
Figure 6: CT trauma image is rendered into volume rendering.
Figure 7: MRI brain image is rendered into volume rendering.
Figure 8: CT image is rendered into surface rendering.
Figure 9: FIB-SEM microscope image is rendered into a tri-planar viewer.
Figure 10: FIB-SEM microscope image is rendered into an MPR viewer.
Figure 11: MRI brain image is rendered into an MPR viewer.
Reference:
- Gupta, Yubraj, Carlos Costa, Eduardo Pinho, and Luís Bastião Silva. 2022. "DICOMization of Proprietary Files Obtained from Confocal, Whole-Slide, and FIB-SEM Microscope Scanners" Sensors 22, no. 6: 2322. https://doi.org/10.3390/s22062322
- Y. Gupta, R. E. D. Guerrero, C. Costa, R. Jesus, E. Pinho and L. A. Bastião Silva, "Interactive Web-based 3D Viewer for Multidimensional Microscope Imaging Modalities," 2022 26th International Conference Information Visualisation (IV), Vienna, Austria, 2022, pp. 379-384, https://doi.org/10.1109/IV56949.2022.00069
- Gupta Y, Costa C, Pinho E, A. Bastião Silva L, Heintzmann R (2022) IMAGE-IN: Interactive web-based multidimensional 3D visualizer for multi-modal microscopy images. PLOS ONE 17(12): e0279825. https://doi.org/10.1371/journal.pone.0279825