FEATURE REQUEST: BioFormats support for CZI Multi Scene images

[MichaelDausmann]

In Short
Zeiss Scanners produce multi-scene images and while BioFormats can read these files, they cannot be viewed together in a 'Whole Slide' view which is acceptable to pathologists. Attempts to 'merge' CZI files prior to loading into Omero were only partially successful and in some cases still not acceptable to pathologists.

In Detail
Scanning software for ZEISS Axioscan scanners permits users to mark out discreet and discontiguous regions of tissue on a slide for detailed scanning. The software bundles the information for these different areas into discreet 'Scenes' in the CZI Image format. The 'Scene' is indicated with an integer value in the 'S' coordinate dimension of the file format against each sub-block of the image.

In addition to having a different 'S' dimension, sub-blocks in each scene are recorded at various resolutions or 'Zoom' levels such that the blocks form an image pyramid. The image pyramid for each scene, while using the same set of zoom levels (e.g. 1/1,1/2.....1/64 etc), are discreet to each scene and the sub-blocks at each zoom level only include pixels from that scene. While the file format enforces that sub-blocks at the '0' zoom level (i.e. zoom 1/1... the most detailed images or the 'base' of the image pyramid) do not overlap, this is not true of sub-blocks at lower resolution such that, if Scenes are close together, as you 'zoom out'. the less detailed sub-blocks for adjacent scenes may indeed overlap.

Compositing a single view across the scenes at these lower resolutions therefore becomes problematic. The raw image data is bitmap data which does not support transparency however in addition to the bitmap data, the CZI format is bundled with a 'Mask' or alpha data which indicates which pixels in the bitmap are significant. Correct compositing of CZI multi-scene images at lower resolutions therefore requires consumers of the file to correctly interpret this 'Mask' information. It is notable that Not even the libCZI library provided by Zeiss can support this compositing.

The Current and proposed OME reader interpret scenes as different 'Series' and makes no attempt to combine them. When loaded into Omero, different scenes can only be viewed discretely, not all together in a 'Whole Slide' view. This is not acceptable to most Pathologists, in particular when they need to asses quantitative outcomes like total % necrosis in the tissue section.

Other software e.g. FIJI, are similarly limited in how they can open and view CZI Files due to this limitation in the BioFormats reader.

Workaround - Scene Merging pre-BioFormats/Omero
I have attempted with some help to create a 'RemoveScenes' utility which attempts to merge scenes in a CZI file by setting the Scene dimension of all Scenes to 0. In addition, the RemoveScenes utility performs a crude pyramid normalisation step to force all scenes to have the same number of zoom levels, essentially by discarding sub-blocks at lower resolutions. The utility also alters the image metadata (indicating the length of the 'S' dimension=1) so that the file is considered 'valid' and can be ready by the BioFormats reader.

CZI files merged using this utility can indeed be imported into Omero as a single 'Series' and viewed in Omero in a 'Whole Slide' view, however because the BioFormats reader does not interpret the mask information, rendering of images with closely adjacent scenes causes tiles to 'glitch' in and out as you zoom into and out of the image. This artifact is distracting and unacceptable to pathologists.

In order for the Scene Merging utility to produce an acceptable result file, the utility would need to discard all sub-blocks not at the 1/1 zoom level, then remove the scene information, then build up the image pyramid of the combined scenes from the ground up. While this seems feasible, it is not simple.

Disambiguation: Autostitch
The Bioformats reader does support an autostitch feature which can be enabled/disabled. Autostitch does not stitch or merge scenes together.

More Links
Forum Discussion: https://forum.image.sc/t/zoom-from-overview-to-detailed-scan-for-imported-czi-files/85002
Example multi-scene CZI file: https://zenodo.org/records/8423633
Alternate Reader: #4092
AutoStitch Fails issue (should be closed in favour of this): #4102
Zeiss issue explaining issue with overlapping pyramids and masking: zeiss-microscopy/libCZI#25
CZI Documentation: https://zeiss.github.io/libczi/index.html

[NicoKiaru]

Side note, I've added an option to stitch scenes in the alternate reader:
https://github.com/BIOP/quick-start-czi-reader/blob/32c6b6fafdd44a554127c6a33b6091cc6d3d9cad/src/main/java/ch/epfl/biop/formats/in/ZeissQuickStartCZIReader.java#L239

It works by 'cropping' the resolution level of all scenes to make sure that all data remain there. One issue though is that currently this option is not accessible in the GUI. Also, I'm still unsure how these bio-formats options can be activated within OMERO.

Note: if you want to test stitching in the alternate reader, you have to activate the Zeiss Quick Start CZI Reader update site, and then execute this groovy script:

ij.Prefs.set("bioformats.enabled.ZeissCZI", false);
ij.Prefs.set("bioformats.enabled.ZeissQuickStartCZI", true);

ij.Prefs.set("bioformats.zeissczi.allow.autostitch", true);

ch.epfl.biop.formats.in.ZeissQuickStartCZIReader.STITCH_SCENES_HACK = true

[MichaelDausmann]

thanks @NicoKiaru, great news on the scene stitching feature. I was able to install latest version of the quick start reader in Fiji and load my problem file. Looks like your 'cropping' solution ran into the same issue with compositing lower resolution layers that I mentioned above

Unfortunately I don't have a test file that exhibits this issue that I can share but will work with my Histopathology department to organise one.

[NicoKiaru]

Looks like your 'cropping' solution ran into the same issue with compositing lower resolution layers that I mentioned above

Indeed! And I've never seen any trace of any alpha channel on the java side of things. Without that information the remaining way is probably, as you mentioned, to get the layout of resolution level 0 and be clever when writing pixels at lower resolution levels. I'll open an issue - but I won't have any time soon to work on that. An example file would be nice.