Handle Zarr data that is downsampled in Z

[melissalinkert]

As discussed with @chris-allan.

The test in 09670b0 writes a Zarr dataset that is downsampled in Z; this test should fail without the fix in ae60b7f.

I can add more tests if needed, but wanted to open this first to make sure I'm on the right track.

[sbesson]

Is this the right calculation? Using the example of a resolution 0 with [1, 4, 218, 632, 370] and a resolution 1 with [1, 4, 109, 316, 185], this gives

jshell> int fullResZ = 218;
fullResZ ==> 218

jshell> int arrayZ=109;
arrayZ ==> 109

jshell> int zStep = fullResZ / arrayZ;
zStep ==> 2

jshell> Map<Integer, Integer> zIndexMap  = new HashMap<Integer, Integer>();
zIndexMap ==> {}

jshell> for (int z=0; z<fullResZ; z++) {
   ...> zIndexMap.put(z, z * zStep);
   ...> }

jshell> zIndexMap
zIndexMap ==> {0=0, 1=2, 2=4, 3=6, 4=8, 5=10, 6=12, 7=14, 8=16, 9=18, 10=20, 11=22, 12=24, 13=26, 14=28, 15=30, 16=32, 17=34, 18=36, 19=38, 20=40, 21=42, 22=44, 23=46, 24=48, 25=50, 26=52, 27=54, 28=56, 29=58, 30=60, 31=62, 32=64, 33=66, 34=68, 35=70, 36=72, 37=74, 38=76, 39=78, 40=80, 41=82, 42=84, 43=86, 44=88, 45=90, 46=92, 47=94, 48=96, 49=98, 50=100, 51=102, 52=104, 53=106, 54=108, 55=110, 56=112, 57=114, 58=116, 59=118, 60=120, 61=122, 62=124, 63=126, 64=128, 65=130, 66=132, 67=134, 68=136, 69=138, 70=140, 71=142, 72=144, 73=146, 74=148, 75=150, 76=152, 77=154, 78=156, 79=158, 80=160, 81=162, 82=164, 83=166, 84=168, 85=170, 86=172, 87=174, 88=176, 89=178, 90=180 ...  181=362, 182=364, 183=366, 184=368, 185=370, 186=372, 187=374, 188=376, 189=378, 190=380, 191=382, 192=384, 193=386, 194=388, 195=390, 196=392, 197=394, 198=396, 199=398, 200=400, 201=402, 202=404, 203=406, 204=408, 205=410, 206=412, 207=414, 208=416, 209=418, 210=420, 211=422, 212=424, 213=426, 214=428, 215=430, 216=432, 217=434}

[sbesson]

I was able to get some decent results looking at a tile using the same z,c,t,x,y,w,h value and different resolutions with the following changes

-            int zStep = fullResZ / arrayZ;
             for (int z=0; z<fullResZ; z++) {
-                zIndexMap.put(z, z * zStep);
+                zIndexMap.put(z, Math.round(z * arrayZ / fullResZ));
             }

The exact calculation including whether round is the correct rounding operation should obviously be reviewed

[melissalinkert]

Thanks, that makes sense. Proposed arithmetic fix is in f5cb42d.

[sbesson]

Why the loop over planes?

[sbesson]

Functionally tested with a sample OME-Zarr dataset with the following dimensions for each resolution array

$ find . -iname .zarray -exec cat {} \; | grep -A 6 shape
    "shape": [
        1,
        4,
        218,
        632,
        370
    ],
--
    "shape": [
        1,
        4,
        109,
        316,
        185
    ],
--
    "shape": [
        1,
        4,
        55,
        158,
        93
    ],

After updating the omero-zarr-pixel-buffer JAR under both the server and the image-region micro-service, it was possible to navigate through all Z-sections of all resolutions of the imported OME-Zarr

The proposal is to get these improvements merged into the development line and schedule their deployment on development instances for additional feedback/testing.

Follow-up investigation should also review a few aspects that were not covered by the initial testing above including:

• review the performance of reading 2D downsampled OME-Zarr datasets
• ZarrPixelBuffer.getStackDirect and ZarrPixelBuffer.getTimePointDirect

[sbesson]

Went back and forth on whether this should become a Map<Integer, Map<Integer, Integer>> and store all the mappings for all the resolutions once rather than invalidating them and recomputing every time setResolutionLevel is called.

At least in the context of the image-region endpoints invoked by a viewer like PathViewer, my understanding is that each resolution call will initialize its own ZarrPixelBuffer in a separate thread. So the value of such computation would be limited.