Display image overlay annotations

[naglepuff]

Implement client support for image overlay annotation elements.

Changes include:

• Draw image overlays when annotations containing them are selected in the Girder large_image UI
• Remove image overlays when their annotations are deselected
• Support rendering image overlays with opacity specified in the annotation JSON
• Support image overlay offset
• Support image overlay transform via affine matrix

(opening as draft PR to kick start discussion about implementation details)

Closes #678

[manthey]

You'll probably need to merge or rebase on master for CI to have any chance of passing since Pillow 9.0.0 broke some of the tests.

[manthey]

Please remove this.

[naglepuff]

Removed in 5b3f52b

[manthey]

I see two issues. I placed a small image (one using the PIL tile source) as an overlay on a much larger image with an identity matrix specified as the transform.

Internally, I still see transform parameters being applied (gcs: "+proj=longlat +axis=enu +s11=1 +s12=0 +s21=0 +s22=1 +xoff=-0 +yoff=0" -- note the -0.

More importantly, though, the small image is magnified. Fundamentally, the scale is incorrect, since the function that calculates the layer properties is making level 0 tiles appear at the same scale for the two sources. This might be as simple as scaling down the transform matrix based on the number of tile levels between the two images, but it might be more complex than that.

[naglepuff]

Internally, I still see transform parameters being applied (gcs: "+proj=longlat +axis=enu +s11=1 +s12=0 +s21=0 +s22=1 +xoff=-0 +yoff=0" -- note the -0.

This is easy enough to fix. The issue lies with the check matrix === [[1, 0], [0, 1]].

More importantly, though, the small image is magnified. Fundamentally, the scale is incorrect, since the function that calculates the layer properties is making level 0 tiles appear at the same scale for the two sources. This might be as simple as scaling down the transform matrix based on the number of tile levels between the two images, but it might be more complex than that.

I think I see this as well. If I overlay a small image (~4k by 2k) onto a very large image (~90k by 90k), the small image is blown up well beyond the space it should take up. With my test images, the smaller has 6 zoom levels, the large has 10. How does this ratio inform how to appropriately scale the smaller image?

[manthey]

GeoJS has a utility function (pixelCoordinateParams) that assumes the image will take up the whole map space. The correct thing to do is to either have a different function where this assumption is not made, or where we adjust the layer parameter outputs of that function.

Alternately, we can probably twiddle the affine transform to make up for the fact that this isn't so -- by scaling the image by 2**(maxlevel of base image - maxlevel of overlayimage) to account for the number of tile levels difference and by (tile width of base image / tile width of overlay image), (tile height of base image / tile height of overlay image). This will probably just work, but the extending or adjusting the results from pixelCoordinateParams would be "more correct" and probably by slightly more efficient.

I'll see if I can generalize this, as we've wanted it once before on a different project.

[naglepuff]

GeoJS has a utility function (pixelCoordinateParams) that assumes the image will take up the whole map space. The correct thing to do is to either have a different function where this assumption is not made, or where we adjust the layer parameter outputs of that function.

Which parameter outputs would need to be adjusted? tilesAtZoom and tilesMaxBounds?

Alternately, we can probably twiddle the affine transform to make up for the fact that this isn't so

I've been fiddling around with this and it seems to be a decent workaround. However, I'd prefer to do something more "correct." I'm leaning towards adjusting the output of pixelCoordinateParams, as I don't want to increase the scope to include changes to geojs at this point. Let me know if you have any thoughts.

[manthey]

So I dug through an old example I had where I did this somewhere else. There is code below. item is a an object with id as the Girder ID and zoffset the base image's max levels - overlay image's max levels. The big other change is the url isn't just a template string, it is actually a function that returns the url.

    if (item.zoffset) {
      params.layer.url = (x, y, z) => 'api/v1/item/' + item.id + `/tiles/zxy/${z-item.zoffset}/${x}/${y}`;
      params.layer.minLevel = item.zoffset;
      params.layer.maxLevel += item.zoffset;
      params.layer.tilesAtZoom = (level) => {
        var scale = Math.pow(2, params.layer.maxLevel - level);
        return {
          x: Math.ceil(item.meta.sizeX / item.meta.tileWidth / scale),
          y: Math.ceil(item.meta.sizeY / item.meta.tileHeight / scale)
        };
      };
      params.layer.tilesMaxBounds = (level) => {
        var scale = Math.pow(2, params.layer.maxLevel - level);
        return {
          x: Math.floor(item.meta.sizeX / scale),
          y: Math.floor(item.meta.sizeY / scale)
        };
      };
    }

[manthey]

This should NOT be abs -- it could be negative.

[naglepuff]

Fixed in a1743d5

[manthey]

tileWidth and tileHeight are swapped in this call.

[naglepuff]

Fixed in a1743d5