Include support for neuroglancer url for umembargoed zarr and nifti

[aaronkanzer]

As per @satra's suggestion for consolidation of LINC --> DANDI, here is some consolidation of providing a Neuroglancer EXTERNAL_SERVICE link for public Nifti and Zarr assets at this time.

Cc @kabilar

@yarikoptic @satra @waxlamp -- if you are curious, here is some documentation for how on the LINC side we went about rendering private assets (e.g. in DANDI case, embargoed essentially): https://github.com/lincbrain/linc-archive/blob/0b1fe19cfdb7075b1c5a5d5a47fec702db948fd7/doc/design/linc_permissions.md#cloudfront-distribution-with-origin-access-identity-control-for-relevant-s3-buckets

I'm hesitant to fully deploy the LINC CloudFront strategy to DANDI until further conversations happen with the Neuroglancer folks. The key blocker here with the LINC neuroglancer<>CloudFront strategy is that it requires 1. another fork of neuroglancer to maintain, and 2. will probably break once data is sourced from MIT infra -- thus conversations need to be had with core neuroglancer regarding injection of creds for private assets (not just those that live on Google Cloud Platform, which seems to be the only cloud vendor natively supported)

Happy to expound further here need be -- thanks all

[aaronkanzer]

Example staging URL for convenience: https://deploy-preview-2063--gui-staging-dandiarchive-org.netlify.app/dandiset/215229/draft/files?location=sub-MITU01%2Fses-20210626h17m19s58%2Fmicr&page=1

[waxlamp]

@aaronkanzer, I've started looking at this. I need to grok better what the problem is (and your solution); once I do, I think I have a better general solution for situations like this. Stay tuned...

[aaronkanzer]

@aaronkanzer, I've started looking at this. I need to grok better what the problem is (and your solution); once I do, I think I have a better general solution for situations like this. Stay tuned...

I assume you are alluding to the problem of rendering private zarr in generalneuroglancer at scale? If so, sounds good -- if you'd like a walkthrough of how we set up management of cookies with CloudFront and neuroglancer, happy to do so

Cc @kabilar

[aaronkanzer]

@waxlamp et. al

For further context, we had this thread with the Google folks prior: google/neuroglancer#507

I wasn't crazy about a solution that exposes the AWS Keys in plaintext in the URL, or having to set up Cognito (which I'm not certain would've effectively solved our problem either), thus the CloudFront solution seemed sensible -- not to mention, we got the performance boost of CloudFront as a CDN (we have frequent Neuroglancer LINC users in Europe), and haven't seen any major cost constraints

[yarikoptic]

I think we should aim for adjusting the "schema" here to reflect this new need -- that we need

Suggested change

	endpoint: '', // defaults to redirectNeuroglancerUrl logic
	item_handler: 'redirectNeuroglancerUrl',

and then use that in the service code?

[yarikoptic]

as for embargoed -- we are providing URLs unconditionally ATM but they would not even work for embargoed. Filed more generic:

• Disable external services for embargoed data/dandisets #2065
• Provide access for external services to embargoed content #2066