Anywidget static assets

[jpambrun]

Hi,

I thought anywidget + marimo looked very interesting and wanted try to ingrate cornerstonejs, a medical image viewing component. I can create a _esm bundle that works to an extent, but it requires webworkers that it tries to load from new Worker(new URL("./decodeImageFrameWorker.js", import.meta.url), { type: "module" }); This resolves to /@files/decodeImageFrameWorker.js. This @file seems to be marimo's way to access "data outputs". What is the recommended way to access static assets like images/workers/etc? Am I missing something?

[mscolnick]

We have some private methods for registering the file to the '/@file' route. We can look into exposing some as public api. Or you could inline your worker script - if your bundler supports it. If you can share your bundler or setup, I can potentially give you a recommendation/snippet, if you want to take that approach.

[jpambrun]

I am using esbuild very naively (see below), but the challenge is here. I doubt I could get them to inline the worker source, but, even if I did, my next hurdle will be wasm binary blobs.

esbuild --bundle --external:fs --external:path --format=esm \
     --outdir=build \
     dicom_widget.mjs \
     decodeImageFrameWorker=node_modules/@cornerstonejs/dicom-image-loader/dist/esm/decodeImageFrameWorker.js

import {
  init as dicomImageLoaderInit
}  from '@cornerstonejs/dicom-image-loader';

import {
  RenderingEngine,
  Enums,
  imageLoader,
  metaData,
  volumeLoader,
  init as coreInit,
}  from '@cornerstonejs/core';


async function render({ model, el }) {
  await coreInit();
  await dicomImageLoaderInit();

  const element = el;
  console.log(model);
  console.log(el);
  const renderingEngineId = 'myRenderingEngine';
  const renderingEngine = new RenderingEngine(renderingEngineId);
  const viewportId = 'CT_AXIAL_STACK';
  el.style.height = '512px';

  const viewportInput = {
    viewportId,
    type: Enums.ViewportType.STACK,
    element,
    defaultOptions: {
      background: [0.2, 0, 0.2],
    },
  };

  renderingEngine.enableElement(viewportInput);
  const viewport = renderingEngine.getViewport(viewportId);

  const imageIds = ["wadouri:https://raw.githubusercontent.com/OHIF/viewer-testdata/refs/heads/master/dcm/Juno/1.3.6.1.4.1.25403.345050719074.3824.20170125113420.3"]

  viewport.setStack(imageIds, 0);
  viewport.render();


}
export default { render };

[jpambrun]

More broadly, for anywidget to work across runtimes (i.e. jupyter, marimo, etc) I think it needs a standard way to serve static assets like image, icons, wasm, webworkers, etc.

For some context, I first started with esm.sh, but I hit a unexpected snag where webworkers just can't be served from another origin, and apparently no amount of cors headers can help. I hacked a reverse proxy together to make is seems like it came from the same origin and got past that. Of course this would make distributing this widget impossible. Regardless, I stumbled on another problem with the esm.sh build where some circular (?) dependency in cornerstone cause some issue where global variable to a module are split across 2 instances and are not really thus global breaking it. I think esbuild doesn't suffer from that, but now I can't load the worker from the local filesystem.

[mscolnick]

yea web-workers are difficult to distribute with a CDN.

@manzt may have more insight/ideas regarding serving assets with anywidget. (and maybe already scoped out, just needs ecosystem authors to implement it)