-
-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Cannot display Float32 images on iOS Safari #3027
Comments
Safari on iOS doesn't report the |
ha, actually the code produces a warning "failed to load OES_text_float_linear", but because I don't have access to the web inspector on the iPhone I couldn't see it 😅 |
A question I'm asking myself about the fallback mentioned above: is it ever desirable to truncate the items of a Float32Array to Uint8? If this happens "under the hood", magically, how can the client know and about it and adapt the window values accordingly? |
I'm using the following patch for now, which seems to fix things for the images I care about. I'm not sure if it's good enough as a general solutions though... The patch is against the built version of vtk.js, so it's compatible with yarn patch or pnpm patch: diff --git a/Rendering/OpenGL/Texture.js b/Rendering/OpenGL/Texture.js
index a48e383723bf3e32afaf03f5e7c6d866b3e4cbbd..b1e091d9f27d2b8a77de63a14160775702cb3c67 100644
--- a/Rendering/OpenGL/Texture.js
+++ b/Rendering/OpenGL/Texture.js
@@ -932,8 +932,8 @@ function vtkOpenGLTexture(publicAPI, model) {
}
function processDataArray(dataArray, preferSizeOverAccuracy) {
const numComps = dataArray.getNumberOfComponents();
- const dataType = dataArray.getDataType();
- const data = dataArray.getData();
+ let dataType = dataArray.getDataType();
+ let data = dataArray.getData();
// Compute min max from array
// Using the vtkDataArray.getRange() enables caching
@@ -955,6 +955,12 @@ function vtkOpenGLTexture(publicAPI, model) {
if (!model.useHalfFloat) {
publicAPI.getOpenGLDataType(dataType, true);
}
+
+ if (dataType === VtkDataTypes.FLOAT && !!model.oglNorm16Ext && !model.context.getExtension('OES_texture_float_linear') && !minArray.some(x => x < -32767) && !maxArray.some(x => x > 32767)) {
+ data = new Int16Array(data);
+ dataType = VtkDataTypes.SHORT;
+ }
+
return {
numComps,
dataType, The code should probably be moved to |
@abustany, thanks for all the context it is super useful!
Thanks for looking into it and hope you might be able to share some additional info - either way have a great day! |
Floating point textures have limited support for Safari on iOS. See https://developer.mozilla.org/en-US/docs/Web/API/OES_texture_float_linear
In your patch you're choosing the type based on value range. Instead, I would cast the data to |
@sankhesh thanks for the feedback! Yes, that would be better, but then I need to scale the window/levels based on those shift/scale values in the shader too right? The model you describe is superior, but sounded a bit more complex to implement → I didn't go down that route because I wanted a quick fix yesterday 😬 I might have more time after this month to try to implement a proper fix, but if the current fix is "good enough" for my usecase I can't guarantee I'll have time to implement that proper fix (1-person engineering team here 😅). |
The Pixel 3a I tested on supported float textures, but I suppose this will vary across devices.
Yes, we are using Vite. It works just fine with VTK, but will give you headaches with Cornerstone for various reasons: circular dependencies that Rollup can't bundle correctly, webpack-specific web worker loading, and broken webpack automatic public path detection. There are workarounds for all of those, happy to help if needed.
I only use stack images so far, so I can't speak for volumes sorry 😬 |
…-load-issue WON-3206: apply unofficial patch fix for float issue on ios Kitware#3027 (comment)
Based on my investigation the issue is Float Linear interpolation (OES_texture_float_linear) is not supported on iOS devices and was incorrectly reported as supported up until recently in iOS 17 |
hmm the original bug reports was done on iOS 17, which does not report this extension as far as I remember. As you can see in the hacky patch I sent above, there's a check for OES_texture_float_linear. If I'm not mistaken, the issue is rather that vtk.js does not do the proper checks in that specific part of the code, and sends a Float32 texture to the GPU - but it's been couple of weeks since I dealt with this issue so I might remember it wrong. |
I created this PR which would be good enough for a lot of use cases (except the PT Scaled data) |
Reporting that this issue also exists with volumes. Tried to use the patch provided above, the error did not appear in the console anymore, but the volume was still not rendered unfortunately. As expected, volume render does work with other data types on ios: https://kitware.github.io/vtk-js/examples/TestVolumeTypes.html |
The issue is iOS and Safari not us, you can read more here cornerstonejs/cornerstone3D#1212 I'm not sure how to do this honestly, it is a shame that is it not supported in iOS |
Bug description
Note: this bug originally comes from cornerstonejs/cornerstone3D#1133
Cornerstone loads grayscale DICOM images into Float32Array pixel data. Putting this data into a dataArray/imageData/imageMapper/imageSlice fails to display on an iPhone XS running iOS17 (and couple more iPhone models I could check using Browserstack).
Steps to reproduce
https://stackblitz.com/edit/vitejs-vite-gjcs6u?file=src%2Fmain.ts
Detailed Behavior
Image renders correctly on:
Image fails to render on:
Expected Behavior
Image should render identically on all platforms
Environment
The text was updated successfully, but these errors were encountered: