Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: force half float in texture if it's exact and OES_texture_float_linear is not available #3117

Merged
merged 2 commits into from
Sep 5, 2024

Conversation

slak44
Copy link
Contributor

@slak44 slak44 commented Aug 28, 2024

Context

The origin of the problem is this bug in OHIF, a consumer of vtk: OHIF/Viewers#4286
For certain mobile Android devices, instead of rendering a texture, the canvas was solid gray/black. It happens regardless of OS version/browser, but eventually I narrowed it down to devices that use Mali GPUs (which is likely because they don't support OES_texture_float_linear).

Since it was working everywhere previously and I had a good version of OHIF, I bisected until this first bad commit, which among other things, bumps the version of vtk.js from 29.3.0 to 29.7.0. After checking all the vtk versions in that range, I found that 29.5.0 was the last good one, and 29.5.1 was the first bad one with gray/black canvases. This issue has been present on master ever since.

Luckily (or unluckily) this was a patch release that only included 2 commits. Of them, the relevant one is this.

The commit changed the texture format to sometimes use *32F versions, and added this warning:
image
The problem with using full floats is in webgl2 they aren't texture filterable without OES_texture_float_linear, while half floats are always filterable even without the extension. Using full floats triggers the warning/black textures.

I assume all of these are the same problem and would be fixed by this change:

Results

With this change, canvases are no longer black in OHIF and render the images again everywhere.

Changes

This change forces half floats in cases where:

  1. Full floats would have been used instead
  2. The extension for texture filtering is not supported in the gl context
  3. The data can be stored accurately in half floats

I don't know if there's any way to solve this if the data doesn't fit in halfs, maybe with another option similar to preferSizeOverAccuracy.

PR and Code Checklist

  • semantic-release commit messages
  • Run npm run reformat to have correctly formatted code

Testing

I only reproduced this on Androids with Mali GPUs. I would assume it happens on any device that supports webgl2 but not OES_texture_float_linear.
If you have such a device, you can see the problem here (a gray square is rendered, instead of an image of a brain CT): https://viewer.ohif.org/viewer?StudyInstanceUIDs=2.16.840.1.114362.1.11972228.22789312658.616067305.306.2

For testing the changes, the easiest way I know is to clone, compile and run OHIF, and configure resolutions in its package.json to use these changes in vtk:

  "resolutions": {
    "@kitware/vtk.js": "link:../vtk-js/dist/esm",

Then open this: http://localhost:3000/viewer?StudyInstanceUIDs=2.16.840.1.114362.1.11972228.22789312658.616067305.306.2

With the changes enabled you should see the brain CT, without the changes you should not.

…linear is not available

Half floats are always texture filterable in webgl2, while full *32F floats are not (unless the
extension is present)
@floryst floryst requested a review from sankhesh August 30, 2024 21:23
Copy link
Collaborator

@sankhesh sankhesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@floryst floryst added this pull request to the merge queue Sep 5, 2024
Merged via the queue into Kitware:master with commit 0a4d8ab Sep 5, 2024
3 checks passed
Copy link

github-actions bot commented Sep 5, 2024

🎉 This PR is included in version 32.1.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot added the released Automated label label Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released Automated label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants