Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EXT_materials_bump #2339

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

bhouston
Copy link
Contributor

@bhouston bhouston commented Oct 22, 2023

This is a custom extension that we are using. Figured it may be of interest to others.

Implemented for ThreeJS here: mrdoob/three.js#27036

Benefits of bump maps are primarily these two:

  1. Bump maps are much more compact that normal maps and lead to smaller glTF files as well as less GPU memory requirements. This is because the normal map has to store a full 3D vector per texel, while the bump map only needs to store a single scalar value per texel. Thus bump maps require 3 times less data than normal maps for transmission and GPU storage. Normal maps are also very sensitive to texture compressions methods, while bump maps are less sensitive.

  2. Bump maps are useful for capturing small details in combination with a normal map for large scale structures. For example, when rendering human faces, there are usually two layers of surface pertubation detail, the unwrapped normal map is used for representing wrinkles, and then a repeating bump map is used for skin pore details. This is also the case for many other materials such as wood, metal, and stone, where there is both macro and micro details.

I can also add code to derive a normal map from a bump map and vice versa to the extension text. Normal and bump maps are interchangeable for the most part.

@bhouston
Copy link
Contributor Author

Example glTF that uses this extension exported via ThreeJS's glTF exporter is here: https://drive.google.com/file/d/1NX_LGF0Yp_F5ceWhStcoYbH_UGpuhFqz/view?usp=share_link. You can also import it into the ThreeJS editor via the ThreeJS glTF importer. I can contribute this example to glTF-SampleAssets if desired.

@bhouston
Copy link
Contributor Author

@donmccurdy would it make you feel better if I rename this to be EXT_materials_displacement? And then I can add in text that says this can be rendered either as normal perturbation method or as a surface displacement? I think the main difference is that if I call it EXT_materials_displacement, the displacementScale factor can go above 1. Let me know...

@donmccurdy
Copy link
Contributor

My main concern would be (potentially) causing bump maps to become much more common in the ecosystem. There is currently no other way to achieve detail maps in glTF, so tools would have to implement an outdated feature (bump maps) in order to support a very useful modern feature (detail maps). Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

Perhaps I'm overthinking the risk. I don't need to block this if others are happy with the proposal! I see displacement and parallax maps more positively.


Aside: I'd be very interested in the MaterialX story for vertex displacement and animation. Does that mean MaterialX input to a displacementTexture socket, or is it a separate thing? But maybe off topic here. 😅

@bhouston
Copy link
Contributor Author

@donmccurdy wrote:

Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

You do realize that right now Sketchfab just discards the bump map right if it exists? They also support, like ThreeJS both a normal and a bump. So we currently just have information loss. This addresses that.

@bhouston
Copy link
Contributor Author

My main concern would be (potentially) causing bump maps to become much more common in the ecosystem.

It is possible to convert on load to a normal map from a bump map. So you can stick with normal mapping through out your engine if you so desired.

I'd be very interested in the MaterialX story for vertex displacement and animation.

That is great too!

I guess I do not see this as a step backwards, but rather just supporting what is already in the ecosystem.

@bhouston
Copy link
Contributor Author

bhouston commented Oct 24, 2023

so tools would have to implement an outdated feature (bump maps) in order to support a very useful modern feature (detail maps)

BTW MaterialX supports bump maps: https://materialx.org/docs/api/class_height_to_normal_node_glsl.html. Pixar Renderman also has bump map support: https://rmanwiki.pixar.com/display/REN24/PxrBumpMixer (which was all reworked based on new 2020-era research outlined here: https://jcgt.org/published/0009/03/04/). V-Ray and Arnold and Blender also support bump maps.

@echadwick-artist
Copy link
Contributor

echadwick-artist commented Oct 24, 2023

As far as packing goes, I compiled a list of which channels are still unused by extensions which pack textures. However there's the downside of potentially incurring more compression noise from having disparate data in neighboring channels. /shrug

KHR_materials_anisotropy

RGB = anisotropyTexture
Alpha = unused

KHR_materials_clearcoat

RGB = clearcoatNormalTexture
Red = clearcoatTexture
Green = clearcoatRoughnessTexture
Blue = unused
Alpha = Unused

KHR_materials_iridescence

Red = iridescenceTexture
Green = iridescenceThicknessTexture
Blue = unused
Alpha = Unused

KHR_materials_sheen

RGB = sheenColorTexture
Alpha = sheenRoughnessTexture

KHR_materials_specular

RGB = specularColorTexture
Alpha = specularTexture

KHR_materials_transmission

Red = transmissionTexture

KHR_materials_volume

Green = thicknessTexture
Blue = unused
Alpha = Unused

@echadwick-artist
Copy link
Contributor

As far as packing goes, I compiled a list of which channels are still unused by extensions which pack textures.

If the plan is to pack the bump into a shared texture, I would prefer we use the Blue channel, since there are three textures where it can fit.

@lexaknyazev
Copy link
Member

If the plan is to pack the bump into a shared texture, I would prefer we use the Blue channel, since there are three textures where it can fit.

Adding an unrelated blue channel to already uncorrelated two-component textures would greatly increase compression artifacts, in some cases preventing use of compressed data altogether.

@bhouston
Copy link
Contributor Author

bhouston commented Oct 26, 2023

@lexaknyazev Threekit now supports this extension and mrdoob over at ThreeJS is also supportive - will merge my PR to ThreeJS once it is merged here: mrdoob/three.js#27036 (comment). This means that we have two implementations.

How do we get this merged into Khronos as a community standard then?

I can also commit to adding it to the @donmccurdy gltf-transform tool as an optional extension there as well if he lets me.

@bhouston
Copy link
Contributor Author

Adding an unrelated blue channel to already uncorrelated two-component textures would greatly increase compression artifacts, in some cases preventing use of compressed data altogether.

I suggest we just use the red channel and do not merge it with any other textures for the time being. I think it doesn't fit well with any of the ones @echadwick-artist mentions - it isn't a natural fit with those extensions anyhow.

@donmccurdy
Copy link
Contributor

donmccurdy commented Oct 26, 2023

Or if Blender and Sketchfab began writing bump maps with EXT_materials_bump, just because it's what the user uploaded, rather than converting to normal maps, that'd seem unfortunate.

You do realize that right now Sketchfab just discards the bump map right if it exists? They also support, like ThreeJS both a normal and a bump. So we currently just have information loss.

I did not; that is unfortunate. 😭

...would it make you feel better if I rename this to be EXT_materials_displacement? And then I can add in text that says this can be rendered either as normal perturbation method or as a surface displacement?

I think it's probably best to keep these as two separate techniques.


If only three.js and ThreeKit are interested in supporting the extension, it should probably use a THREE_* prefix. If more then EXT_* makes sense.

I don't feel strongly enough to say more than I have, and will bow out here. It would be helpful if other vendors could give some rough sense of positive/neutral/negative signal — however you feel is best for your applications and communities. 🙂

@bhouston
Copy link
Contributor Author

bhouston commented Oct 26, 2023

BTW Alban over at Sketchfab liked the extension when I mentioned it to him - finally all those models over at Sketchfab that have bumps can actually export that info: https://twitter.com/benhouston3d/status/1716640855455412305

@lexaknyazev lexaknyazev self-requested a review October 26, 2023 15:41
@bhouston
Copy link
Contributor Author

This is now officially supported in ThreeJS, was merged today. Thus it is supported in 2 separate systems - ThreeJS and Threekit. :)

"description": "The strength of the bump displacement.",
"default": 1.0,
"minimum": 0.0,
"maximum": 100.0,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The maximum limit seems arbitrary. Could it be removed?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest removing the minimum as well to allow for negative scales, like the normalTexture.


## Overview

This extension adds one parameters to the metallic-roughness material: `bump`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: one parameter


This extension adds one parameters to the metallic-roughness material: `bump`.

`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel. The height is then used to perturb the normal direction.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change "grayscale" to "single-channel".


`bump` allows users to perturb the surface normal via a bump map. The bump map is a grayscale texture, where the value of each texel represents the height of the surface at that texel. The height is then used to perturb the normal direction.

Bump maps were introduced into computer graphics in 1978 by Blinn in "Simulation of Wrinkled Surfaces, while normal maps were introduced in 1989 by Cohen et al, "Appearance-Preserving Simplification."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please split the rest of this section (starting from this line) into a new section called "Motivation" with the following text prepended:

This section is non-normative.

Comment on lines +63 to +64
| **bumpFactor** | `number` | The strength of the specular reflection. | No, default: `1.0`|
| **bumpTexture** | [`textureInfo`](https://www.khronos.org/registry/glTF/specs/2.0/glTF-2.0.html#reference-textureinfo) | A texture that defines the strength of the specular reflection, stored in the red (`R`) channel. This will be multiplied by bumpFactor. | No |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Descriptions are clearly from a different extension.

"bumpTexture": {
"allOf": [ { "$ref": "textureInfo.schema.json" } ],
"description": "A texture that defines the displacement in the red channel.",
"gltf_detailedDescription": "A texture that defines the displacement in the red channel. This will be multiplied by bumpFactor."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"bumpFactor" -> "the bump factor"

"default": 1.0,
"minimum": 0.0,
"maximum": 100.0,
"gltf_detailedDescription": "This parameter scales the amount of normal perturbation by the bump map on the surface normal."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please prepend the detailed description with the short description from L10.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants