Sprite masks

[colinjneville]

Objective

Adds the ability to mask Sprites with other Images.
Implements #4210

Solution

Usage

• Spawn an Entity with Mask containing a handle to the Image. Most of the other fields work like the ones from Sprite.
• Spawn a SpriteBundle(s) with Masked components taking the Entity with the Mask component.

A Mask will be applied to all rendered Masked Sprites pointing to it. A Sprite can only be affected by a single Mask.

Pipeline implementation

• Masked sprites add 2 additional specialized pipelines for with and without threshold values.
• The mask texture and sampler form uniform group(2), optional uniforms (i.e. just threshold currently) form group(3). Adds SIXTEEN_BYTE_ALIGNMENT define to align uniforms when targeting WebGL2.
• Masked sprites use an alternative vertex buffer that also includes the mask's affine transformation.
• Sprite batches are now delineated by a change in either the Sprite Image or Mask.

Shader implementation

• In the vertex shader, for each Masked vertex, calculate the effective Mask uv coordinates at that point. (If the vertex is not contained within the Mask bounds, the UVs will be outside of the 0 to 1 range and use the sampler's AddressMode.)
• In the fragment shader, sample the mask texture with these UVs, taking the red value¹
• If a threshold value is set, map this value to 0.0 if it is less than the threshold, otherwise 1.0.
• Multiply the fragment output alpha by this value.

Design concerns

• Linking Masked to Mask with just an Entity is not great, but I don't know of anything better until kinded entities or relations are available? This also makes spawning a self-masking Entity awkward.
• Supporting polygonal masks (discussed in Apply color to sprite according to mask #3357, Provide an ability to define custom bound checking for UI element interaction #5180) in the future would be messy at best with this method.
• Should the names for Mask and Masked be more specific? To differentiate from potential future UI or polygon-based masking?

Sampler AddressModes

Changing the AddressMode of the mask's sampler is useful to set masking state outside of the mask bounds, or to create a repeating mask. Currently, setting the sampler can't be be done as a part of Image loading, it must be done after loading is complete #6893. The recent asset loading changes could make this easy to allow with ImageLoaderSettings, if ImageSampler were serializable.

Note, AddressMode::ClampToBorder is not currently supported by WebGPU (gpuweb/gpuweb#1305), though it can be replicated with ClampToEdge and adding a slight border to the mask.

---

Changelog

• Added the Mask and Masked components. Sprites with Masked components will be masked by the linked Mask entity.

Migration Guide

• Added fields to ExtractedSprite, ExtractedSprites, SpritePipeline, SpriteMeta, SpriteBatch, and ImageBindGroups.

Footnotes

1. To allow single-channel grayscale masks ↩

[github-actions]

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

[github-actions]

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

[superdump]

My first thought is that this should be implemented like alpha mask for 3D. But, if you want to do masking of transparent materials separate to its transparency, then a different method is needed indeed.

[alice-i-cecile]

@colinjneville can you resolve merge conflicts? I'd like to wrangle review effort for 0.13

[BD103]

Making this up for adoption, after some discussion on Discord.

[BenjaminBrienen]

triage:

Still up for adoption, no change