Add SVG support for textures in GLTF specification

[fire]

We believe it is possible to have two implementations: one in three.js and another in Godot Engine.

I am opening this proposal for discussion. Please feel free to share your comments.

[donmccurdy]

Is "SVG as texture" something that is done in engines today? three.js does have an SVG loader, but it generates triangulated geometry, usually as a one-time operation. While we could then draw that geometry to a texture, it may be inefficient as a way to solve the stated goals of this extension. Streaming greater detail at closer distances sounds difficult — more in the territory of OpenSubDiv — and I'm not sure how to implement that on SVG.

Crisp, legible text in a 3D scene is an important problem, and not easily handled in standard formats today. If that's a goal for glTF, I think it would be important that we either (a) offer quality and performance at least as good as SDF/MSDF fonts, or (b) provide the source text and layout hints so that implementations are free to improve over time. I'm worried that transmitting text as SVG would not satisfy either of those goals.

[fire]

Storing the SVG allows for decomposition via importer-time scaling to a raster image, curve conversion to mesh geometry or (multichannel) signed distance fields textures. These method can match SDF/MSDF fonts in quality and performance. However, raw svg is resource-intensive and may not be feasible for real-time applications or systems with limited resources compared to its decomposed forms.

Making mesh geometry efficient is outside the scope of this pr. See meshoptimizer, msft lod and etc. However, it is possible to use a series of LODs to provide performance without using dynamic subdivision on the mesh.

While SVG might not be the optimal format for delivering beziers and raster embeds, its wide availability makes it a practical choice. Another format that is available is lottie for vector graphics animation.

For providing source text and layout hints for future improvements, storing intermediary SVG inputs might not be feasible, especially when they are procedural generated or involve fonts that may not be available on all systems. For reference, PDF/A archive ensures its files are self contained.

Having yet another 2d vector standard inside of GLTF is possible too.

• https://github.com/MerlinVR/Unity-MSDF-Fonts
• Add MSDF and mipmap generation project settings for the default font godotengine/godot#60513
• https://docs.unity3d.com/Manual/com.unity.textmeshpro.html
• https://docs.unrealengine.com/4.27/en-US/Basics/Actors/3DText/
• https://en.wikipedia.org/wiki/PDF/A
• https://github.com/zeux/meshoptimizer
• https://www.thorvg.org/ (lottie and svg)

[fire]

Lottie is in the process of standardization with the Linux Foundation group.

[aaronfranke]

@donmccurdy SVG stands for Scalable Vector Graphics. It is also a 2 dimensional format. While you could triangulate it as a mesh, and another extension could do that, the simplest and most natural way to interpret a 2D graphic is an image.

[elalish]

Personally, I find this most interesting in conjunction with animated textures - this is why we started supporting Lottie animations in <model-viewer>: https://modelviewer.dev/examples/scenegraph/#animatedTexturesExample

For static textures, vector graphics have all the problems @donmccurdy points out, since with traditional rendering you would rasterize the graphic at a fixed resolution ahead of time. However, for animated textures, you are re-rasterizing the textures every frame anyway, which gives potential to also adjust the resolution on the fly. It'll still be resource intensive though.

I know various parties have interest in a video texture extension for glTF - maybe that would be a good place to start talking about the potential for vector formats.

[fire]

I have greatly supported @hermet on thorvg, he is working on standardizing lottie. Perhaps there is interested in a lottie gltf extension.

[hermet]

Hello, in most practical scenarios, rendering SVG/Lottie scenes requires several rendering stages for composition, including blending, masking, and applying layered effects, etc which is significantly more complex than text / video drawing. This means that even if we attempt to draw the vector contents directly on the main surface, it would necessitate generating one or more intermediate frame buffers(textures) for these compositions. Definitely there are pros/cons for introducing texture-based approach, but from my experience, supporting svg/lottie is quite challenging and burdensome due to the extensive nature of its specifications also, as vector contents complexity increases, retaining the scene (texture) for Lottie/SVG can improve performance among various user scenarios.

I think it would be beneficial to discuss whether glTF is likely to support the Lottie feature, thorvg is the most powerful for Lottie support.