vulkano-shaders: support glam as a linear algebra backend

[jmi2k]

Fixes #2478.

Things to be considered:

1. glam provides Vec3A and Mat3A types which currently are not handled.

2. Make match arms more readable.

3. Rethink cases marked as unreachable!().

---

1. Update documentation to reflect any user-facing changes - in this repository.

2. Make sure that the changes are covered by unit-tests.

3. Run cargo clippy on the changes.

4. Run cargo +nightly fmt on the changes.

5. Please put changelog entries in the description of this Pull Request
   if knowledge of this change could be valuable to users. No need to put the
   entries to the changelog directly, they will be transferred to the changelog
   file by maintainers right after the Pull Request merge.

   Please remove any items from the template below that are not applicable.

6. Describe in common words what is the purpose of this change, related
   Github Issues, and highlight important implementation aspects.

Changelog:

### Additions
- Support for `linalg_type: "nalgebra"`.

[marc0246]

I'm afraid that this is unsound. There's a reason vulkano-shaders still can't generate glam types even though I would like it to. Namely, unlike cgmath and nalgebra, glam has overaligned types (Vec3A, Vec4, Mat2, Mat3A, Mat4). On the other hand, SPIR-V types can be laid out any which way as long as they are at least aligned to their scalar alignment and appropriate features are enabled.

Examples

Overaligned array elements

#version 460
#extension GL_EXT_scalar_block_layout : enable

struct Example {
    vec4 v;
    float x;
};

layout(binding = 0, scalar) buffer Examples {
    Example examples[];
};

void main() {}

The above GLSL compiles to the following SPIR-V:

; SPIR-V
; Version: 1.0
; Generator: Google Shaderc over Glslang; 11
; Bound: 13
; Schema: 0
               OpCapability Shader
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical GLSL450
               OpEntryPoint Vertex %main "main"
               OpSource GLSL 460
               OpSourceExtension "GL_EXT_scalar_block_layout"
               OpSourceExtension "GL_GOOGLE_cpp_style_line_directive"
               OpSourceExtension "GL_GOOGLE_include_directive"
               OpName %main "main"
               OpName %Example "Example"
               OpMemberName %Example 0 "v"
               OpMemberName %Example 1 "x"
               OpName %Examples "Examples"
               OpMemberName %Examples 0 "examples"
               OpName %_ ""
               OpMemberDecorate %Example 0 Offset 0
               OpMemberDecorate %Example 1 Offset 16
               OpDecorate %_runtimearr_Example ArrayStride 20
               OpMemberDecorate %Examples 0 Offset 0
               OpDecorate %Examples BufferBlock
               OpDecorate %_ DescriptorSet 0
               OpDecorate %_ Binding 0
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
      %float = OpTypeFloat 32
    %v4float = OpTypeVector %float 4
    %Example = OpTypeStruct %v4float %float
%_runtimearr_Example = OpTypeRuntimeArray %Example
   %Examples = OpTypeStruct %_runtimearr_Example
%_ptr_Uniform_Examples = OpTypePointer Uniform %Examples
          %_ = OpVariable %_ptr_Uniform_Examples Uniform
       %main = OpFunction %void None %3
          %5 = OpLabel
               OpReturn
               OpFunctionEnd

The important line being:

               OpDecorate %_runtimearr_Example ArrayStride 20

On the other hand, if you naively generate this Rust code:

#[repr(C)]
struct Example {
    v: glam::Vec4,
    x: f32,
}

#[repr(C)]
struct Examples {
    examples: [Example],
}

Then size_of::<Example> == 32 and the stride of Examples::examples is therefore also 32, not 20.

Overaligned struct offsets

#version 460
#extension GL_EXT_scalar_block_layout : enable

struct Example1 {
    vec4 v;
};

struct Example2 {
    Example1 example1;
};

layout(binding = 0, scalar) buffer Examples {
    float x;
    Example2 example2;
};

void main() {}

The above GLSL compiles to the following SPIR-V:

; SPIR-V
; Version: 1.0
; Generator: Google Shaderc over Glslang; 11
; Bound: 13
; Schema: 0
               OpCapability Shader
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical GLSL450
               OpEntryPoint Vertex %main "main"
               OpSource GLSL 460
               OpSourceExtension "GL_EXT_scalar_block_layout"
               OpSourceExtension "GL_GOOGLE_cpp_style_line_directive"
               OpSourceExtension "GL_GOOGLE_include_directive"
               OpName %main "main"
               OpName %Example1 "Example1"
               OpMemberName %Example1 0 "v"
               OpName %Example2 "Example2"
               OpMemberName %Example2 0 "example1"
               OpName %Examples "Examples"
               OpMemberName %Examples 0 "x"
               OpMemberName %Examples 1 "example2"
               OpName %_ ""
               OpMemberDecorate %Example1 0 Offset 0
               OpMemberDecorate %Example2 0 Offset 0
               OpMemberDecorate %Examples 0 Offset 0
               OpMemberDecorate %Examples 1 Offset 4
               OpDecorate %Examples BufferBlock
               OpDecorate %_ DescriptorSet 0
               OpDecorate %_ Binding 0
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
      %float = OpTypeFloat 32
    %v4float = OpTypeVector %float 4
   %Example1 = OpTypeStruct %v4float
   %Example2 = OpTypeStruct %Example1
   %Examples = OpTypeStruct %float %Example2
%_ptr_Uniform_Examples = OpTypePointer Uniform %Examples
          %_ = OpVariable %_ptr_Uniform_Examples Uniform
       %main = OpFunction %void None %3
          %5 = OpLabel
               OpReturn
               OpFunctionEnd

The important line being:

               OpMemberDecorate %Examples 1 Offset 4

And if you naively generate this Rust code:

#[repr(C)]
struct Example1 {
    v: glam::Vec4,
}

#[repr(C)]
struct Example2 {
    example1: Example1,
}

#[repr(C)]
struct Examples {
    x: f32,
    example2: Example2,
}

Then Examples::example2 is going to have an offset of 16, not 4.

Solutions

As you can probably tell from the above examples, a SPIR-V struct itself doesn't tell us anything about alignment of it or its fields, only at which offsets the fields are. It could therefore happen, as can be seen in the first example, that the vec4 is aligned adequately in its struct, but the array it is used in has a lower stride than if the struct were aligned to 16 bytes. Or the struct can be used in another struct, as can be seen in the second example, which puts it at an offset that is not aliged to 16. As can also be seen in the second example, this can happen with an arbitrary amount of nesting. You can also nest structs in arrays and vice-versa. I only showcased the vector case but the same thing applies to the overaligned matrices.

All in all, if you want to implement this safely, it will essentially require a rewrite of the structs reflection. You will need to keep track of every struct that is encountered so that you can recursively check all other structs it is contained within to determine its alignment. It can also happen that the same struct is used in two other structs, which cause it to have a different alignement for each.

[jmi2k]

Thanks for your in-depth explanation @marc0246. Indeed, the problem is much deeper than I naively expected and I managed to miss the hard mark.

Being completely honest, I don't have the amount of free, uninterrupted time to tackle this myself, yet I want to keep your message as a cautionary tale to whoever finds themselves in the same situation. I'll let you decide what to do with the PR and the issue (either keeping them as-is or closing them)

[marc0246]

The issue should stay open and this PR closed if you don't feel like implementing it, which is completely understandable.