Update SPIRV-Tools and fix new validation errors.

[csyonghe]

Closes #6508.

Issue

Recent SPIRV-Tools update adds a new verification rule that disallows explicit layouts (e.g. Offset decoration) on types used from Function/Private/Workgroup etc. address spaces after SPIRV 1.4.

Background

SPIRV makes a distinction of physical and logical types. A physical type is a type with explicit memory layout and can be stored in a user-transparent way in memory. A logical type in contrast does not have a defined layout, and cannot be decorated with any explicit layout deocrations, such as Offset, ArrayStride etc.

The SPIRV spec requires that all buffer elements or global input/outputs to be a physical type, and all variables in the function/private(i.e. ordinary global vars)/workgroup address spaces to be logical.

This means that if the user writes:

struct Foo { int x, y; }

StructuredBuffer<Foo> f;  // or ConstantBuffer, etc.

void main()
{
     Foo localVar;
     localVar = f[0];
     f[1] = localVar;
}

We need to represent Foo as two distinct types when generating SPIRV: the one used inside StructuredBuffer must be a physical type with explicit layout (specifying the offset of Foo::x and Foo::y), and the one used for localVar must be a logical type. Therefore we generate something like:

struct Foo { int x, y; }
struct Foo_std430 { [Offset(0)] int x;  [Offset(4)] int y; }
// If `Foo` is used in a `ConstantBuffer`, we would generate a `Foo_std140` type too.

StructuredBuffer<Foo_std430> f;  // or ConstantBuffer, etc.

Foo_std430 packStorage(Foo f) { ... }
Foo unpackStorage(Foo_Std430 f) {...}

void main()
{
     Foo localVar;
     localVar = unpackStorage(f[0]);
     f[1] = packStorage(localVar);
}

Where Foo_std430 is the physical type for Foo that can be placed in a buffer, and packStorage and unpackStorage functions to convert between physical and logical type when needed.

The above translation is what the existing lowerBufferElementTypeToStorageType is doing.

Currently, when we emit SPIRV, we always emit Offset decorations for both Foo and Foo_std430. However this is against the SPIRV rules, which disallows a type to be declared as a local variable, if the type has explicit layout.

Solution

To fix the SPIRV validation error complaining Foo cannot be used to declare localVar because it has explicit layout, we simply need to mark the lowered storage types (Foo_std430 in the above example) with a [PhysicalType] decoration, and make sure we only emit SPIRV layout decorations (ArrayStride, MatrixStride, ColMajor, RowMajor, Offset) for structs with the [PhyiscalType] decoration.

The rest of the PR is to deal with a bunch of consequences after changing the way we emit SPIRV explicit layout based on whether or not a type is marked [PhysicalType].

• GLSLStorageBuffer type weren't lowered by lowerBufferElementTypeToStorageType previously, but now needs to be in order to produce valid SPIRV so that the element type is properly marked as physical.
• This means that lowerBufferElementTypeToStorageType needs to generate pack/unpack functions for unsized array types that can appear in glsl storage buffers, which we weren't handling previously.
• Array types without explicit stride are now treated as logical type and won't result in a stride being decorated in the resulting SPIRV. We need to be careful to never declare a local variable whose type is an array with explicit stride operand -- that will lead to invalid SPIRV. This leads to a change in the next bullet.
• packStorage/unpackStorage functions can no longer operate on SSA values, and has to operate on memory pointers instead, because we are no longer to elementExtract from a SSA register of a physical array type (can't copy it to a OpVariable and then do AccessChain, because OpVariable must be a logical type). These functions are inlined before emitting final code, because SPIRV doesn't allow us to pass pointers to functions without using VariablePointers capability. Because we no longer need to lower ElementExtract(array, i) to first copy to local var then do access chain, the resulting code should be much simpler for pack/unpack of large arrays and should be faster to compile by the downstream compiler.
• When packStorage/unpackStorage operate on pointers, we are effectively deferring the load of a composite to load of separate leaf fields, this means that loadAlign(ptr) is going to be split into a lot of smaller loads, so we must propagate the alignment on the load through the addresses, so that each individual load/store gets the correct alignment.
• Reimplement ConstBufferType as a wrapper of native pointers instead of a special builtin type, because we no longer want to maintain that code that special cases for ConstBufferTypes with this change. This means we need to:
• Implemented pointers for GLSL target. The only reason for ConstBufferPointer to exist as a builtin was that it provides some minimal coverage of buffer_reference when targeting GLSL. With pointers supported for GLSL, there is no reason to keep it as a compiler-builtin.

The updated lowerBufferElementTypeToStorageType pass now translates the above example code to the following:

struct Foo { int x, y; }
[PhysicalType] struct Foo_std430 { [Offset(0)] int x;  [Offset(4)] int y; }

StructuredBuffer<Foo_std430> f;  // or ConstantBuffer, etc.

// Pack/unpack function are now inlined by Slang, because they
// need to work on references instead of values.
[ForceInline] void packStorage(Foo_std430* dest, Foo f) { ... }
[ForceInline] Foo unpackStorage(Foo_Std430* src) {...}

void main()
{
     Foo localVar;

     // unpack function takes in a pointer to the source value, instead of an SSA value.
     localVar = unpackStorage(&f[0]);

     // destination driven packing function that writes directly into a pointer,
     // instead of returning an SSA value.
     packStorage(&f[1], localVar); 
}

Copilot Summary

This pull request includes significant changes to the handling of pointer types and various optimizations in the SPIR-V and GLSL code generation. The most important changes include the refactoring of ConstBufferPointerType, updates to GLSL source emission, and improvements to SPIR-V emission and optimization.

Refactoring and Simplification:

• source/slang/hlsl.meta.slang: Refactored ConstBufferPointer to simplify its structure and improve inline functions.
• source/slang/slang-ast-type.h: Removed the ConstBufferPointerType class to streamline the pointer type definitions.

GLSL Source Emission:

• source/slang/slang-emit-glsl.cpp: Updated various methods to handle the new pointer type structure, including emitBufferPointerTypeDefinition, tryEmitInstExprImpl, and emitSimpleTypeImpl. [1] [2] [3]

SPIR-V Emission and Optimization:

• source/slang/slang-emit-spirv.cpp: Improved handling of forward-declared pointers and added checks to emit array stride and offset decoration only for types marked as physical. [1] [2]
• source/slang/slang-emit.cpp: Added force inlining and other optimization steps after lowerBufferElementTypeToStorageType pass. [1] [2]

These changes collectively enhance the handling of pointer types and improve the efficiency and correctness of the generated GLSL and SPIR-V code.

[jkwak-work]

Looks good to me.

We were talking about the logical storage and physical storage in WorkGraphs meeting today coincidently.
I guess this is a thing going around recently.

[kaizhangNV]

LGTM