Ray Tracing Pipeline (NV and KHR extensions)

[bwrsandman]

• Added an entry to CHANGELOG_VULKANO.md or CHANGELOG_VK_SYS.md if knowledge of this change could be valuable to users
• Updated documentation to reflect any user-facing changes - in this repository
• Updated documentation to reflect any user-facing changes - PR to the guide that fixes existing documentation invalidated by this PR.

I'm proposing these additions to vulkano. I'm setting as draft because I would like some help/feedback. I'm fairly new to Rust and there may be some tricks I don't know about.

There are five main parts to this addition (I also fixed compute shader to swapchain storage image):

Extension

There are two extensions which allow ray tracing: khr_ray_tracing and nv_ray_tracing.
The NV extension is likely only available with select NVidia cards. I have a 1070 which supports this on Linux so it is not specifically limited to the RTX line of cards.

The KHR extension is newer and should have wider adoption in the future. As it is now, for NVidia, it requires a beta driver to test. I have not finished implementing this.
The two extensions differ slightly but I wrapped builders to have a unique API. This is done with a bool but maybe it could be better with a Generics.

Ray Tracing Pipeline

This is different from both Compute and Graphics pipeline.
There can be multiple stages of the Ray Tracing Pipelines of the same type, for example, there can be two miss shaders.

Additionally, there is a concept of shader groups. Ray Generation, Miss and Execution Shaders are alone in their vulkan groups. Actual groups include optional Closest Hit, Any Hit and Intersection Shaders. There can be more than one of these groups.

If a group has an Intersection Shader, the geometry is procedural, otherwise, the geometry should be triangles. This automatically handled in this PR.

This has been challenging to me as I don't know of a way to make variable length generics or to wrap the builder stages in traits. It would be nice to have help here.

In the sample, I create procedural geometry and I would like to add a triangle group but this requires a solution to the variable length of generics.

Note that shaderc links to an old version of glslang which does not support GL_EXT_ray_tracing. For this reason, I did not finish implementing the khr side of the implementation.

Acceleration Structures

To ray trace, the pipeline must have access to the full scene in the trace command. Vulkan offers acceleration structures (a bounding volume hiearchy of axis-aligned bounding boxes) to quickly traverse the geometry (bounding boxes for procedural geometry or triangles).

For one scene representation, Vulkan offers multiple bottom level acceleration structure for different meshes and only one top level acceleration structure to group them.

A static scene only needs to be built once but a dynamic scene requires a rebuild with the build_acceleration_structure command at each change.

In this PR, AccelerationStructureBuilder automatically creates the top level acceleration structure and all lower level structure created with add_aabbs for procedural geometry and add_triangles for triangle meshes.

The structure requires memory for storage and a scratch buffer for rebuilding.

Adding triangles is not yet implemented.

Shader Binding Tables

The shader binding table is a pointer list the shaders use to call each other.
In this PR, the table has to be built and uploaded in a buffer manually, but it might be possible to do this automatically in the pipeline. Different trace calls can use different tables and offsets.

nv-ray-tracing example

Help needed

I need some assistance on the following points:

• In vulkano-shaders, the added StorageClass entries require either khr_ray_tracing or nv_ray_tracing. As far as I can tell, there is no way to create a requirement which is satisfied by one extension OR another.
• Should an AcclereationStructureAccess trait be added?
• Automate shader binding tables/simpler user-facing operations.
• Modify builder to have any amount of ray-generation, miss, callable shaders and any number of groups of closest-hit, any-hit, intersection shaders. The current Generics implementation does not allow this.
• The shaderc create needs to be updated upstream to support newer versions of glslang (8.13.3743) which can recognize nv_ray_tracing and khr_ray_tracing extensions.
• VK_KHR_ray_tracing requires other extenions: VK_EXT_descriptor_indexing, VK_KHR_buffer_device_address, VK_KHR_deferred_host_operations, VK_KHR_pipeline_library. Should vulkano add those to the list if DeviceExtensions::khr_ray_tracing is enabled?
• khr_ray_tracing requires enabling vk::PhysicalDeviceRayTracingFeaturesKHR::rayTracing and vk::PhysicalDeviceBufferDeviceAddressFeatures::bufferDeviceAddress by passing the structs in a pNext chain in a vk::PhysicalDeviceFeatures2 for vk::DeviceCreateInfo.pNext and for vk::GetPhysicalDeviceFeatures2. The current set-up does not allow this to be done easily from codegen.rs or the instance builder. features::Features needs to be expanded to support these other features.
• Is there a better way to encapsulate the NV and KHR implementation than my use_nv_extension bool check at runtime?
• Memory alloc and pools need to allocate with vk::MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR in pNext when attached to a buffer created with vk::BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_KHR which is used in the khr_ray_tracing implementation for the Acceleration Structures.
• Architecture, Style and improvements to my rust code (I'm fairly new)

[Eliah-Lakhin]

@bwrsandman Any updates regarding the progress on this feature?

It's hard to find volunteers to help with development unfortunately, but the feature would be definitely very beneficial for Vulkano infrastructure.

[bwrsandman]

I haven't progressed as I require a bit of help / review on the the mentioned points.

[Rua]

This is something I would like to help with, but I don't have drivers that support these extensions, so I can only write code, I can't do any testing.

Since this is such a huge package of changes, I think it may be better to implement it in smaller PRs one at a time. That is, add the smaller pieces first, maybe one PR per extension, even if they are initially useless. Then at the end you implement the main extension that ties it all together. That avoids the current situation where a large amount of work is tied up in a PR without going anywhere.

Some of the points you raised have been solved since you started this PR:

• Vk-sys is replaced with Ash, which is much more up to date.
• A lot of stuff is now auto-generated from vk.xml, including extension and feature dependencies. So you don't need to worry about those anymore.
• capability_requirement in Vulkano-shaders now does exactly what you need.
• Regarding the difference between NV and KHR variants, I suggest doing only KHR first, and adding NV in a separate PR later. While I don't think Vulkano has anything against vendor-specific functionality, it is not as important as KHR and EXT.

[MarijnS95]

• Regarding the difference between NV and KHR variants, I suggest doing only KHR first, and adding NV in a separate PR later. While I don't think Vulkano has anything against vendor-specific functionality, it is not as important as KHR and EXT.

The KHR extension has wide enough adoption that it's wasteful to support NV, unless you wish to use Vulkano on one specific ancient Nvidia driver. The semantics changed a bit in some key areas, too.

[bwrsandman]

Good to know. I hope to have some time in the coming weeks to do as you suggested.
Should we create an issue to track the smaller portions and as a point of contact for implementation questions?

[VictorBulba]

Hello 👋 I'm trying to rewrite the acceleration structure to work with new versions of vulkano. If someone is also doing this, let's discuss

[VictorBulba]

Hey, I have created a PR with an acceleration structure draft #1736

[bwrsandman]

What way should raytracing pipelines be implemented considering the infinite amount of shaders that can be attached?
I did not find a way using generics the same way as Compute and Graphics.

[Rua]

I'm planning to rework ComputeEntryPoint and GraphicsEntryPoint in the future, so that they extract their information from the shader. In the meantime though, you can make a RayTracingEntryPoint?