RTAS and GraphicsBuffer

Hello,

I am trying to use the new raytracing capabilities not for graphics rendering but for collision detection during physics simulation on GPU. My initial tests are promising. I have got a deformable triangular meshes such as cloths or skinned avatars held in Compute Buffers and entirely simulated on GPU.

I build the GraphicsBuffer called dynamicMeshAABBsBuffer, which contains RTAS AABBs as described in the docs (two min/max float3s per triangle):

Code (CSharp):

1. RWStructuredBuffer<float3> dynamicMeshAABBsBuffer;
2.  
3. [numthreads(32, 1, 1)]
4. void ComputeAABBs(uint3 id : SV_DispatchThreadID)
5. {
6.     if (id.x < trianglesCount)
7.     {
8.         int id0 = dynamicMeshIndicesBuffer[id.x * 3 + 0];
9.         int id1 = dynamicMeshIndicesBuffer[id.x * 3 + 1];
10.         int id2 = dynamicMeshIndicesBuffer[id.x * 3 + 2];
11.  
12.         float3 v0 = dynamicMeshVertexBuffer[id0].xyz;
13.         float3 v1 = dynamicMeshVertexBuffer[id1].xyz;
14.         float3 v2 = dynamicMeshVertexBuffer[id2].xyz;
15.  
16.         float3 minV = min(v0, min(v1, v2));
17.         float3 maxV = max(v0, max(v1, v2));
18.  
19.         dynamicMeshAABBsBuffer[id.x * 2 + 0] = minV;
20.         dynamicMeshAABBsBuffer[id.x * 2 + 1] = maxV;
21.     }
22. }

I am pretty sure that the AABBs are correct and are drawn below for the avatar mesh.
Now I need to substep physics 16-32 times per frame. In each substep I need to do raycasts on updated meshes. I tried to feed the GraphicsBuffer with AABBs to AddInstance but it does not work:

Code (CSharp):

1.                
2. RayTracingAccelerationStructure.RASSettings rtSetup = new RayTracingAccelerationStructure.RASSettings();
3. rtSetup.layerMask = -1;
4. rtSetup.managementMode = RayTracingAccelerationStructure.ManagementMode.Manual;
5. rtSetup.rayTracingModeMask = RayTracingAccelerationStructure.RayTracingModeMask.DynamicGeometry;
6. rtAccelerationStructure = new RayTracingAccelerationStructure(rtSetup);
7. meshCompute.SetBuffer(computeAABBsKernelId, "dynamicMeshAABBsBuffer", m_cntr.m_dynamicMeshAABBsBuffer);
8. meshCompute.SetBuffer(computeAABBsKernelId, "dynamicMeshVertexBuffer", m_cntr.m_dynamicMeshVertexBuffer);
9. meshCompute.SetBuffer(computeAABBsKernelId, "dynamicMeshIndicesBuffer", m_cntr.m_dynamicMeshIndicesBuffer);
10. meshCompute.Dispatch(computeAABBsKernelId, dynamicMeshTrianglesGroupCount, 1, 1);
11.  
12. // this works fine
13. //     rtAccelerationStructure.AddInstance(skinMesh);
14.  
15. //this does not
16. rtAccelerationStructure.AddInstance(m_cntr.m_dynamicMeshAABBsBuffer, (uint)m_cntr.m_dynamicMeshTrianglesCount, skinMesh.material, false);
17.    
18. rtAccelerationStructure.Build();
19.  

Am I missing something to make this work? For performance I need to keep everything on the GPU.

Another question is how to make just a quick refit of the AABB for such use case without a rebuild.

Thanks!

EDIT: Now when I think about it seems there is no way the RTAS knows anything about actual mesh vertices and indices... How can I hook these up?

 

OK, so I read bit more on this here https://microsoft.github.io/DirectX...riangle-intersection---triangle-mesh-geometry :

So my question would be how can I pass a triangular mesh geometry from Compute/Graphics buffer, without need to manually writing an intersection shader for ray vs. triangle? This is already optimally implemented by the GPU driver, possibly even in hardware.

Cheers

 

Apologies for a third post in a row. So I have figured out how to feed the simulated mesh topology (i.e. its vertex and index buffers) from ComputeBuffers and I would like to share with others for the reference.

This seems to work as expected besides the inconvenience of having the code also in material shader. But luckily this should be solved when DXR 1.1 supports come out.

Now my question is, whether such approach is slower than the built-in ray-triangle intersection:

Does "built-in" means here that the algorithms is implemented in hardware so ithe intersection test is faster than implementing it as shader? @INedelcu what do you think?

Cheers

Code (CSharp):

1.  SubShader
2.     {
3.         Pass
4.         {
5.             Name "RayTracing"
6.             Tags { "LightMode" = "RayTracing" }
7.  
8.             HLSLPROGRAM
9.      
10.             #pragma raytracing test
11.             #include "Common.hlsl"
12.  
13.             StructuredBuffer<float4> POSITIONS0;
14.             StructuredBuffer<float4> POSITIONS1;
15.             StructuredBuffer<float3> NORMALS0;
16.             StructuredBuffer<float3> NORMALS1;
17.             StructuredBuffer<int> INDICES;
18.  
19.             float meshInterpolationT;
20.  
21.             static const float EPSILON = 1e-8;
22.  
23.             bool IntersectTriangle(float3 rayOrigin, float3 rayDirection, float3 vert0, float3 vert1, float3 vert2, inout float t, inout float u, inout float v, inout uint hitKind);
24.  
25.             struct TriAttributeData
26.             {
27.                 float2 barycentrics;
28.                 float3 normal;
29.                 float3 position;
30.             };
31.  
32.             [shader("intersection")]
33.             void Intersection()
34.             {
35.                 int triId = PrimitiveIndex();
36.  
37.                 int id0 = INDICES[triId * 3 + 0];
38.                 int id1 = INDICES[triId * 3 + 1];
39.                 int id2 = INDICES[triId * 3 + 2];
40.  
41.                 float3 v0 = lerp(POSITIONS0[id0], POSITIONS1[id0], meshInterpolationT);
42.                 float3 v1 = lerp(POSITIONS0[id1], POSITIONS1[id1], meshInterpolationT);
43.                 float3 v2 = lerp(POSITIONS0[id2], POSITIONS1[id2], meshInterpolationT);
44.              
45.                 float3 n0 = lerp(NORMALS0[id0], NORMALS1[id0], meshInterpolationT);
46.                 float3 n1 = lerp(NORMALS0[id1], NORMALS1[id1], meshInterpolationT);
47.                 float3 n2 = lerp(NORMALS0[id2], NORMALS1[id2], meshInterpolationT);
48.  
49.                 float3 origin = WorldRayOrigin();
50.                 float3 direction = WorldRayDirection();
51.  
52.                 uint hitKind;
53.                 float t, u, v;
54.                 if (IntersectTriangle(origin, direction, v0, v1, v2, t, u, v, hitKind))
55.                 {
56.                     float3 bar = float3(1.0 - u - v, u, v);
57.                     float3 position = INTERPOLATE_RAYTRACING_ATTRIBUTE(v0, v1, v2, bar);
58.                     float3 normal = INTERPOLATE_RAYTRACING_ATTRIBUTE(n0, n1, n2, bar);
59.  
60.                     TriAttributeData attr;
61.                     attr.barycentrics = float2(u, v);
62.                     attr.position = position;
63.                     attr.normal = normal;
64.  
65.                     ReportHit(t, hitKind, attr);
66.                 }
67.              
68.             }
69.          
70.  
71.             [shader("closesthit")]
72.             void ClosestHitShader(inout RayIntersection rayIntersection : SV_RayPayload, TriAttributeData attributeData : SV_IntersectionAttributes)
73.             {
74.                 rayIntersection.hitKind = HitKind();
75.                 rayIntersection.t = RayTCurrent();
76.                 rayIntersection.position = attributeData.position;
77.                 rayIntersection.normal = float4(attributeData.normal, 1);
78.             }
79.  
80.             ENDHLSL
81.         }
82.     }
83. }
84.  

And the code for the RTAS which is built once per frame and reused during all substeps:

Code (CSharp):

1. RayTracingAccelerationStructure.RASSettings rtSetup = new RayTracingAccelerationStructure.RASSettings();
2. rtSetup.layerMask = -1;
3. rtSetup.managementMode = RayTracingAccelerationStructure.ManagementMode.Manual;
4. rtSetup.rayTracingModeMask = RayTracingAccelerationStructure.RayTracingModeMask.Everything;
5.  
6. rtAccelerationStructure = new RayTracingAccelerationStructure(rtSetup);
7.  
8. //recomputes the AABBs based on mesh at frame start and end state
9. meshCompute.SetBuffer(computeAABBs2KernelId, "dynamicMeshAABBsBuffer", m_cntr.m_dynamicMeshAABBsBuffer);
10. meshCompute.SetBuffer(computeAABBs2KernelId, "dynamicMeshVerticesOut0Buffer", m_cntr.m_dynamicMeshVertexOut0Buffer);
11. meshCompute.SetBuffer(computeAABBs2KernelId, "dynamicMeshVerticesOut1Buffer", m_cntr.m_dynamicMeshVertexOut1Buffer);
12. meshCompute.SetBuffer(computeAABBs2KernelId, "dynamicMeshIndicesBuffer", m_cntr.m_dynamicMeshIndicesBuffer);
13. meshCompute.Dispatch(computeAABBs2KernelId, dynamicMeshTrianglesGroupCount, 1, 1);
14.  
15. //hook up things for the intersection shader
16. skinMeshMat.SetBuffer("POSITIONS0", m_cntr.m_dynamicMeshVertexOut0Buffer);
17. skinMeshMat.SetBuffer("POSITIONS1", m_cntr.m_dynamicMeshVertexOut1Buffer);
18. skinMeshMat.SetBuffer("NORMALS0", m_cntr.m_dynamicMeshNormalsOut0Buffer);
19. skinMeshMat.SetBuffer("NORMALS1", m_cntr.m_dynamicMeshNormalsOut1Buffer);
20. skinMeshMat.SetBuffer("INDICES", m_cntr.m_dynamicMeshIndicesBuffer);
21. skinMeshMat.SetFloat("meshInterpolationT", 0);
22. rtAccelerationStructure.AddInstance(m_cntr.m_dynamicMeshAABBsBuffer, (uint)m_cntr.m_dynamicMeshTrianglesCount, skinMeshMat, false);
23.    
24. rtAccelerationStructure.Build();

 

Thank you @INedelcu for your thorough reply. That is what I was worried about that my approach - that it won't make the most out of RT capabilities of modern GPUs. For accurate and stable simulation I need to run collision detection multiple times (16-32) per frame so performance is critical. It is also important to be able to make quick refits to the AABBs as the changes each frame are rather small.

For sure the more flexibility the better. Unfortunately, all my physics simulation meshes are stored in ComputeBuffers and rendered directly from there. So I don't use any Mesh or SkinnedMesh.

What would solve my problem would be adding the followin method to the API, which would use built-in ray-triangle pipeline:

Code (CSharp):

1. public void AddInstance(GraphicsBuffer vertexBuffer, GraphicsBuffer indexBuffer, GraphicsBuffer aabbBuffer, uint numElements, Material material, Matrix4x4 instanceTransform, bool isCutOff, bool enableTriangleCulling, bool frontTriangleCounterClockwise, uint mask, bool reuseBounds);

Collision detection in physics simulation is the main performance bottleneck. Speeding it up with RT-Cores would be a big thing for my research and work.

Any chances of adding such method anytime soon?

Big thanks in advance and keep up the good work!

 

Well, the triangles (vertices and indices) and their AABBs are all we need to setup the RTAS, right? I am sorry, but it is not clear to me why would I need a Mesh in-between to do this. I mean I get the need for Mesh API for other things, but what would be a problem here to have also such a "direct" method?

No, this method would not use intersection shader but the built-in ray-triangle acceleration, just like with the Mesh and SkinnedMesh now. That is the point.

Regarding rendering from ComputeBuffers, I was not too precise. I do have a regular Mesh but I modified the Standard Shader, which it uses so that it reads the vertex positions from ComputeBuffer. Please see video below. However, I use Mesh only in the last step - in rendering. Before that I have got dozens of compute kernels of physics simulation and it is where I need to plugin the RT collision detection. I might be wrong but I am afraid that having a Mesh API in-between my kernel call would just complicate things.

Thanks

 

Hi @INedelcu, just wanted to ask how far you guys are from adding this:

I haven't seen any mentions in latest 2021.1 beta or 2021.2 alpha

Thanks