Question Running a geometry shader with ComputeBuffer in HDRP

I'm having a rather peculiar issue regarding shaders in HDRP. It might not be purely shader-related, but I will still post it here is it mainly focuses on that.

For a research project, we need to create a live-feed 3D point cloud which has to be rendered into a Mixed Reality environment. I will provide as much context as I can, just for comprehension and to offer a broad searching ground for solutions.

1. The point cloud is tracked via an Azure Kinect Camera and rendered in Unity via Keijiro's Pcx package (https://github.com/keijiro/Pcx). This package stores the points in a ComputeBuffer and uses a custom geometry shader ("Disk") to render the points. As instanced points are not an option performance-wise - we're dealing with 1000-10000 points per frame, with 30 frames per second here - this solution works very well and performant so far.

2. For the MR environment we use a Varjo XR headset, which is able to compose Unity's rendering with images from its video-see-through (VST) cameras. Since multiple augmented virtuality scenarios are involved - which means users in virtual reality environments still have to see specific elements in actual reality - the need for VST masking came up. Varjo provides possibilities for this, but unfortunately it required us to step up to the HD Rendering Pipeline. The documentation is rather scarce (https://developer.varjo.com/docs/unity-xr-sdk/masking-with-varjo-xr-plugin), experimenting with the standard pipeline turned out to be fruitless, and Varjo headsets are not well-distributed enough to find any support online.

Both of those factors are now unluckily clashing with each other: After upgrading to HDRP, the rendered point cloud is still visible in the scene perspective of the Unity Editor, but no longer through the in-game camera. It seems that HDRP is not supporting the shader used by Pcx.

I'm not actually into shaders, but searching through multiple threads on here tells me that it seems impossible to simulate this shader via shader graph, let alone using a ComputeBuffer with it. It also seems that documentation on custom HDRP shaders is also very scarce since they want to move users to using shader graphs, which seems not to be applicable for our scenario. This makes me think that HDRP is highly unsuited for working with point clouds, but also required for working with Varjo's VST masking.

I am at a loss of how we can procede to get both point cloud and VST masking at the same time. I don't rely much on VST masking to work outside of HDRP, so there has to be way to get the point cloud back working again. Does someone know any magical solution to achieve that?

HDRP Version: 12.1.12
Unity Version: 2021.3.27f1
OS: Windows 10 Enterprise

 

HDRP shaders is just ordinary GPU shaders that need a bit of special setup to work with HDRP. They are complicated because HDRP is complex RP with big variety features support (Forward/Deferred renderers, raytracing, etc.). But for simple shaders like you are using it is possible to adapt them with a little effort. You need to add HDRP compatibility tag to the Subshader of “Point Cloud/Disk” shader:

Code (CSharp):

1. Tags{ "RenderPipeline" = "HDRenderPipeline" }

Then change “ForwardBase” pass name to “ForwardOnly”.

As side note, making disk geometry by making vertices using geometry shader is not very performant solution. Making static unit disc mesh and scaling it in shader or procedurally submitting necessary number of vertices and constructing it in vertex shader (with almost same code as in geometry shader) allows rendering more points with same GPU time.

 

Thanks for your input. I added

HDRenderPipeline

and updated to

ForwardOnly

, but unfortunately this didn't have any effect. Could you confirm that those changes make this shader work in HDRP? If you want, I can try to prepare a bare minimum project example for you to check on.

 

Yes, I can look.

 

Here you go. The Assets Folder includes a Sample Scene containing two static point clouds. One rendered via ComputeBuffer - it is missing from the camera view - and one as a mesh, which is rendering fine. The packages already includes Pcx with the modified shader. Maybe you will figure something out...

 

PointCloudRenderer.cs script in your project has incompatibilities with HDRP. OnRenderObject callback used to render point cloud is not valid for HDRP (ignored). As quick fix you can do the following:
* Change signature of OnRenderObject function to the:

void DrawPoints(Camera camera)

* Remove

camera = Camera.current

line from function body.
* Make function to handle camera rendering after all camera rendering was done:

Code (CSharp):

1. void DrawCameras(ScriptableRenderContext sc, Camera[] cameras)
2. {
3.       foreach (var c in cameras)
4.       {
5.            DrawPoints(c);
6.       }
7. }
8.  

* Add created function as end frame callback:

Code (CSharp):

1. private void Start()
2. {
3.     RenderPipelineManager.endFrameRendering += DrawCameras;
4. }
5.  

* No shader changes required.

 

It works! Thank you so much for your help!

 

Must note that given method will not integrate your point rendering into HDRP at all. None of the features of HDRP (shadows, volumes, post effects, all of them) will work. With this approach its no matter what SRP you are using (HDRP, URP or custom). For real integration shader and draw submission changes are necessary.

 

This is not too relevant for my case, the point cloud is supposed to stay as dry as it is.

I'm encountering different problems now, though. For once, the rendering somehow still seems to be affected by some kind of exposure, since it partially fades into transparency. Only when moving the camera it's sometimes back to fully visual for a split second. Also while the point cloud is visible in the game camera now, it's still not composited onto the Varjo headset. I'll see if I can find any further information about that.

UPDATE: I figured out the exposure issue, it was merely a Z testing problem. Adding

ZWrite Off

to the Subshader fixed this for me.