Question Custom pass depth buffer extraction from multiple cameras

Hi there,

I am trying to sample and stream multiple cameras' outputs in Unity. I have 3 RGB cameras and 4 Depth only cameras.

Depth cameras are created dynamically by

Code (CSharp):

1.             Camera[] Cameras = new Camera[numCameras];
2.             for (int i = 0; i < numCameras; i++)
3.             {
4.                 GameObject CameraObject = new GameObject();
5.                 CameraObject.name = CameraTag + i;
6.                 CameraObject.transform.SetParent(transform);
7.                 CameraObject.transform.localRotation = Quaternion.Euler(0, i * 360.0f / numCameras, 0);
8.                 CameraObject.transform.localPosition = new Vector3(0, 0, 0);
9.                 //CameraObject.layer = LayerMask.NameToLayer(LidarLayer);
10.                 CameraObject.AddComponent<Camera>();
11.                 Camera cam = CameraObject.GetComponent<Camera>();
12.  
13.                 var depthBuffer = new RenderTexture(frustumTemplate.pixelWidth, frustumTemplate.pixelHeight, 16, format);
14.                 depthBuffer.depth = (int)DepthBufferPrecision;
15.                 cam.targetTexture = depthBuffer;
16.                 cam.depth = 1;

In the HDRP Custom pass I go through each depth camera and render depth info only:

Code (CSharp):

1.             for (int i = 0; i < cameras.Length; i++)
2.             {
3.                 Camera bakingCamera = cameras[i];
4.                 RenderTexture targetTexture = bakingCamera.targetTexture;
5.                 ScriptableCullingParameters cullingParams;
6.                 bakingCamera.TryGetCullingParameters(out cullingParams);
7.                 cullingParams.cullingOptions = CullingOptions.ShadowCasters;
8.                 cullingResult = renderContext.Cull(ref cullingParams);
9.                 var result = new RendererListDesc(shaderTags, cullingResult, bakingCamera)
10.                 {
11.                     rendererConfiguration = PerObjectData.None,
12.                     renderQueueRange = RenderQueueRange.all,
13.                     sortingCriteria = SortingCriteria.BackToFront,
14.                     excludeObjectMotionVectors = false,
15.                     layerMask = -1,
16.                 };
17.              .
18.              .
19.              .
20.  
21.                 CoreUtils.SetRenderTarget(cmd, targetTexture, ClearFlag.Depth);
22.                 HDUtils.DrawRendererList(renderContext, cmd, RendererList.Create(result));

What seems to be the problem is that, somehow, rgb cameras and depth are sharing textures. To be honest, I don't know how and why, because for each camera a new RT is created.

I created a test quad in the scene just to render the depth info on it and debug the problem. I have set it to render the first depth camera output, but it turns out that it renders depth info of the first camera in the object tree it finds. If I set the depth of the camera bellow zero, then it skips the camera and renders the next camera it finds.

What could cause such a behaviour?

 

After a week wasted trying to fix the problem, I still got nowhere.

 

The problem seems to only be present in the Unity2020 with HDRP 11 or newer.

My next approach is to use HDCamera.customRender per every depth camera. The results are the same as before, multiple cameras draw on the same textures..

There should be a lot of unnecessery code here, regarding setting render targets multiple times etc., but I just cannot make it work.

Code (CSharp):

1.         public static void CustomRender(ScriptableRenderContext context, HDCamera camera)
2.         {
3.             // Debug.Log(camera.camera.name);
4.             var cameraBuffer = new CommandBuffer { name="Render Camera" + camera.camera.name};
5.             cameraBuffer.SetRenderTarget(camera.camera.targetTexture);
6.             cameraBuffer.ClearRenderTarget(true, false, Color.black);
7.        
8.             cameraBuffer.BeginSample("Render camera" + camera.camera.name);
9.             context.ExecuteCommandBuffer(cameraBuffer);
10.             cameraBuffer.Clear();
11.  
12.             camera.camera.TryGetCullingParameters(out var cullingParameters);
13.             var cullingResults = context.Cull(ref cullingParameters);
14.  
15.             cameraBuffer.SetRenderTarget(camera.camera.targetTexture);
16.             context.ExecuteCommandBuffer(cameraBuffer);
17.             cameraBuffer.Clear();
18.  
19.             context.SetupCameraProperties(camera.camera);
20.             var drawingSettings = new DrawingSettings();
21.             drawingSettings.SetShaderPassName(0, new ShaderTagId("DepthOnly"));
22.             drawingSettings.SetShaderPassName(1, new ShaderTagId("DepthForwardOnly"));
23.             var filteringSettings = new FilteringSettings(RenderQueueRange.all);
24.             context.DrawRenderers(cullingResults, ref drawingSettings, ref filteringSettings);
25.             cameraBuffer.EndSample("Render camera" + camera.camera.name);
26.             context.ExecuteCommandBuffer(cameraBuffer);
27.             cameraBuffer.Clear();
28.             context.Submit();
29.         }