Issue when baking _CameraDepthTexture

Yup,
I'm trying to bake the depth texture of the main camera (orthographic projection, forward rendering path).
I started with a simple vert/frag shader that reads from _CameraDepthTexture and outputs the encoded value (frag returns EncodeFloatRGBA()) via camera.RenderWithShader().
The issue is that the depth of objects in queues > AlphaTest (like trees' leaves) is calculated without taking in account their alpha (so their entire quads are shown in the depth texture).
I started fiddling with the render queue of the shader with no avail;
then I tried doing the same with a surface shader with fullforwardshadows (thinking that in doing so the replacement pass would have found the necessary SubShader(s)): nope.
Then I tried modifying the Internal-DepthNormalsTexture (Unity built-in shader) to return the EncodedFloatRGBA in each SubShader (since to me it seems that it has all the SubShaders and RenderTypes necessary): still nope.
Meanwhile in the frame debugger after the DepthPassJob I can see the depth texture as I need it: I don't get why when accessing _CameraDepthTexture I get a different texture.
The only way in which I can get the "correct" depth texture is by blitting in play using the first vert/frag shader (the most "dumb" one: reading from _CameraDepthTexture and outputting the encoded value).
I don't get why it should not work outside of Play mode.

If anyone can help me understand, I would very much appreciate it.
Thanks!



P.S. In order: shaded, depth texture from frame debugger, what I get when baking the depth texture.

P.P.S. Sorry for any mistake in what I wrote, it's what I ended up understanding in a day of Googling.

 

Can you show some of your code? To me it sounds like you’re potentially using RenderWithShader incorrectly.

Also I’m not sure why you’re sampling the camera depth texture to generate a depth texture. See how the the depth normals shader is getting the depth, or just use the camera depth texture itself.

 

Yup, of course.

Code (CSharp):

1.     private Texture2D RenderDepthTexture(ref Camera _cam,Vector3 localCamPosition,Vector2Int depthTextureSize)
2.     {
3.         RenderTexture renderTexture = new RenderTexture(depthTextureSize.x, depthTextureSize.y, 24, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
4.         Texture2D _render = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.ARGB32, true);
5.  
6.         RenderTexture _cameraPreviousRenderTexture;
7.         Camera _camera = Instantiate(_cam,_cam.transform);
8.         _camera.depthTextureMode = DepthTextureMode.Depth;
9.  
10.         _camera.targetTexture = renderTexture;
11.        
12.         _camera.transform.localPosition = localCamPosition;
13.         _camera.transform.localRotation = Quaternion.identity;
14.  
15.         if (RenderTexture.active != renderTexture) RenderTexture.active = renderTexture;
16.         _camera.RenderWithShader((Shader)p_depthCameraShader.objectReferenceValue, "");
17.         _render.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
18.         _render.Apply();
19.  
20.         RenderTexture.active = null;
21.  
22.  
23.         _camera.targetTexture = _cameraPreviousRenderTexture;
24.         DestroyImmediate(_camera.gameObject);
25.         return _render;
26.     }

The shader (tried different queues but with no avail):

Code (CSharp):

1. Shader "Custom/RenderDepthTexture"
2. {
3.     SubShader
4.     {
5.         Tags{ "RenderType" = "Opaque" }
6.  
7.         Pass
8.         {
9.             CGPROGRAM
10.             #pragma vertex vert
11.             #pragma fragment frag
12.             #include "UnityCG.cginc"
13.  
14.             sampler2D _CameraDepthTexture;
15.  
16.             struct v2f
17.             {
18.                 float4 pos : SV_POSITION;
19.                 float4 scrPos:TEXCOORD1;
20.             };
21.  
22.             //Vertex Shader
23.             v2f vert(appdata_base v)
24.             {
25.                 v2f o;
26.                 o.pos = UnityObjectToClipPos(v.vertex);
27.                 o.scrPos = ComputeScreenPos(o.pos);      
28.                 return o;
29.             }
30.  
31.             //Fragment Shader
32.             float4 frag(v2f i) : COLOR
33.             {
34.                 float depthValue = tex2Dproj(_CameraDepthTexture, UNITY_PROJ_COORD(i.scrPos)).r;
35.                 float4 depth;
36.  
37.                 depth = EncodeFloatRGBA(depthValue);
38.  
39.                 return depth;
40.             }
41.             ENDCG
42.         }
43.     }
44.     FallBack "Diffuse"
45. }

This method is called non during play mode and results in a wrong depth texture.
If instead I use the same shader in

Code (CSharp):

1. void OnRenderImage(RenderTexture source, RenderTexture destination)
2.     {
3.         Graphics.Blit(source, destination, matContainingShader);
4.  
5. }

and then I read and save the active RenderTexture the depth texture is correct.

Sorry, I do not understand.
I'm using the camera depth texture itself: what I want to accomplish is to get it "baked" for later re-use in gameplay (I'm working on a prerendered game à la Pillars of Eternity)
I tried using a modified version of the depth normals shader (Unity built-in's Internals-DepthNormalsTexture.shader) in the Camera.RenderWithShader where I replaced the return value of each frag with the encoded depth so to cover the same RenderTypes, but the resulting texture is still wrong (it doesn't take in account alpha cut out)

Code (CSharp):

1. Shader "Custom/DepthTextureAlpha" {
2. Properties {
3.     _MainTex ("", 2D) = "white" {}
4.     _Cutoff ("", Float) = 0.5
5.     _Color ("", Color) = (1,1,1,1)
6. }
7.  
8. SubShader {
9.     Tags { "RenderType"="Opaque" }
10.     Pass {
11. CGPROGRAM
12. #pragma vertex vert
13. #pragma fragment frag
14. #include "UnityCG.cginc"
15. struct v2f {
16.     float4 pos : SV_POSITION;
17.     float4 nz : TEXCOORD0;
18.     UNITY_VERTEX_OUTPUT_STEREO
19. };
20. v2f vert( appdata_base v ) {
21.     v2f o;
22.     UNITY_SETUP_INSTANCE_ID(v);
23.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
24.     o.pos = UnityObjectToClipPos(v.vertex);
25.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
26.     o.nz.w = COMPUTE_DEPTH_01;
27.     return o;
28. }
29. fixed4 frag(v2f i) : SV_Target {
30.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
31.     return EncodeFloatRGBA(i.nz.w);
32. }
33. ENDCG
34.     }
35. }
36.  
37. SubShader {
38.     Tags { "RenderType"="TransparentCutout" }
39.     Pass {
40. CGPROGRAM
41. #pragma vertex vert
42. #pragma fragment frag
43. #include "UnityCG.cginc"
44. struct v2f {
45.     float4 pos : SV_POSITION;
46.     float2 uv : TEXCOORD0;
47.     float4 nz : TEXCOORD1;
48.     UNITY_VERTEX_OUTPUT_STEREO
49. };
50. uniform float4 _MainTex_ST;
51. v2f vert( appdata_base v ) {
52.     v2f o;
53.     UNITY_SETUP_INSTANCE_ID(v);
54.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
55.     o.pos = UnityObjectToClipPos(v.vertex);
56.     o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
57.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
58.     o.nz.w = COMPUTE_DEPTH_01;
59.     return o;
60. }
61. uniform sampler2D _MainTex;
62. uniform fixed _Cutoff;
63. uniform fixed4 _Color;
64. fixed4 frag(v2f i) : SV_Target {
65.     fixed4 texcol = tex2D( _MainTex, i.uv );
66.     clip( texcol.a*_Color.a - _Cutoff );
67.  
68.     return EncodeFloatRGBA(i.nz.w);
69.  
70.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
71.    
72. }
73. ENDCG
74.     }
75. }
76.  
77. SubShader {
78.     Tags { "RenderType"="TreeBark" }
79.     Pass {
80. CGPROGRAM
81. #pragma vertex vert
82. #pragma fragment frag
83. #include "UnityCG.cginc"
84. #include "Lighting.cginc"
85. #include "UnityBuiltin3xTreeLibrary.cginc"
86. struct v2f {
87.     float4 pos : SV_POSITION;
88.     float2 uv : TEXCOORD0;
89.     float4 nz : TEXCOORD1;
90.     UNITY_VERTEX_OUTPUT_STEREO
91. };
92. v2f vert( appdata_full v ) {
93.     v2f o;
94.     UNITY_SETUP_INSTANCE_ID(v);
95.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
96.     TreeVertBark(v);
97.  
98.     o.pos = UnityObjectToClipPos(v.vertex);
99.     o.uv = v.texcoord.xy;
100.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
101.     o.nz.w = COMPUTE_DEPTH_01;
102.     return o;
103. }
104. fixed4 frag( v2f i ) : SV_Target {
105.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
106.     return EncodeFloatRGBA(i.nz.w);
107. }
108. ENDCG
109.     }
110. }
111.  
112. SubShader {
113.     Tags { "RenderType"="TreeLeaf" }
114.     Pass {
115. CGPROGRAM
116. #pragma vertex vert
117. #pragma fragment frag
118. #include "UnityCG.cginc"
119. #include "Lighting.cginc"
120. #include "UnityBuiltin3xTreeLibrary.cginc"
121. struct v2f {
122.     float4 pos : SV_POSITION;
123.     float2 uv : TEXCOORD0;
124.     float4 nz : TEXCOORD1;
125.     UNITY_VERTEX_OUTPUT_STEREO
126. };
127. v2f vert( appdata_full v ) {
128.     v2f o;
129.     UNITY_SETUP_INSTANCE_ID(v);
130.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
131.     TreeVertLeaf(v);
132.  
133.     o.pos = UnityObjectToClipPos(v.vertex);
134.     o.uv = v.texcoord.xy;
135.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
136.     o.nz.w = COMPUTE_DEPTH_01;
137.     return o;
138. }
139. uniform sampler2D _MainTex;
140. uniform fixed _Cutoff;
141. fixed4 frag( v2f i ) : SV_Target {
142.     half alpha = tex2D(_MainTex, i.uv).a;
143.     clip (alpha - _Cutoff);
144.     return EncodeFloatRGBA(i.nz.w);
145.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
146. }
147. ENDCG
148.     }
149. }
150.  
151. SubShader {
152.     Tags { "RenderType"="TreeOpaque" "DisableBatching"="True" }
153.     Pass {
154. CGPROGRAM
155. #pragma vertex vert
156. #pragma fragment frag
157. #include "UnityCG.cginc"
158. #include "TerrainEngine.cginc"
159. struct v2f {
160.     float4 pos : SV_POSITION;
161.     float4 nz : TEXCOORD0;
162.     UNITY_VERTEX_OUTPUT_STEREO
163. };
164. struct appdata {
165.     float4 vertex : POSITION;
166.     float3 normal : NORMAL;
167.     fixed4 color : COLOR;
168.     UNITY_VERTEX_INPUT_INSTANCE_ID
169. };
170. v2f vert( appdata v ) {
171.     v2f o;
172.     UNITY_SETUP_INSTANCE_ID(v);
173.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
174.     TerrainAnimateTree(v.vertex, v.color.w);
175.     o.pos = UnityObjectToClipPos(v.vertex);
176.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
177.     o.nz.w = COMPUTE_DEPTH_01;
178.     return o;
179. }
180. fixed4 frag(v2f i) : SV_Target {
181.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
182.     return EncodeFloatRGBA(i.nz.w);
183. }
184. ENDCG
185.     }
186. }
187.  
188. SubShader {
189.     Tags { "RenderType"="TreeTransparentCutout" "DisableBatching"="True" }
190.     Pass {
191.         Cull Back
192. CGPROGRAM
193. #pragma vertex vert
194. #pragma fragment frag
195. #include "UnityCG.cginc"
196. #include "TerrainEngine.cginc"
197.  
198. struct v2f {
199.     float4 pos : SV_POSITION;
200.     float2 uv : TEXCOORD0;
201.     float4 nz : TEXCOORD1;
202.     UNITY_VERTEX_OUTPUT_STEREO
203. };
204. struct appdata {
205.     float4 vertex : POSITION;
206.     float3 normal : NORMAL;
207.     fixed4 color : COLOR;
208.     float4 texcoord : TEXCOORD0;
209.     UNITY_VERTEX_INPUT_INSTANCE_ID
210. };
211. v2f vert( appdata v ) {
212.     v2f o;
213.     UNITY_SETUP_INSTANCE_ID(v);
214.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
215.     TerrainAnimateTree(v.vertex, v.color.w);
216.     o.pos = UnityObjectToClipPos(v.vertex);
217.     o.uv = v.texcoord.xy;
218.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
219.     o.nz.w = COMPUTE_DEPTH_01;
220.     return o;
221. }
222. uniform sampler2D _MainTex;
223. uniform fixed _Cutoff;
224. fixed4 frag(v2f i) : SV_Target {
225.     half alpha = tex2D(_MainTex, i.uv).a;
226.  
227.     clip (alpha - _Cutoff);
228.     return EncodeFloatRGBA(i.nz.w);
229.  
230.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
231.     //return EncodeFloatRGBA(i.nz.w);
232. }
233. ENDCG
234.     }
235.     Pass {
236.         Cull Front
237. CGPROGRAM
238. #pragma vertex vert
239. #pragma fragment frag
240. #include "UnityCG.cginc"
241. #include "TerrainEngine.cginc"
242.  
243. struct v2f {
244.     float4 pos : SV_POSITION;
245.     float2 uv : TEXCOORD0;
246.     float4 nz : TEXCOORD1;
247.     UNITY_VERTEX_OUTPUT_STEREO
248. };
249. struct appdata {
250.     float4 vertex : POSITION;
251.     float3 normal : NORMAL;
252.     fixed4 color : COLOR;
253.     float4 texcoord : TEXCOORD0;
254.     UNITY_VERTEX_INPUT_INSTANCE_ID
255. };
256. v2f vert( appdata v ) {
257.     v2f o;
258.     UNITY_SETUP_INSTANCE_ID(v);
259.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
260.     TerrainAnimateTree(v.vertex, v.color.w);
261.     o.pos = UnityObjectToClipPos(v.vertex);
262.     o.uv = v.texcoord.xy;
263.     o.nz.xyz = -COMPUTE_VIEW_NORMAL;
264.     o.nz.w = COMPUTE_DEPTH_01;
265.     return o;
266. }
267. uniform sampler2D _MainTex;
268. uniform fixed _Cutoff;
269. fixed4 frag(v2f i) : SV_Target {
270.     fixed4 texcol = tex2D( _MainTex, i.uv );
271.     clip( texcol.a - _Cutoff );
272.  
273.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
274.     return EncodeFloatRGBA(i.nz.w);
275. }
276. ENDCG
277.     }
278.  
279. }
280.  
281. SubShader {
282.     Tags { "RenderType"="TreeBillboard" }
283.     Pass {
284.         Cull Off
285. CGPROGRAM
286. #pragma vertex vert
287. #pragma fragment frag
288. #include "UnityCG.cginc"
289. #include "TerrainEngine.cginc"
290. struct v2f {
291.     float4 pos : SV_POSITION;
292.     float2 uv : TEXCOORD0;
293.     float4 nz : TEXCOORD1;
294.     UNITY_VERTEX_OUTPUT_STEREO
295. };
296. v2f vert (appdata_tree_billboard v) {
297.     v2f o;
298.     UNITY_SETUP_INSTANCE_ID(v);
299.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
300.     TerrainBillboardTree(v.vertex, v.texcoord1.xy, v.texcoord.y);
301.     o.pos = UnityObjectToClipPos(v.vertex);
302.     o.uv.x = v.texcoord.x;
303.     o.uv.y = v.texcoord.y > 0;
304.     o.nz.xyz = float3(0,0,1);
305.     o.nz.w = COMPUTE_DEPTH_01;
306.     return o;
307. }
308. uniform sampler2D _MainTex;
309. fixed4 frag(v2f i) : SV_Target {
310.     fixed4 texcol = tex2D( _MainTex, i.uv );
311.     clip( texcol.a - 0.001 );
312.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
313.  
314.     return EncodeFloatRGBA(i.nz.w);
315. }
316. ENDCG
317.     }
318. }
319.  
320. SubShader {
321.     Tags { "RenderType"="GrassBillboard" }
322.     Pass {
323.         Cull Off
324. CGPROGRAM
325. #pragma vertex vert
326. #pragma fragment frag
327. #include "UnityCG.cginc"
328. #include "TerrainEngine.cginc"
329.  
330. struct v2f {
331.     float4 pos : SV_POSITION;
332.     fixed4 color : COLOR;
333.     float2 uv : TEXCOORD0;
334.     float4 nz : TEXCOORD1;
335.     UNITY_VERTEX_OUTPUT_STEREO
336. };
337.  
338. v2f vert (appdata_full v) {
339.     v2f o;
340.     UNITY_SETUP_INSTANCE_ID(v);
341.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
342.     WavingGrassBillboardVert (v);
343.     o.color = v.color;
344.     o.pos = UnityObjectToClipPos(v.vertex);
345.     o.uv = v.texcoord.xy;
346.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
347.     o.nz.w = COMPUTE_DEPTH_01;
348.     return o;
349. }
350. uniform sampler2D _MainTex;
351. uniform fixed _Cutoff;
352. fixed4 frag(v2f i) : SV_Target {
353.     fixed4 texcol = tex2D( _MainTex, i.uv );
354.     fixed alpha = texcol.a * i.color.a;
355.     clip( alpha - _Cutoff );
356.  
357.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
358.     return EncodeFloatRGBA(i.nz.w);
359. }
360. ENDCG
361.     }
362. }
363.  
364. SubShader {
365.     Tags { "RenderType"="Grass" }
366.     Pass {
367.         Cull Off
368. CGPROGRAM
369. #pragma vertex vert
370. #pragma fragment frag
371. #include "UnityCG.cginc"
372. #include "TerrainEngine.cginc"
373. struct v2f {
374.     float4 pos : SV_POSITION;
375.     fixed4 color : COLOR;
376.     float2 uv : TEXCOORD0;
377.     float4 nz : TEXCOORD1;
378.     UNITY_VERTEX_OUTPUT_STEREO
379. };
380.  
381. v2f vert (appdata_full v) {
382.     v2f o;
383.     UNITY_SETUP_INSTANCE_ID(v);
384.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
385.     WavingGrassVert (v);
386.     o.color = v.color;
387.     o.pos = UnityObjectToClipPos(v.vertex);
388.     o.uv = v.texcoord;
389.     o.nz.xyz = COMPUTE_VIEW_NORMAL;
390.     o.nz.w = COMPUTE_DEPTH_01;
391.     return o;
392. }
393. uniform sampler2D _MainTex;
394. uniform fixed _Cutoff;
395. fixed4 frag(v2f i) : SV_Target {
396.     fixed4 texcol = tex2D( _MainTex, i.uv );
397.     fixed alpha = texcol.a * i.color.a;
398.     clip( alpha - _Cutoff );
399.     //return EncodeDepthNormal (i.nz.w, i.nz.xyz);
400.     return EncodeFloatRGBA(i.nz.w);
401.    
402. }
403. ENDCG
404.     }
405. }
406. Fallback Off
407. }

If I blit in the shader instead (same "dumb" shader that simply reads from _CameraDepthTexture) I get the correct result: that's what is puzzling me.

 

Yep, my suspicions were correct. That line is wrong. You're not setting the replacement tag, which means it's just going to use the first pass of your shader and nothing else.
https://docs.unity3d.com/ScriptReference/Camera.RenderWithShader.html

You want:

Code (csharp):

1. _camera.RenderWithShader((Shader)p_depthCameraShader.objectReferenceValue, "RenderType");

That's telling the system to match the "RenderType" tag on the shaders in the scene with those in the replacement shader itself. If you leave that blank it doesn't know to do that.

 

The "dumb" shader should only be used as a blit because otherwise the

_CameraDepthTexture

might not exist. I'd actually say it's almost a bug that it works at all depending on when you do the blit.

Unity renders the

_CameraDepthTexture

at the start of rendering the scene and essentially throws it out at the end of rendering the scene. If you call

Blit()

during an

Update()

or in the editor there's no guarantee it exists at that time to sample from. So it's kind of a fluke that it works in play mode.

It's totally not surprising that it doesn't work when using

RenderWithShader()

since, again, the

_CameraDepthTexture

shouldn't exist yet. There may even be code somewhere deep in Unity that's unbinding the texture when that runs meaning that even if it does exist it can't be accessed by shaders.

 

Not to hijack this thread - however it sounds like it's part of the same issue I've been having, as when I try to use the

_CameraDepthTexture

it's in a Blit operation during Update.

So we could perhaps use the

_LastCameraDepthTexture

in this case?
https://docs.unity3d.com/Manual/SL-CameraDepthTexture.html

// Wildstyle

 

Uh...
I totally misunderstood the documentation: I thought that passing an empty string would have the behaviour you outlined and by specifying a tag (i.e. "Solid") would have rendered with the shader only that RenderType.
The example in https://docs.unity3d.com/Manual/SL-ShaderReplacement.html now makes way more sense.
Thanks for clarifying it.