Pre-rendered depth doesn't work with perspective camera

I am working on a project where I supply depth values to the depth buffer from depth values stored in a png file rendered out from Maya. It's the effect they used a lot in the old days so they could get dynamic objects to cull with a pre-rendered 3D scene.

Anyway, I can render an orthographic scene and the depth values work as expected, but as soon as I try it with a perspective camera it's a no go. It acts like the depth is way towards the front of the camera, but I've tried scaling and offsetting values and nothing works as predicted.

Is there something that I don't understand regarding depth values? Aren't they just values that range from 0 to 1? I've looked all over for answers and nothing seems to touch on this subject specifically.

The following shader is called by Graphics.Blit in my post effect that transfers the depth of my png into the depth buffer. Does anything look out of place? Do I need to adjust the values before transferring the depth from the image and into the buffer?

Code (CSharp):

1. Shader "Hidden/DepthPainting"
2. {
3.     Properties
4.     {
5.         _MainTex ("Texture", 2D) = "white" {}
6.         _PaintingTex ("Painting", 2D) = "white" {}
7.         _PaintingDepthTex ("Painting Depth", 2D) = "white" {}
8.         _DepthScale ("Depth Scale", Float) = 1.0
9.         _DepthOffset ("Depth Offset", Float) = 0.0
10.         _DepthInvert ("Depth Invert", int) = 0.0
11.     }
12.  
13.     SubShader
14.     {
15.         Cull Off
16.         ZWrite On
17.         ZTest Off
18.  
19.         Blend SrcAlpha OneMinusSrcAlpha
20.  
21.         Pass
22.         {
23.             CGPROGRAM
24.             #pragma vertex vert
25.             #pragma fragment frag
26.            
27.             #include "UnityCG.cginc"
28.  
29.             sampler2D _MainTex;
30.             float4  _MainTex_TexelSize;
31.             sampler2D _PaintingTex;
32.             sampler2D _PaintingDepthTex;
33.  
34.             float _DepthScale;
35.             float _DepthOffset;
36.             int _DepthInvert;
37.  
38.             struct appdata
39.             {
40.                 float4 vertex : POSITION;
41.                 float2 uv : TEXCOORD0;
42.             };
43.  
44.             struct v2f
45.             {
46.                 float2 uv : TEXCOORD0;
47.                 float4 vertex : SV_POSITION;
48.             };
49.  
50.             struct fo
51.             {
52.                 half4 color : COLOR;
53.                 float depth : DEPTH;
54.             };
55.  
56.             v2f vert (appdata v)
57.             {
58.                 v2f o;
59.  
60.                 o.vertex = UnityObjectToClipPos(v.vertex);
61.                 o.uv = v.uv;
62.  
63.                 if(_MainTex_TexelSize.y < 0)
64.                 {
65.                     o.uv.y = 1.0 - o.uv.y;
66.                 }
67.  
68.                 return o;
69.             }
70.  
71.             fo frag (v2f i)
72.             {
73.                 // fragment out
74.                 fo o;
75.  
76.                 half4 p = tex2D(_PaintingTex, i.uv);
77.                 float d = (tex2D(_PaintingDepthTex, i.uv).r);
78.  
79.                 o.color = p;
80.                 o.depth = d * _DepthScale + _DepthOffset;
81.  
82.                 #ifdef UNITY_REVERSED_Z
83.                     o.depth = 1 - o.depth;
84.                 #endif
85.  
86.                 if(_DepthInvert == 1.0)
87.                 {
88.                     o.depth = 1 - o.depth;
89.                 }
90.  
91.                 return o;
92.             }
93.             ENDCG
94.         }
95.     }
96. }

 

Here's a picture that represents a debug scene I created showing the issue. White represents the far plan, and grey (0.5, 0.5, 0.5) should be exactly in between the near and far planes. It works for orthographic cameras, but not for perspective.

 

Hi aleksandrk,

From what I just read it sounds like non-linear depth is the distance from the camera/eye origin to the fragment, and linear is distance to the camera/eye plane. I thought it just stored depth one way. I will play around with some things and come back if I have any more questions.

Thanks

 

My hat goes off to the math guys who came up with all this stuff. I thought I had a pretty good understanding of the graphics pipeline, that is until I hit the depth buffer challenge .

The depth buffer is still a little bit of a mystery to me, but, I did stumble upon a solution. My idea was to convert the linear depth value to view space, multiply it with the projection matrix, then store it into SV_Depth. Here was the conversion (which for those excited for a solution, this isn't it):

Code (CSharp):

1. float4 vertex = i.vertex; // initially assign entire incoming vertex pos
2.  
3. // assign my pre rendered z position (I was just guessing with the w at this point)
4. vertex.z = preRenderedZ * vertex.w;
5.  
6. vertex = mul(UNITY_MATRIX_P, vertex)
7.  
8. o.depth = vertex.z;

So yeah, that didn't work

Then all this got me wondering about what w was for in the first place, so I went on a quest to learn more about it. I found this site which explains it pretty well.
https://learnopengl.com/#!Getting-started/Coordinate-Systems

However, that still didn't help me find what I was looking for, so I did a search for "unity pre rendered depth". Duh right? Well, I did searches similar to that before but the sites were always about enabling depth in a render texture, which isn't what I wanted. This time I found this post (it's the answer at the bottom that is of interest):
https://gamedev.stackexchange.com/questions/97034/3d-game-with-pre-rendered-background

At the bottom they guy points everyone to this place, which was in fact what I was looking for:
http://mehm.net/blog/?p=1218

It took a while to figure this one out, but well worth it, because now I know what w is used for and I know about linear and non-linear depth buffers.

 

I noticed that there is a depth sacle and a offset in your code. Why there is such a thing? I find that my orthographic view need a scale and offset to work, but I don't know why.