Rendering deferred fog only where useful

Hello,

The deferred fog, in our implementation, is currently a full screen quad pass that blends with the skybox for nice effects. But on most pixels, it actually does nothing since there is no fog effect to perform as pixels are too close. I'd like to use the depth buffer to avoid executing the fragment shader on those pixels.

For this purpose I started modifying the vertex shader so that the Z value corresponds to the start of the fog in clip space. I thought it would be: (fogStart - nearClip) / (farClip - nearClip). But it doesn't work! I wonder if there is something involved due to perspective division, reverse Z or something.

I started poking around values just to see if the concept would work and it does. For a fog start of 300, a near clip of 5 and a far clip of 2500, FOV at 45°, a Z value of 0.0063 properly culls all the pixels just as if I had clipped in the fragment shader. But I have no idea how to come up with that value from the parameters that I know of (near clip, far clip, fog start)



In this screenshot, we see that the fragment shader is not executed on pixels too close. The green/orange is debugging values. Green shows where the fog computation is supposed to occur and red shows how much fog to apply.

Looking at the cameraProjection matrix in the frame debugger, I notice that it is not affected at all by near and far clip values. It's quite an unconventional projection matrix. Further digging shows that the actual projection UNITY_MATRIX_P is actually called glstate_projection_matrix and it's also quite weird. It looks like the usual A and B values of the matrix are A = near / (far - near) and B = near. What kind of perspective projection matrix is Unity using?



Any idea how to compute a proper Z value for depth testing to work for fog?

 

You want to convert a view space depth to clip space depth? Mostly you just need to apply the

UNITY_MATRIX_P

and you’re done... correcting for clip space w.

Code (csharp):

1. // view space fog start position, x&y don’t matter, w must be 1
2. float4 fogStartView = float4(0,0, _FogStart, 1);
3.  
4. // apply projection transform to view space position to transform into clip space
5. float4 fogStartClip = mul(UNITY_MATRIX_P, fogStartView);
6.  
7. // do perspective divide (transform from clip to normalized device coordinates, aka NDC)
8. float fogStartNDCZ = fogStartClip.z / fogStartClip.w;
9.  
10. // calculate clip space position of vertex
11. o.pos = UnityObjectToClipPos(v.vertex);
12.  
13. // convert NDC z back to clip space using the vertex clip pos w.
14. o.pos.z = fogStartNDCZ * o.pos.w;

 

A bit of explanation about why the “real” projection matrix looks like it does. First, matrixes in c# are defined in OpenGL form, and Unity transforms that into whatever from is needed for the given platform. So that means if you’re using Windows, it’ll be transformed into a Direct3D projection matrix, but the unity_CameraProjection matrix is the original OpenGL form. In addition Unity uses a reversed Z depth when rendering with anything other than OpenGL, which requires inverting the projection matrix. Look up “reversed z depth” to find out why.

The funny thing is no platform except Android uses OpenGL by default anymore. Windows is D3D11. Linux is Vulkan. MacOS & iOS are both Metal. Android currently defaults to OpenGLES, but you can use Vulkan on most new phones from the last few years. In a few more years I suspect no platform that Unity supports will still use the original OpenGL projection matrix on the GPU.

This all stems from Unity originally being an OpenGL only engine btw.

 

Yes, that's what I also thought initially but surprisingly it doesn't work. This is a post process shader ran from a commandbuffer in BeforeImageEffectsOpaque so it might change things a bit. Here's my code by the way:

Code (CSharp):

1. Shader "Hidden/PostProcessing/DeferredFog"
2. {
3.     HLSLINCLUDE
4.  
5.         #pragma multi_compile __ FOG_LINEAR FOG_EXP FOG_EXP2
6.         #include "../StdLib.hlsl"
7.         #include "Fog.hlsl"
8.  
9.         TEXTURE2D_SAMPLER2D(_MainTex, sampler_MainTex);
10.         TEXTURE2D_SAMPLER2D(_CameraDepthTexture, sampler_CameraDepthTexture);
11.  
12.         #define SKYBOX_THREASHOLD_VALUE 0.9999
13.  
14.         struct AttributesFog
15.         {
16.             float4 vertex : POSITION;
17.             float4 texcoord : TEXCOORD0;
18.             float4 texcoord1 : TEXCOORD1;
19.         };
20.  
21.         struct VaryingsFogFade
22.         {
23.             VaryingsDefault deffault;
24.             float3 ray : TEXCOORD2;
25.         };
26.  
27.         CBUFFER_START(UnityPerFrame)
28.             float4x4 glstate_matrix_projection;
29.         CBUFFER_END
30.  
31.         VaryingsFogFade VertFogFade(AttributesFog v)
32.         {
33.             VaryingsFogFade o;
34.  
35.             float near = _ProjectionParams.y;
36.  
37.             AttributesDefault defAttribs;
38.             defAttribs.vertex = v.vertex;
39.             o.deffault = VertDefault(defAttribs);
40.  
41. #if FOG_LINEAR
42.             float4 projected = mul(glstate_matrix_projection, float4(0, 0, FOG_START, 1));
43.             o.deffault.vertex.z = projected.z / projected.w * o.deffault.vertex.w;
44.             //o.deffault.vertex.z = 0.0063;
45. #else
46.             o.deffault.vertex.z = 0;
47. #endif
48.  
49.             float tanHalfFovX = 1 / unity_CameraProjection[0][0];
50.             float tanHalfFovY = 1 / unity_CameraProjection[1][1];
51.             float3 right = unity_WorldToCamera[0].xyz * near * tanHalfFovX;
52.             float3 top = unity_WorldToCamera[1].xyz * near * tanHalfFovY;
53.             float2 corner = v.vertex.xy;
54. #if UNITY_UV_STARTS_AT_TOP
55.             corner.y = -corner.y;
56. #endif
57.             float3 origin = unity_WorldToCamera[2].xyz * near;
58.             o.ray = origin + corner.x * right + corner.y * top;
59.  
60.             float _SkyRotation = FOG_SKYBOX_ROTATION;
61.             o.ray = RotateAroundYAxis(o.ray, _SkyRotation);
62.  
63.             return o;
64.         }
65.  
66.         float4 Frag(VaryingsDefault i) : SV_Target
67.         {
68.             half4 color = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.texcoordStereo);
69.  
70.             float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, sampler_CameraDepthTexture, i.texcoordStereo);
71.             depth = Linear01Depth(depth);
72.             float dist = ComputeFogDistance(depth);
73.             half fog = 1.0 - ComputeFog(dist);
74.  
75.             return lerp(color, _FogColor, fog);
76.         }
77.  
78.         float4 FragExcludeSkybox(VaryingsDefault i) : SV_Target
79.         {
80.             half4 color = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.texcoordStereo);
81.  
82.             float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, sampler_CameraDepthTexture, i.texcoordStereo);
83.             depth = Linear01Depth(depth);
84.             float skybox = depth < SKYBOX_THREASHOLD_VALUE;
85.             float dist = ComputeFogDistance(depth);
86.             half fog = 1.0 - ComputeFog(dist);
87.  
88.             return lerp(color, _FogColor, fog * skybox);
89.         }
90.  
91.         float4 FragFadeToSkybox(VaryingsFogFade i) : SV_Target
92.         {
93.             half4 color = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.deffault.texcoordStereo);
94.  
95.             float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, sampler_CameraDepthTexture, i.deffault.texcoordStereo);
96.             depth = Linear01Depth(depth);
97.             float dist = ComputeFogDistance(depth) - FOG_START;
98.             half fog = 1.0 - ComputeFog(dist);
99.             float skybox = depth > SKYBOX_THREASHOLD_VALUE;
100.  
101.             half factor = saturate(fog + skybox);
102.             //clip(factor - 0.001);
103.  
104.             return half4(factor, (factor > 0) * 0.2, 0, 1);
105.  
106.             // Look up the skybox color.
107.             half3 skyColor = texCUBE(_SkyCubemap, i.ray);
108.             skyColor *= _FogColor.rgb * _FogColor.a * 4.59479380; // _FogColor.a contains exposure
109.  
110.             // Lerp between source color to skybox color with fog amount.
111.             half4 sceneColor = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.deffault.texcoordStereo);
112.             return lerp(sceneColor, half4(skyColor, 1), factor);
113.         }
114.  
115.     ENDHLSL
116.  
117.     SubShader
118.     {
119.         Cull Off
120.         ZWrite Off
121.         ZTest Less
122.  
123.         Pass
124.         {
125.             HLSLPROGRAM
126.                 #pragma vertex VertDefault
127.                 #pragma fragment Frag
128.             ENDHLSL
129.         }
130.  
131.         Pass
132.         {
133.             HLSLPROGRAM
134.                 #pragma vertex VertDefault
135.                 #pragma fragment FragExcludeSkybox
136.             ENDHLSL
137.         }
138.  
139.         Pass
140.         {
141.             HLSLPROGRAM
142.                 #pragma vertex VertFogFade
143.                 #pragma fragment FragFadeToSkybox
144.             ENDHLSL
145.         }
146.     }
147. }
148.  

Running the math manually shows the value ends up a lot greater than 0.0063. Hence why I started this thread.
EDIT: actually, it's not greater, my memories were wrong. It's less than 0!

 

Whoops, view space is negative Z forward on the GPU, an positive view space Z is guaranteed to behind the camera. Try using

-_FogStart

.

 

Wow, I tried many things but nothing so simple haha! Thank you, it works!