Custom Lighting Function only works in transparent render queue?

Hello,

I'm working on a shader for a procedurally generated landscape, that comebines a basic surface shader with vertex colors with a secondary flat colored surface shader with a custom lighting model. Here is my shader code so far:

Code (csharp):

1. Shader "Custom/VertexColorLandscapeShader" {
2.     Properties {
3.         _Smoothness("Smoothness", Range(0,1)) = 0.5
4.         _Metallic("Metallic", Range(0,1)) = 0.0
5.         _Pov("Point of View", Vector) = (0,0,0,0)
6.         _Color("Color Back", Color) = (0,1,0,1)
7.         _CutOffDistance("Cut off at distance", Float) = 32
8.         _FlattenDistance("Flatten at distance", Float) = 31
9.     }
10.     SubShader {
11.         Tags { "RenderType"="Opaque" }
12.         Cull Back
13.    
14.         CGPROGRAM
15.         #pragma surface surf Standard fullforwardshadows
16.         #pragma target 3.0
17.  
18.         struct Input {
19.             float3 worldPos;
20.             float4 color : COLOR;
21.         };
22.  
23.         fixed4 _Color;
24.         float4 _Pov;
25.         float _CutOffDistance;
26.         float _FlattenDistance;
27.  
28.         void surf (Input IN, inout SurfaceOutputStandard o) {
29.             o.Albedo = IN.color;
30.             if(distance(float3(IN.worldPos.x, 0, IN.worldPos.z), float3(_Pov.x, 0, _Pov.z)) > _FlattenDistance-.5) discard;
31.         }
32.         ENDCG
33.  
34.         Cull Back
35.  
36.         CGPROGRAM
37.         #pragma surface surf _Uniform
38.         #pragma vertex vert
39.         #pragma target 3.0
40.         #include "UnityPBSLighting.cginc"
41.  
42.         struct Input {
43.             float3 worldPos;
44.         };
45.  
46.         fixed4 _Color;
47.         float4 _Pov;
48.         float _CutOffDistance;
49.         float _FlattenDistance;
50.  
51.         inline half4 Lighting_Uniform(SurfaceOutputStandard s, half3 viewDir, UnityGI gi) {
52.             half4 c;
53.             c.rgb = s.Albedo * (_LightColor0.rgb + UNITY_LIGHTMODEL_AMBIENT.rgb);
54.             c.a = 1;
55.             return c;
56.         }
57.  
58.         //New Standard Lighting model requires a GI function
59.         inline void Lighting_Uniform_GI(
60.             SurfaceOutputStandard s,
61.             UnityGIInput data,
62.             inout UnityGI gi)
63.         {
64.             gi = UnityGlobalIllumination (data, s.Occlusion, s.Smoothness, s.Normal);
65.         }
66.  
67.         void vert(inout appdata_full v, out Input o) {
68.             UNITY_INITIALIZE_OUTPUT(Input,o);
69.             float3 worldPos = mul(unity_ObjectToWorld, v.vertex);
70.             if(distance(float3(worldPos.x, 0, worldPos.z), float3(_Pov.x, 0, _Pov.z)) > _FlattenDistance) v.vertex.y = -0.5;
71.         }
72.  
73.         void surf (Input IN, inout SurfaceOutputStandard o) {
74.             o.Albedo = _Color;
75.             if(distance(float3(IN.worldPos.x, 0, IN.worldPos.z), float3(_Pov.x, 0, _Pov.z)) < _FlattenDistance-.5) discard;
76.             if(distance(float3(IN.worldPos.x, 0, IN.worldPos.z), float3(_Pov.x, 0, _Pov.z)) > _CutOffDistance+.5) discard;
77.             if(IN.worldPos.y < -0.2) discard;
78.         }
79.         ENDCG
80.    
81.     }
82.     FallBack "Diffuse"
83. }

The problem is: it only works on the transparent render queue. when set to anything else (geometry or alpha test) the parts with the custom lighting are not drawn at all. There are no errors or warnings, they are just invisible (the solid grey parts in the first image):



set to geometry:



I've got no idea why this is happening, and I'd be happy about any hints in the right direction.

Thank you.

 

(Edited the question and thread title since my problem evolved, and my first attempt was badly described anyways)

 

Are you using the deferred rendering path?

Custom lighting functions don't work in deferred, so it'll fall back to forward rendering when it's used normally. However in this case because there are two Surface shaders the first one is generating a deferred pass. If the render path is set to deferred and a shader had a deferred pass, it ignores all forward passes.

 

You are right, bgolus, switching to forward path solved it. Thank you.

Sadly, SSAO and some other image effects don't work correctly in forward path with cutout shaders: https://issuetracker.unity3d.com/issues/ssao-in-forward-rendering-does-not-use-cutout-from-shader

SSAO is visible on transparent pixels:



The documentation is very vague on custom lighting in deferred path, but I found this: https://docs.unity3d.com/Manual/SL-SurfaceShaderLighting.html
I have no idea how to get the desired effect out of this. What I got so far is:

Code (csharp):

1.  
2.     inline half4 Lighting_Uniform_Deferred (SurfaceOutputStandard s, UnityGI gi, out half4 outDiffuseOcclusion, out half4 outSpecSmoothness, out half4 outNormal) {
3.         outDiffuseOcclusion = half4(_Color.rgb, 1);
4.         outSpecSmoothness = half4(0,0,0,0);
5.         outNormal = half4(1,1,1,0);
6.  
7.         return outDiffuseOcclusion;
8.     }
9.  

It works, but is far from what I tried to achieve:



Maybe my best chance is to wait for unity to resolve the SSAO issue and disable it for now. Or is there a way to get a flat color effect with a deferred custom lighting function?

 

The bug that person is having is you need to set your shader to use Tags { "RenderType"="TransparentCutout" } to let Unity know it needs to be treated as a cutout for the depth normals texture, or if you're doing vertex modifiers you need to add a custom RenderType and update the Internal-DepthNormalsTexture.shader and override it in the graphics settings.

Yeah, I just skip surface shaders entirely when I'm doing custom deferred stuff. However try setting the outDiffuseOcclusion to half4(0,0,0,1) and returning the same half4(_Color.rgb, 1.0) you already are. You don't want to set the diffuse color since you don't want it to be affected by lighting. The value returned by that function should be the ambient lighting * diffuse color + emission normally. In this case since you just want a solid color, just return the solid color.

 

Thank you for your help so far. I tried some of your suggestions:

Well, the second part of that went right over my head. I tried setting RenderType to TransparentCutout, but this does not do anything in regards to SSAO (or any other image effects BTW).

That works - except it misses with SSAO as well. Still this seems to be the most promising direction to take, so I will experiment with it some more. Thanks again bgolus!