Multipass Cutout Shader appearing black

I have a cutout shader which I am trying to split into a 2 passes. I have reached the limit of the number of allowed textures, so I am attempting to have the alpha calculations occurring in the second pass. A texture is used to calculate the final alpha. However for some reason in the two pass shader the parts that are supposed to be visible (opaque) are rendering as black while the parts that are supposed to be hidden (transparent) are rendering in full color.

Where am I going wrong?

 

Shaders don't work the way you're trying to use them. For the most part they run completely independently from each other with no knowledge of the previous shader pass. By the time you're trying to calculate the alpha the previous pass has already written itself to the frame buffer and the next pass is just rendering on top of that.

Basically, your first pass renders, writes to the frame buffer as fully opaque as the color it calculated. Then the second pass renders with alpha, but since you aren't setting a color it's rendering as black on top of the previous pass. You have to do it as a single pass.*

* Except if you get sneaky.

Terrain shaders work around this by rendering only 4 layers at a time. The first pass is opaque, and the additional passes are rendered as alpha passes on top (actually additive passes, but the layer masks are pre-processed to ensure it renders black in the places later passes will render on top of). This means all passes need to sample the alpha texture.

The second option is to do the alpha as a depth write pass and have the second pass be the one that does the rest of the shader code. This order is important because you can set the Surface Shader to use

ZTest Equal

which tells the GPU to only draw where the depth buffer is already filled with values identical to what is being written, which really only happens when you draw the same geometry twice like you're already doing. Since the first pass is doing the alpha testing, it won't write depth in the transparent areas and the Surface Shader won't render there.

(Also, it should be noted that Surface Shaders aren't really single passes either. They're vertex fragment shader generators that can generate up to 7 passes each!)

 

Thank you @bgolus , that explanation helps a lot. Essentially yes I am trying to have one pass for all of the textures and color (although some of the textures do have an alpha channel/transparency) and then another which does an overall cutout of everything based on an alpha texture.

I have updated the shader based on your advice and moved the transparent pass (Pass 2) to be first and added ZTest Equal to it but this is unfortunately causing everything to be transparent. I have also tried the other ZTest settings but unfortunately the opaque parts stay either black or become transparent.

Is there anything that I am missing? For example would using a Blending mode help?
(I have also removed the redundant parts of the shader to make it easier to read.)

Code (CSharp):

1. Shader "Test 2 Pass" {
2. Properties {
3.     _SpecColor ("Specular Color", Color) = (0, 0, 0, 1)
4.     _Shininess ("Shininess", Range (0.01, 1)) = 0.078125
5.     _Color ("Overall Color", Color) = (0.8,0.8,0.8,1)
6.     _FirstTex ("First Texture (RGB) TransGloss (A)", 2D) = "white" {}
7.     _MainTex ("Transparency", 2D) = "white" {}
8.     _Cutoff ("Alpha cutoff", Range(0,1)) = 0.1
9.     _BumpMap ("Normal Map 1", 2D) = "bump" {}
10.     // Slider to control fading between normalmaps
11.     _BumpMapSlider ("Normal Slider", Range (0, 1)) = 0
12.     // Second normal map
13.     _BumpMap2 ("Normal Map 2", 2D) = "bump" {}
14.     _RimColor ("Rim Color", Color) = (0,0,0,1)
15.     _RimPower ("Rim Power", Range(0.0,8.0)) = 1.0
16. }
17.  
18. SubShader {
19.     Tags {"Queue"="AlphaTest" "IgnoreProjector"="True" "RenderType"="TransparentCutout"}
20.     LOD 400
21.  
22.     // Pass 2
23.     //Pass {
24.         ZTest Equal
25.         Name "Pass 2"  
26.         CGPROGRAM
27.         //#pragma surface surf BumpSpecSkin alphatest:_Cutoff
28.         #pragma surface surf BlinnPhong alphatest:_Cutoff
29.         //#pragma exclude_renderers flash
30.         #pragma target 3.0
31.  
32.         sampler2D _FirstTex;
33.         sampler2D _MainTex;
34.         fixed4 _Color;
35.  
36.         struct Input {
37.             float2 uv_MainTex;
38.         };
39.  
40.         void surf (Input IN, inout SurfaceOutput o) {
41.             o.Alpha = tex2D(_MainTex, IN.uv_MainTex).a; //"_MainTex" name must be used for alpha to work correctly  
42.         }
43.     ENDCG
44.     //}
45.  
46.     // Pass 1
47.     //Pass {
48.         Name "Pass 1"  
49.         CGPROGRAM
50.         //#pragma surface surf BumpSpecSkin alphatest:_Cutoff
51.         #pragma surface surf BlinnPhong alphatest:_Cutoff
52.         //#pragma exclude_renderers flash
53.         #pragma target 3.0
54.  
55.         sampler2D _FirstTex;
56.         //sampler2D _MainTex;
57.         sampler2D _BumpMap;
58.         fixed4 _Color;
59.         half _Shininess;
60.         float _BumpMapSlider;
61.         sampler2D _BumpMap2;
62.         float4 _RimColor;
63.         float _RimPower;
64.  
65.         struct Input {
66.             float2 uv_FirstTex;
67.             float2 uv_BumpMap;
68.             float3 viewDir;
69.         };
70.  
71.         void surf (Input IN, inout SurfaceOutput o) {
72.             half4 c = tex2D(_FirstTex, IN.uv_FirstTex) * _Color; //Use the same uv set (IN.uv_FirstTex) for all textures with the same uvs to lower the number of interpolators used
73.             o.Albedo = c.rgb * _Color.rgb;
74.             o.Gloss = c.a;
75.             //o.Alpha = c.a; //"_MainTex" name must be used for alpha to work correctly
76.             o.Specular = _Shininess;
77.             // Read values from _BumpMap and _BumpMap2
78.             fixed3 normal1 = UnpackNormal(tex2D(_BumpMap, IN.uv_BumpMap));
79.             fixed3 normal2 = UnpackNormal(tex2D(_BumpMap2, IN.uv_BumpMap));
80.             // Interpolate between them and set the result to the o.Normal
81.             o.Normal = lerp(normal1, normal2, _BumpMapSlider);
82.             half4 rim = 1.0 - saturate(dot (IN.viewDir, o.Normal));
83.             o.Emission = _RimColor.rgb * pow (rim, _RimPower);  
84.         }
85.     ENDCG
86.     //}
87.  
88. }
89.  
90. FallBack "Legacy Shaders/Transparent/Cutout/Bumped Specular"
91. }
92.  

 

The second pass should be using

ZTest Equal

, the first pass should be

ZTest LEqual

(Less or Equal), which is the default of nothing is defined. The first pass should probably also not be a Surface Shader.

 

Ok I have added the ZTests as suggested but unfortunately the result is still the same black. Also pardon my lack of knowledge on this, but what do you mean that the first pass shouldn't be a surface shader, how do I need to change it?

Code (CSharp):

1. Shader "Test 2 Pass" {
2. Properties {
3.     _SpecColor ("Specular Color", Color) = (0, 0, 0, 1)
4.     _Shininess ("Shininess", Range (0.01, 1)) = 0.078125
5.     _Color ("Overall Color", Color) = (0.8,0.8,0.8,1)
6.     _FirstTex ("First Texture (RGB) TransGloss (A)", 2D) = "white" {}
7.     _MainTex ("Transparency", 2D) = "white" {}
8.     _Cutoff ("Alpha cutoff", Range(0,1)) = 0.1
9.     _BumpMap ("Normal Map 1", 2D) = "bump" {}
10.     // Slider to control fading between normalmaps
11.     _BumpMapSlider ("Normal Slider", Range (0, 1)) = 0
12.     // Second normal map
13.     _BumpMap2 ("Normal Map 2", 2D) = "bump" {}
14.     _RimColor ("Rim Color", Color) = (0,0,0,1)
15.     _RimPower ("Rim Power", Range(0.0,8.0)) = 1.0
16. }
17. SubShader {
18.     Tags {"Queue"="AlphaTest" "IgnoreProjector"="True" "RenderType"="TransparentCutout"}
19.     LOD 400
20.     // Pass 2
21.     //Pass {
22.        ZTest LEqual
23.         Name "Pass 2"
24.         CGPROGRAM
25.         //#pragma surface surf BumpSpecSkin alphatest:_Cutoff
26.         #pragma surface surf BlinnPhong alphatest:_Cutoff
27.         //#pragma exclude_renderers flash
28.         #pragma target 3.0
29.         sampler2D _FirstTex;
30.         sampler2D _MainTex;
31.         fixed4 _Color;
32.         struct Input {
33.             float2 uv_MainTex;
34.         };
35.         void surf (Input IN, inout SurfaceOutput o) {
36.             o.Alpha = tex2D(_MainTex, IN.uv_MainTex).a; //"_MainTex" name must be used for alpha to work correctly
37.         }
38.     ENDCG
39.     //}
40.     // Pass 1
41.     //Pass {
42.        ZTest Equal
43.         Name "Pass 1"
44.         CGPROGRAM
45.         //#pragma surface surf BumpSpecSkin alphatest:_Cutoff
46.         #pragma surface surf BlinnPhong alphatest:_Cutoff
47.         //#pragma exclude_renderers flash
48.         #pragma target 3.0
49.         sampler2D _FirstTex;
50.         //sampler2D _MainTex;
51.         sampler2D _BumpMap;
52.         fixed4 _Color;
53.         half _Shininess;
54.         float _BumpMapSlider;
55.         sampler2D _BumpMap2;
56.         float4 _RimColor;
57.         float _RimPower;
58.         struct Input {
59.             float2 uv_FirstTex;
60.             float2 uv_BumpMap;
61.             float3 viewDir;
62.         };
63.         void surf (Input IN, inout SurfaceOutput o) {
64.             half4 c = tex2D(_FirstTex, IN.uv_FirstTex) * _Color; //Use the same uv set (IN.uv_FirstTex) for all textures with the same uvs to lower the number of interpolators used
65.             o.Albedo = c.rgb * _Color.rgb;
66.             o.Gloss = c.a;
67.             //o.Alpha = c.a; //"_MainTex" name must be used for alpha to work correctly
68.             o.Specular = _Shininess;
69.             // Read values from _BumpMap and _BumpMap2
70.             fixed3 normal1 = UnpackNormal(tex2D(_BumpMap, IN.uv_BumpMap));
71.             fixed3 normal2 = UnpackNormal(tex2D(_BumpMap2, IN.uv_BumpMap));
72.             // Interpolate between them and set the result to the o.Normal
73.             o.Normal = lerp(normal1, normal2, _BumpMapSlider);
74.             half4 rim = 1.0 - saturate(dot (IN.viewDir, o.Normal));
75.             o.Emission = _RimColor.rgb * pow (rim, _RimPower);
76.         }
77.     ENDCG
78.     //}
79. }
80. FallBack "Legacy Shaders/Transparent/Cutout/Bumped Specular"
81. }

 

Here's an example shader with a depth only alpha test vertex fragment pass and a subsequent surface shader that has no alpha testing... but the result is identical to if it did.

Code (CSharp):

1. Shader "Custom/PreAlphaTestPass"
2. {
3.     Properties
4.     {
5.         _Color ("Color", Color) = (1,1,1,1)
6.         _MainTex ("Albedo (RGB) Alpha (A)", 2D) = "white" {}
7.         _Cutoff ("Alpha cutoff", Range(0,1)) = 0.5
8.     }
9.     SubShader
10.     {
11.         Tags {"Queue"="AlphaTest" "IgnoreProjector"="True" "RenderType"="TransparentCutout"}
12.         LOD 200
13.  
14.         // Alpha test depth only pass
15.         Pass {
16.             // default ZTest, here to make sure it's not overriden by the later one
17.             ZTest LEqual
18.  
19.             // only render to depth buffer
20.             ColorMask 0
21.  
22.             CGPROGRAM
23.             #pragma vertex vert
24.             #pragma fragment frag
25.  
26.             #include "UnityCG.cginc"
27.  
28.             struct v2f {
29.                 float4 pos : SV_POSITION;
30.                 float2 uv : TEXCOORD0;
31.             };
32.  
33.             sampler2D _MainTex;
34.             float4 _MainTex_ST;
35.             fixed _Cutoff;
36.             fixed4 _Color;
37.  
38.             // super basic vertex shader
39.             void vert (appdata_base v, out v2f o)
40.             {
41.                 o.pos = UnityObjectToClipPos(v.vertex);
42.                 o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
43.             }
44.  
45.             // frag shader with no return value! it only renders to depth so we don't need one!
46.             void frag(v2f i)
47.             {
48.                 clip(tex2D(_MainTex, i.uv).a * _Color.a - _Cutoff);
49.             }
50.             ENDCG
51.         }
52.  
53.         // only allow rendering where the previous pass rendered to
54.         ZTest Equal
55.  
56.         // shouldn't affect the result, but can be a minor perf win
57.         ZWrite Off
58.  
59.         CGPROGRAM
60.         #pragma surface surf Lambert // note no alphatest here
61.         #pragma target 3.0
62.  
63.         sampler2D _MainTex;
64.         fixed4 _Color;
65.  
66.         struct Input
67.         {
68.             float2 uv_MainTex;
69.         };
70.  
71.         void surf (Input IN, inout SurfaceOutput  o)
72.         {
73.             fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
74.             o.Albedo = c.rgb;
75.             // o.Alpha = c.a; // the alpha, it does nothing!
76.         }
77.         ENDCG
78.     }
79.  
80.     Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"
81. }

 

Thank you @bgolus that has done it! Thank you so much I never would have worked that out.

One final small question; I have noticed that if I change the name of the alpha texture "_MainTex" to something else (eg. "_MainTex1") the transparency is visually different with a dark shadow added. Is there a way to change that? If for example I wanted to use multiple textures for the alpha I cannot have multiple _MainTex.

 

The shadow caster is coming from the fallback shader, which looks like this:

Code (CSharp):

1.     // Pass to render object as a shadow caster
2.     Pass {
3.         Name "Caster"
4.         Tags { "LightMode" = "ShadowCaster" }
5.  
6. CGPROGRAM
7. #pragma vertex vert
8. #pragma fragment frag
9. #pragma target 2.0
10. #pragma multi_compile_shadowcaster
11. #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
12. #include "UnityCG.cginc"
13.  
14. struct v2f {
15.     V2F_SHADOW_CASTER;
16.     float2  uv : TEXCOORD1;
17.     UNITY_VERTEX_OUTPUT_STEREO
18. };
19.  
20. uniform float4 _MainTex_ST;
21.  
22. v2f vert( appdata_base v )
23. {
24.     v2f o;
25.     UNITY_SETUP_INSTANCE_ID(v);
26.     UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
27.     TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
28.     o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
29.     return o;
30. }
31.  
32. uniform sampler2D _MainTex;
33. uniform fixed _Cutoff;
34. uniform fixed4 _Color;
35.  
36. float4 frag( v2f i ) : SV_Target
37. {
38.     fixed4 texcol = tex2D( _MainTex, i.uv );
39.     clip( texcol.a*_Color.a - _Cutoff );
40.  
41.     SHADOW_CASTER_FRAGMENT(i)
42. }
43. ENDCG
44.  
45.     }

It expects

_MainTex

,

_Color

, and

_Cutoff

values to come from the material and if they don't exist will fall back to default values. For

_Color

or

_Cutoff

those default to zeros which results in either no shadow or a fully solid shadow depending on which one is missing. If there's no

_MainTex

it'll default to a solid white texture with 100% alpha. If you want your shadows to do something else, then you'll want to write a custom shadow caster and use that instead of the

Fallback

. Usually with Surface Shaders this could be solved with including

addshadow

on the

#pragma surface

line, but since we're splitting the alpha testing and the Surface Shader that's not as easy an option here; adding that here won't do anything.

However hopefully you should be able to understand how to modify the above shader to do what you need. Just be sure to add

ZTest LEqual

and

ZWrite On

to the shadow caster pass since we're explicitly disabling it earlier in the shader and those do need to be on for the shadow caster. (

Fallback

shader passes are unaffected by any shader code in the linking shader which is why we didn't have to worry about it before.)

 

Sure that helps a lot thank you for explaining this.

I have been looking at the stats for the 2 Pass shader compared to the original single pass, and they both appear to have the same amount of setpass calls. Shouldn't each additional pass be an extra setpass call?

 

You're asking about my comment of how a Surface Shader can generate up to 7 passes? Not all of those passes are used all of the time. The 7 passes a Surface Shader can generate are:

1. ForwardBase

   - Used for rendering the main directional light and ambient lighting and/or lightmaps for forward rendering.

2. ForwardAdd

   - Used for rendering a point, spot light, or any shadow casting directional light beyond the "main" one for forward rendering.

3. Deferred

   - Used for rendering to the gbuffers used by the deferred rendering path. Only used by opaque objects when rendering with the deferred rendering path.

4. PrepassBase

   - Used for rendering gbuffers for the legacy deferred "prepass" rendering path.

5. PrepassFinal

   - Used for final rendering for the legacy deferred "prepass" rendering path.

6. ShadowCaster

   - Used for rendering shadow maps and the camera depth texture when using forward rendering.

7. Meta

   - Used for rendering of lightmaps.

During game rendering, only 2 or 3 of these passes from a shader are used at any time for an object.

For forward rendering, only the

ForwardBase

,

ForwardAdd

, and

ShadowCaster

passes are used, and

ForwardAdd

will only be used if there are more than a single directional light in the scene affecting that object. An object can have multiples of either of the forward passes in a

SubShader

and Unity will use them all.

For deferred rendering path only the

Deferred

and

ShadowCaster

passes are used. Only the first

Deferred

pass in the

SubShader

will get used, all others will be ignored. Any objects that render as transparent, or don't have a

Deferred

pass will fall back to its forward passes.

The prepass passes are mostly deprecated. They're a type of deferred rendering that has been essentially abandoned since Unity 5.0 was released. But both passes would be used for all opaque objects. Like the newer deferred rendering path, any transparent objects or those that don't have the necessary prepass passes will use the forward passes instead.

The

ShadowCaster

pass should be self explanatory; used at least once for each shadow casting light that affects it, potentially up to 6 times for point lights per mesh, 1 to 4 times for the main directional light depending on how many cascades you've selected to use, and once for any spot or additional directional light. It also gets called when generating the camera depth texture for the forward rendering path which gets used for forward shadow receiving on desktop and consoles, and for post processing. For deferred rendering it'll additionally use the

ShadowCaster

pass to fill in the gbuffer for any opaque objects that aren't deferred compatible. Only the first shadow caster pass in the

SubShader

will be used. Surface Shaders only generate a

ShadowCaster

pass if using the

addshadow

option.

The

Meta

pass is, as mentioned, used for lightmap rendering. It's only ever used when baking lightmaps and not during realtime game rendering. Only the first Meta pass is used.


So, comparing the two shaders, your original shader with two Surface Shaders would have potentially been generating 12 passes, most of which would end up unused. However it would have had two

ForwardBase

passes that you would have seen being rendered accounting for 2 set pass calls in a scene with only a directional light. If you added a point light to the scene now it would have been 4 set pass calls (2

ForwardBase

and 2

FowardAdd

). (This is ignoring the multiple set pass calls using the shadow caster pass for the camera depth texture and shadow maps.)


With my shader the alpha test pass uses an

Always

pass (the default

"LightMode"

), which is a bit of a misnomer as it's really a ForwardUnlit pass. No mater how many lights are in the scene, it'll always render once. Then the Surface Shader will render once using the

ForwardBase

pass. That would still only be 2 set pass calls for a scene with only a directional light, and only 3 if you added a point light. (Again ignoring the shadow caster pass.)

 

Yes there is only a directional light in the scene, so I had assumed that the new 2 pass shader we made would be 2 set pass calls, while the original single pass shader would be 1 set pass call. However when I simulate the game and switch the material between the two shaders, the number of set pass calls doesn't change. So I was just curious.

 

You mean the original shader without alpha support is two passes? Not sure I have an answer for that.

 

No sorry I meant that it appears they are both rendering in 1 set pass each.