Custom Shadow Caster and Collector Pass

Hi Guys, I'm writing a bit of a complex shader that almost sort of matches the standard shader, but it has vertex animation. This is meant to be a shader for a tree branch so I can get some nice wind animations. It's worth noting too that I am working in the standard unity pipeline and not URP, LWRP, HDRP, or any other mess of an SRP. This is also for a VR project as well, and forward rendering is the path that I have to stick with.

I'm close to completion however I've run into a problem, and it seems that there isn't much info on this on the internet so I'm forced to ask here on the forums. Basically, I have a shader that has custom vertex animation on it. Now for this shader, I want it to cast real-time shadows, and also receive them as well. (In addition to the shader later on I also plan on adding lightmapping support but that's beside the point here) Now adding a fallback or a UsePass from an existing built-in shader does the trick. HOWEVER, there is an issue. The shadows are essentially stuck in place and it seems like I'm gonna have to create my own shadow caster pass because I'm receiving the following issues similar to what this person received as well when messing about with the vertices. - https://forum.unity.com/threads/shadow-artifacts-on-vertex-animation-shader-solved.541292/

The shader I'm writing however is not a surface shader. It's a completely custom shader with a vertex and fragment function. Now the problem with it is that I cannot seem to progress because of an issue that seems to be because of Unity's CGINC files. The error for the record is "undeclared identifier 'TRANSFER_SHADOW'". I've looked in the CGINC files and tried defining TRANSFER_SHADOW myself but to no success. Even with the solution that I tried from the following forum post here - https://forum.unity.com/threads/unrecognized-identifier-shadow_coords-error-in-2017-3.509446/

I've have also been through the unity docs here - https://docs.unity3d.com/Manual/SL-VertexFragmentShaderExamples.html - and still get the same errors.

Image - https://imgur.com/a/enrL8F2

Here is the W.I.P Shader for more context.

Code (CSharp):

1. Shader "Unlit/TreeBranch"
2. {
3.     Properties
4.     {
5.         _Color("Color", Color) = (1,1,1,1)
6.  
7.         _MainTex("Base (RGB) Transparency (A)", 2D) = "white" {}
8.  
9.         _BumpMap("Normalmap", 2D) = "bump" {}
10.  
11.         //note to self: this has to be named _Cutoff or else shadows won't work well
12.         _Cutoff("Alpha Cutoff", Range(0.0, 1.0)) = 0.5
13.  
14.         _Smoothness("Smoothness", Range(0.0, 1.0)) = 0.5
15.  
16.         _VertexAnimationMask("Vertex Animation Mask", 2D) = "white" {}
17.  
18.         [Header(Vertex Animation)]
19.         _NoiseTex("Noise Texture", 2D) = "white" {}
20.         _WindAmount("Wind Amount", Range(0.0, 1.0)) = 0.5
21.         _WindSpeed("Wind Speed", Vector) = (1.0, 1.0, 1.0, 1.0)
22.         _WindScale("Wind Scale", Vector) = (1.0, 1.0, 1.0, 1.0)
23.     }
24.  
25.     CGINCLUDE
26.     #include "UnityCG.cginc"
27.     #include "UnityLightingCommon.cginc"
28.     #include "Lighting.cginc"
29.     #include "AutoLight.cginc"
30.     #include "UnityStandardBRDF.cginc"
31.  
32. //forum solution
33. #ifndef AUTOLIGHT_FIXES_INCLUDED
34. #define AUTOLIGHT_FIXES_INCLUDED
35.  
36. #include "HLSLSupport.cginc"
37. #include "UnityShadowLibrary.cginc"
38.  
39.             // Problem 1: SHADOW_COORDS - undefined identifier.
40.             // Why: Using SHADOWS_DEPTH without SPOT.
41.             // The file AutoLight.cginc only takes into account the case where you use SHADOWS_DEPTH + SPOT (to enable SPOT just add a Spot Light in the scene).
42.             // So, if your scene doesn't have a Spot Light, it will skip the SHADOW_COORDS definition and shows the error.
43.             // Now, to workaround this you can:
44.             // 1. Add a Spot Light to your scene
45.             // 2. Use this CGINC to workaround this scase.  Also, you can copy this in your own shader.
46. #if defined (SHADOWS_DEPTH) && !defined (SPOT)
47. #       define SHADOW_COORDS(idx1) unityShadowCoord2 _ShadowCoord : TEXCOORD##idx1;
48. #endif
49.  
50.  
51. // Problem 2: _ShadowCoord - invalid subscript.
52. // Why: nor Shadow screen neighter Shadow Depth or Shadow Cube and trying to use _ShadowCoord attribute.
53. // The file AutoLight.cginc defines SHADOW_COORDS to empty when no one of these options are enabled (SHADOWS_SCREEN, SHADOWS_DEPTH and SHADOWS_CUBE),
54. // So, if you try to call "o._ShadowCoord = ..." it will break because _ShadowCoord isn't an attribute in your structure.
55. // To workaround this you can:
56. // 1. Check if one of those defines actually exists in any place where you have "o._ShadowCoord...".
57. // 2. Use the define SHADOWS_ENABLED from this file to perform the same check.
58. #if defined (SHADOWS_SCREEN) || defined (SHADOWS_DEPTH) || defined (SHADOWS_CUBE)
59. #    define SHADOWS_ENABLED
60. #endif
61.  
62. #endif
63.  
64.     sampler2D _MainTex;
65.     sampler2D _NoiseTex;
66.     float4 _MainTex_ST;
67.  
68.     sampler2D _BumpMap;
69.     sampler2D _VertexAnimationMask;
70.  
71.     //note to self: this has to be named _Cutoff or else shadows won't work well
72.     float _Cutoff;
73.  
74.     float _Smoothness;
75.  
76.     float4 _Color;
77.  
78.     float4 _WindSpeed;
79.     float4 _WindScale;
80.     float _WindAmount;
81.  
82.     struct appdata
83.     {
84.         float4 vertex : POSITION;
85.         float2 uv : TEXCOORD0;
86.         float4 tangent : TANGENT;
87.         float3 normal : NORMAL;
88.     };
89.  
90.     struct v2f_base
91.     {
92.         float4 pos : SV_POSITION;
93.  
94.         float2 uv : TEXCOORD0;
95.         float4 posWorld : TEXCOORD1; //world space position
96.  
97.         half3 tspace0 : TEXCOORD2; //tangent space 0
98.         half3 tspace1 : TEXCOORD3; //tangent space 1
99.         half3 tspace2 : TEXCOORD4; //tangent space 2
100.  
101.         LIGHTING_COORDS(5, 6)
102.         UNITY_FOG_COORDS(7)
103.     };
104.  
105.     struct v2f_shadow
106.     {
107.         V2F_SHADOW_CASTER;
108.         float2  uv : TEXCOORD1;
109.         UNITY_VERTEX_OUTPUT_STEREO
110.     };
111.  
112.     float Noise3D(sampler2D noiseTex, float3 pos) //moved to textured noise
113.     {
114.         float p = floor(pos.z);
115.         float f = (pos.z - p);
116.  
117.         float invNoiseRes = 1.0 / 32;
118.         float zStretch = 40.0f * invNoiseRes;
119.         float2 coord = pos.xy * invNoiseRes + (p * zStretch);
120.         float2 nise = float2(tex2Dlod(noiseTex, float4(coord.x, coord.y, 0, 0)).x, tex2Dlod(noiseTex, float4(coord.x + zStretch, coord.y + zStretch, 0, 0)).x);
121.  
122.         float final = saturate(lerp(nise.x, nise.y, f));
123.         return final;
124.     }
125.  
126.     v2f_shadow vert_shadow(appdata_base v)
127.     {
128.         v2f_shadow o;
129.         UNITY_SETUP_INSTANCE_ID(v);
130.         UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
131.         TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
132.         o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
133.         return o;
134.     }
135.  
136.     float4 frag_shadow(v2f_shadow i) : SV_Target
137.     {
138.         fixed4 texcol = tex2D(_MainTex, i.uv);
139.         clip(texcol.a - _Cutoff);
140.  
141.         SHADOW_CASTER_FRAGMENT(i)
142.     }
143.  
144.     v2f_base vert_base(appdata v)
145.     {
146.         v2f_base o;
147.  
148.         float2 uvCoords = TRANSFORM_TEX(v.uv, _MainTex);
149.         float vertexMask = tex2Dlod(_VertexAnimationMask, float4(uvCoords, 0, 0)).r;
150.  
151.         float3 vertexPosition = UnityObjectToClipPos(v.vertex).xyz;
152.         float4 worldPosition = mul(unity_ObjectToWorld, v.vertex);
153.  
154.         worldPosition.xyz *= _WindScale.xyz * _WindScale.w;
155.         worldPosition.xyz += float3(sin(_WindSpeed.x * _Time.x), sin(_WindSpeed.y * _Time.y), sin(_WindSpeed.z * _Time.z)) * _WindSpeed.w;
156.  
157.         float noise = Noise3D(_NoiseTex, worldPosition.xyz);
158.         noise = saturate(noise);
159.         noise *= vertexMask;
160.         noise *= _WindAmount;
161.  
162.         v.vertex.xyz += noise;
163.  
164.         o.pos = UnityObjectToClipPos(v.vertex);
165.         o.uv = uvCoords;
166.  
167.         //define our world position vector
168.         o.posWorld = worldPosition;
169.  
170.         //the world normal of the mesh
171.         half3 worldNormal = UnityObjectToWorldNormal(v.normal);
172.  
173.         //the tangents of the mesh
174.         half3 worldTangent = UnityObjectToWorldDir(v.tangent.xyz);
175.  
176.         // compute bitangent from cross product of normal and tangent
177.         half tangentSign = v.tangent.w * unity_WorldTransformParams.w;
178.         half3 worldBiTangent = cross(worldNormal, worldTangent) * tangentSign;
179.  
180.         // output the tangent space matrix
181.         o.tspace0 = half3(worldTangent.x, worldBiTangent.x, worldNormal.x);
182.         o.tspace1 = half3(worldTangent.y, worldBiTangent.y, worldNormal.y);
183.         o.tspace2 = half3(worldTangent.z, worldBiTangent.z, worldNormal.z);
184.  
185.         UNITY_TRANSFER_FOG(o, o.pos);
186.  
187.         TRANSFER_VERTEX_TO_FRAGMENT(o);
188.  
189.         return o;
190.     }
191.  
192.     fixed4 frag_base(v2f_base i) : SV_Target
193.     {
194.         //--------------------------- VECTORS -------------------------------
195.         float2 uv = i.uv.xy;
196.         float4 worldPosition = i.posWorld;
197.         float3 viewDirection = normalize(_WorldSpaceCameraPos.xyz - i.posWorld.xyz);
198.         float3 lightDirection = normalize(_WorldSpaceLightPos0.xyz);
199.         float3 halfDirection = normalize(viewDirection + lightDirection);
200.  
201.         //--------------------------- NORMAL MAPPING -------------------------------
202.         //sample our normal map
203.         float4 normalMap = tex2D(_BumpMap, uv);
204.         normalMap.xyz = UnpackNormal(normalMap);
205.  
206.         //calculate our mesh normals with the normal map into consideration
207.         float3 normalDirection;
208.         normalDirection.x = dot(i.tspace0, normalMap.xyz);
209.         normalDirection.y = dot(i.tspace1, normalMap.xyz);
210.         normalDirection.z = dot(i.tspace2, normalMap.xyz);
211.         normalDirection = normalize(normalDirection);
212.  
213.         //--------------------------- ADDITIONAL VECTORS ------------------------------- (some of these vectors need to take normals into account)
214.         float3 reflectionDirection = reflect(-viewDirection, normalDirection);
215.         float NdotL = dot(normalDirection, lightDirection);
216.         float NdotV = dot(normalDirection, viewDirection);
217.         float LdotH = dot(lightDirection, halfDirection);
218.         float HdotV = dot(halfDirection, viewDirection);
219.  
220.         //--------------------------- TEXTURES -------------------------------
221.         float4 albedo = tex2D(_MainTex, i.uv);
222.         float4 specular = float4(0, 0, 0, 0);
223.         float oneMinusReflectivity = 0.0f;
224.         float smoothness = _Smoothness;
225.  
226.         //alpha cutoff
227.         clip(albedo.a - _Cutoff);
228.  
229.         //--------------------------- LIGHTING -------------------------------
230.         float atten = LIGHT_ATTENUATION(i); // Light attenuation + shadows.
231.         float3 ambientColor = ShadeSH9(half4(normalDirection, 1));
232.  
233.         //_LightColor0
234.  
235.         float perceptualRoughness = SmoothnessToPerceptualRoughness(smoothness);
236.         float diffuseLighting = DisneyDiffuse(NdotV, NdotL, LdotH, perceptualRoughness);
237.         diffuseLighting = max(0.0f, diffuseLighting);
238.  
239.         float4 finalColor = float4(0, 0, 0, albedo.a);
240.  
241.         //--------------------------- COMBINE LIGHTING -------------------------------
242.         //finalColor.rgb += diffuseLighting * atten;
243.         finalColor.rgb += _LightColor0.rgb * atten;
244.         finalColor.rgb += ambientColor;
245.         finalColor.rgb *= albedo.rgb;
246.  
247.         // apply fog
248.         UNITY_APPLY_FOG(i.fogCoord, finalColor);
249.  
250.         return finalColor;
251.     }
252.  
253.     ENDCG
254.  
255.     SubShader
256.     {
257.         Tags
258.         {
259.             "Queue" = "AlphaTest"
260.             "RenderType" = "TransparentCutout"
261.             "IgnoreProjector" = "True"
262.             "DisableBatching" = "LODFading"
263.         }
264.  
265.         LOD 100
266.  
267.         //BASE PASS
268.         Pass
269.         {
270.             Tags {"LightMode" = "ForwardBase"}
271.  
272.             CGPROGRAM
273.             #pragma vertex vert_base
274.             #pragma fragment frag_base
275.  
276.             // make fog work
277.             #pragma multi_compile_fog
278.  
279.             #pragma multi_compile_fwdbase
280.             #pragma fragmentoption ARB_precision_hint_fastest
281.             #pragma alphatest:_Cutoff
282.  
283.             #include "UnityCG.cginc"
284.             #include "UnityLightingCommon.cginc"
285.             #include "UnityStandardBRDF.cginc"
286.             #include "Lighting.cginc"
287.             #include "AutoLight.cginc"
288.  
289.             ENDCG
290.         }
291.  
292.         //FORWARD ADD PASS
293.         Pass
294.         {
295.             Tags {"LightMode" = "ForwardAdd"}
296.  
297.             Blend One One
298.  
299.             CGPROGRAM
300.             #pragma vertex vert_base
301.             #pragma fragment frag_base
302.  
303.             // make fog work
304.             #pragma multi_compile_fog
305.  
306.             #pragma multi_compile_fwdadd_fullshadows
307.             #pragma fragmentoption ARB_precision_hint_fastest
308.             #pragma alphatest:_Cutoff
309.  
310.             #include "UnityCG.cginc"
311.             #include "UnityLightingCommon.cginc"
312.             #include "UnityStandardBRDF.cginc"
313.             #include "Lighting.cginc"
314.             #include "AutoLight.cginc"
315.  
316.             ENDCG
317.         }
318.  
319.         //SHADOW CASTER PASS
320.         Pass
321.         {
322.             Tags {"LightMode" = "ShadowCaster"}
323.  
324.             CGPROGRAM
325.             #pragma vertex v2f_shadow
326.             #pragma fragment frag_shadow
327.  
328.             #pragma target 2.0
329.             #pragma multi_compile_shadowcaster
330.             #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
331.             #pragma alphatest:_Cutoff
332.  
333.             #include "UnityCG.cginc"
334.             ENDCG
335.         }
336.     }
337. }
338.  

NOTE: I am looking to solve the problem in the shader, I'm not looking to buy a shader that does this easily for me, I'm not looking to switching this shader to a surface shader, I'm not looking to switch to another SRP just to get a feature, I'm not looking to recreate the tree model or such in favor of SpeedTree or etc.

This is something that I know can be solved without sidestepping the issue, I just need the right puzzle pieces or information and I can complete the shader without needing to spend money, or avoiding the problem altogether. That shouldn't be too much to ask.

 

You have a custom shadow caster vertex shader, which is good, but you aren't doing any of the vertex animation in it that the base pass is. Every pass of the shader needs to do the exact same vertex animation. That'll be a problem even if you do fix the current problem you have. I'll need to read over what you have more to understand exactly the error you're seeing.

 

Yes, you are right, and I realized this right after posting and corrected it. And while that got me part of the way there I still had issues receiving this new shadow that I was creating and still had a lot of issues with the Unity Macros. I tried fallbacks, usepass's, redefining and even recreating some of the functions and macros used in those .cginc files, even removing alot of the code and replacing it with ones that match the Unity Shader Examples that were related to shadow casting, no such luck.

The solution to that was just to basically restructure the shader back once again into each separate pass to reduce the scope of the variables and that fixed the issues I was having with the Macros. That's not to say that I reverted back to the ones you were "supposed" to use though that I still had to redefine some of them in my own custom way in order to get them to work.

Not a fan of what I arrived at but it works great now, and all that is left is just some cleanup.

For anyone who comes across this, this is the final shader that I landed on (still W.I.P as there are more things I want to do to it as of writing) and hopefully, if anyone has issues, this code will help you find the solution.

Code (CSharp):

1. Shader "Unlit/TreeBranch2"
2. {
3.     Properties
4.     {
5.         _Color("Color", Color) = (1,1,1,1)
6.         _MainTex("Base (RGB) Transparency (A)", 2D) = "white" {}
7.         _BumpMap("Normalmap", 2D) = "bump" {}
8.         _Cutoff("Alpha Cutoff", Range(0.0, 1.0)) = 0.5
9.         _Smoothness("Smoothness", Range(0.0, 1.0)) = 0.5
10.         _VertexAnimationMask("Vertex Animation Mask", 2D) = "white" {}
11.         [Header(Vertex Animation)]
12.         _NoiseTex("Noise Texture", 2D) = "white" {}
13.         _WindAmount("Wind Amount", Range(0.0, 1.0)) = 0.5
14.         _WindSpeed("Wind Speed", Vector) = (1.0, 1.0, 1.0, 1.0)
15.         _WindScale("Wind Scale", Vector) = (1.0, 1.0, 1.0, 1.0)
16.     }
17.     CGINCLUDE
18.     #include "UnityCG.cginc"
19.     sampler2D _MainTex;
20.     sampler2D _NoiseTex;
21.     float4 _MainTex_ST;
22.     sampler2D _BumpMap;
23.     sampler2D _VertexAnimationMask;
24.     float _Cutoff;
25.     float _Smoothness;
26.     float4 _Color;
27.     float4 _WindSpeed;
28.     float4 _WindScale;
29.     float _WindAmount;
30.     float Noise3D(sampler2D noiseTex, float3 pos) //moved to textured noise
31.     {
32.         float p = floor(pos.z);
33.         float f = (pos.z - p);
34.         float invNoiseRes = 1.0 / 32;
35.         float zStretch = 40.0f * invNoiseRes;
36.         float2 coord = pos.xy * invNoiseRes + (p * zStretch);
37.         float2 nise = float2(tex2Dlod(noiseTex, float4(coord.x, coord.y, 0, 0)).x, tex2Dlod(noiseTex, float4(coord.x + zStretch, coord.y + zStretch, 0, 0)).x);
38.         float final = saturate(lerp(nise.x, nise.y, f));
39.         return final;
40.     }
41.     float GetWindNoise(float2 texcoord, float4 vertex)
42.     {
43.         float2 uvCoords = TRANSFORM_TEX(texcoord, _MainTex);
44.         float vertexMask = tex2Dlod(_VertexAnimationMask, float4(uvCoords, 0, 0)).r;
45.         float4 worldPosition = mul(unity_ObjectToWorld, vertex);
46.         worldPosition.xyz *= _WindScale.xyz * _WindScale.w;
47.         worldPosition.xyz += float3(sin(_WindSpeed.x * _Time.x), sin(_WindSpeed.y * _Time.y), sin(_WindSpeed.z * _Time.z)) * _WindSpeed.w;
48.         float noise = Noise3D(_NoiseTex, worldPosition.xyz);
49.         noise *= vertexMask;
50.         noise *= _WindAmount;
51.         return noise;
52.     }
53.     ENDCG
54.     SubShader
55.     {
56.         Tags
57.         {
58.             "Queue" = "AlphaTest"
59.             "RenderType" = "TransparentCutout"
60.             "IgnoreProjector" = "True"
61.             "DisableBatching" = "LODFading"
62.         }
63.         LOD 100
64.         //BASE PASS
65.         Pass
66.         {
67.             Tags {"LightMode" = "ForwardBase"}
68.             CGPROGRAM
69.             #pragma vertex vert_base
70.             #pragma fragment frag_base
71.             // make fog work
72.             #pragma multi_compile_fog
73.             #pragma multi_compile_fwdbase
74.             #pragma fragmentoption ARB_precision_hint_fastest
75.             #include "UnityCG.cginc"
76.             #include "UnityLightingCommon.cginc"
77.             #include "UnityStandardBRDF.cginc"
78.             #include "Lighting.cginc"
79.             #include "AutoLight.cginc"
80.             #include "UnityShadowLibrary.cginc"
81.             struct v2f_base
82.             {
83.                 float4 pos : SV_POSITION;
84.                 float2 uv : TEXCOORD0;
85.                 float4 posWorld : TEXCOORD1; //world space position
86.                 half3 tspace0 : TEXCOORD2; //tangent space 0
87.                 half3 tspace1 : TEXCOORD3; //tangent space 1
88.                 half3 tspace2 : TEXCOORD4; //tangent space 2
89.                 DECLARE_LIGHT_COORDS(5)
90.                 unityShadowCoord4 _ShadowCoord : TEXCOORD6;
91.                 UNITY_FOG_COORDS(7)
92.             };
93.             v2f_base vert_base(appdata_tan v)
94.             {
95.                 v2f_base o;
96.                 v.vertex.xyz += GetWindNoise(v.texcoord, v.vertex);
97.                 o.pos = UnityObjectToClipPos(v.vertex);
98.                 o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
99.                 //define our world position vector
100.                 o.posWorld = mul(unity_ObjectToWorld, v.vertex);
101.                 //the world normal of the mesh
102.                 half3 worldNormal = UnityObjectToWorldNormal(v.normal);
103.                 //the tangents of the mesh
104.                 half3 worldTangent = UnityObjectToWorldDir(v.tangent.xyz);
105.                 // compute bitangent from cross product of normal and tangent
106.                 half tangentSign = v.tangent.w * unity_WorldTransformParams.w;
107.                 half3 worldBiTangent = cross(worldNormal, worldTangent) * tangentSign;
108.                 // output the tangent space matrix
109.                 o.tspace0 = half3(worldTangent.x, worldBiTangent.x, worldNormal.x);
110.                 o.tspace1 = half3(worldTangent.y, worldBiTangent.y, worldNormal.y);
111.                 o.tspace2 = half3(worldTangent.z, worldBiTangent.z, worldNormal.z);
112.                 TRANSFER_VERTEX_TO_FRAGMENT(o);
113.                 UNITY_TRANSFER_FOG(o, o.pos);
114.                 return o;
115.             }
116.             fixed4 frag_base(v2f_base i) : SV_Target
117.             {
118.                 //--------------------------- VECTORS -------------------------------
119.                 float2 uv = i.uv.xy;
120.                 float4 worldPosition = i.posWorld;
121.                 float3 viewDirection = normalize(_WorldSpaceCameraPos.xyz - i.posWorld.xyz);
122.                 float3 lightDirection = normalize(_WorldSpaceLightPos0.xyz);
123.                 float3 halfDirection = normalize(viewDirection + lightDirection);
124.                 //--------------------------- NORMAL MAPPING -------------------------------
125.                 //sample our normal map
126.                 float4 normalMap = tex2D(_BumpMap, uv);
127.                 normalMap.xyz = UnpackNormal(normalMap);
128.                 //calculate our mesh normals with the normal map into consideration
129.                 float3 normalDirection;
130.                 normalDirection.x = dot(i.tspace0, normalMap.xyz);
131.                 normalDirection.y = dot(i.tspace1, normalMap.xyz);
132.                 normalDirection.z = dot(i.tspace2, normalMap.xyz);
133.                 normalDirection = normalize(normalDirection);
134.                 //--------------------------- ADDITIONAL VECTORS ------------------------------- (some of these vectors need to take normals into account)
135.                 float3 reflectionDirection = reflect(-viewDirection, normalDirection);
136.                 float NdotL = dot(normalDirection, lightDirection);
137.                 float NdotV = dot(normalDirection, viewDirection);
138.                 float LdotH = dot(lightDirection, halfDirection);
139.                 float HdotV = dot(halfDirection, viewDirection);
140.                 //--------------------------- TEXTURES -------------------------------
141.                 float4 albedo = tex2D(_MainTex, i.uv);
142.                 float4 specular = float4(0, 0, 0, 0);
143.                 float oneMinusReflectivity = 0.0f;
144.                 float smoothness = _Smoothness;
145.                 //alpha cutoff
146.                 clip(albedo.a - _Cutoff);
147.                 //--------------------------- LIGHTING -------------------------------
148.                 float atten = LIGHT_ATTENUATION(i); // Light attenuation + shadows.
149.                 float3 ambientColor = ShadeSH9(half4(normalDirection, 1));
150.                 //_LightColor0
151.                 float perceptualRoughness = SmoothnessToPerceptualRoughness(smoothness);
152.                 float diffuseLighting = DisneyDiffuse(NdotV, NdotL, LdotH, perceptualRoughness);
153.                 diffuseLighting = max(0.0f, diffuseLighting);
154.                 float4 finalColor = float4(0, 0, 0, albedo.a);
155.                 //--------------------------- COMBINE LIGHTING -------------------------------
156.                 //finalColor.rgb += diffuseLighting * atten;
157.                 finalColor.rgb += _LightColor0.rgb * atten;
158.                 finalColor.rgb += ambientColor;
159.                 finalColor.rgb *= albedo.rgb;
160.                 // apply fog
161.                 UNITY_APPLY_FOG(i.fogCoord, finalColor);
162.                 return finalColor;
163.             }
164.             ENDCG
165.         }
166.         //FORWARD ADD PASS
167.         Pass
168.         {
169.             Tags {"LightMode" = "ForwardAdd"}
170.             Blend One One
171.             CGPROGRAM
172.             #pragma vertex vert_base
173.             #pragma fragment frag_base
174.             // make fog work
175.             #pragma multi_compile_fog
176.             #pragma multi_compile_fwdadd_fullshadows
177.             #pragma fragmentoption ARB_precision_hint_fastest
178.             #include "UnityCG.cginc"
179.             #include "UnityLightingCommon.cginc"
180.             #include "UnityStandardBRDF.cginc"
181.             #include "Lighting.cginc"
182.             #include "AutoLight.cginc"
183.             #include "UnityShadowLibrary.cginc"
184.             struct v2f_base
185.             {
186.                 float4 pos : SV_POSITION;
187.                 float2 uv : TEXCOORD0;
188.                 float4 posWorld : TEXCOORD1; //world space position
189.                 half3 tspace0 : TEXCOORD2; //tangent space 0
190.                 half3 tspace1 : TEXCOORD3; //tangent space 1
191.                 half3 tspace2 : TEXCOORD4; //tangent space 2
192.                 DECLARE_LIGHT_COORDS(5)
193.                 unityShadowCoord4 _ShadowCoord : TEXCOORD6;
194.                 UNITY_FOG_COORDS(7)
195.             };
196.             v2f_base vert_base(appdata_tan v)
197.             {
198.                 v2f_base o;
199.                 v.vertex.xyz += GetWindNoise(v.texcoord, v.vertex);
200.                 o.pos = UnityObjectToClipPos(v.vertex);
201.                 o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
202.                 //define our world position vector
203.                 o.posWorld = mul(unity_ObjectToWorld, v.vertex);
204.                 //the world normal of the mesh
205.                 half3 worldNormal = UnityObjectToWorldNormal(v.normal);
206.                 //the tangents of the mesh
207.                 half3 worldTangent = UnityObjectToWorldDir(v.tangent.xyz);
208.                 // compute bitangent from cross product of normal and tangent
209.                 half tangentSign = v.tangent.w * unity_WorldTransformParams.w;
210.                 half3 worldBiTangent = cross(worldNormal, worldTangent) * tangentSign;
211.                 // output the tangent space matrix
212.                 o.tspace0 = half3(worldTangent.x, worldBiTangent.x, worldNormal.x);
213.                 o.tspace1 = half3(worldTangent.y, worldBiTangent.y, worldNormal.y);
214.                 o.tspace2 = half3(worldTangent.z, worldBiTangent.z, worldNormal.z);
215.                 UNITY_TRANSFER_FOG(o, o.pos);
216.                 TRANSFER_VERTEX_TO_FRAGMENT(o);
217.                 return o;
218.             }
219.             fixed4 frag_base(v2f_base i) : SV_Target
220.             {
221.                 //--------------------------- VECTORS -------------------------------
222.                 float2 uv = i.uv.xy;
223.                 float4 worldPosition = i.posWorld;
224.                 float3 viewDirection = normalize(_WorldSpaceCameraPos.xyz - i.posWorld.xyz);
225.                 float3 lightDirection = normalize(_WorldSpaceLightPos0.xyz);
226.                 float3 halfDirection = normalize(viewDirection + lightDirection);
227.                 //--------------------------- NORMAL MAPPING -------------------------------
228.                 //sample our normal map
229.                 float4 normalMap = tex2D(_BumpMap, uv);
230.                 normalMap.xyz = UnpackNormal(normalMap);
231.                 //calculate our mesh normals with the normal map into consideration
232.                 float3 normalDirection;
233.                 normalDirection.x = dot(i.tspace0, normalMap.xyz);
234.                 normalDirection.y = dot(i.tspace1, normalMap.xyz);
235.                 normalDirection.z = dot(i.tspace2, normalMap.xyz);
236.                 normalDirection = normalize(normalDirection);
237.                 //--------------------------- ADDITIONAL VECTORS ------------------------------- (some of these vectors need to take normals into account)
238.                 float3 reflectionDirection = reflect(-viewDirection, normalDirection);
239.                 float NdotL = dot(normalDirection, lightDirection);
240.                 float NdotV = dot(normalDirection, viewDirection);
241.                 float LdotH = dot(lightDirection, halfDirection);
242.                 float HdotV = dot(halfDirection, viewDirection);
243.                 //--------------------------- TEXTURES -------------------------------
244.                 float4 albedo = tex2D(_MainTex, i.uv);
245.                 float4 specular = float4(0, 0, 0, 0);
246.                 float oneMinusReflectivity = 0.0f;
247.                 float smoothness = _Smoothness;
248.                 //alpha cutoff
249.                 clip(albedo.a - _Cutoff);
250.                 //--------------------------- LIGHTING -------------------------------
251.                 float atten = LIGHT_ATTENUATION(i); // Light attenuation + shadows.
252.                 float3 ambientColor = ShadeSH9(half4(normalDirection, 1));
253.                 //_LightColor0
254.                 float perceptualRoughness = SmoothnessToPerceptualRoughness(smoothness);
255.                 float diffuseLighting = DisneyDiffuse(NdotV, NdotL, LdotH, perceptualRoughness);
256.                 diffuseLighting = max(0.0f, diffuseLighting);
257.                 float4 finalColor = float4(0, 0, 0, albedo.a);
258.                 //--------------------------- COMBINE LIGHTING -------------------------------
259.                 //finalColor.rgb += diffuseLighting * atten;
260.                 finalColor.rgb += _LightColor0.rgb * atten;
261.                 finalColor.rgb += ambientColor;
262.                 finalColor.rgb *= albedo.rgb;
263.                 // apply fog
264.                 UNITY_APPLY_FOG(i.fogCoord, finalColor);
265.                 return finalColor;
266.             }
267.             ENDCG
268.         }
269.         //SHADOW CASTER PASS
270.         Pass
271.         {
272.             Tags {"LightMode" = "ShadowCaster"}
273.             ZWrite On
274.             CGPROGRAM
275.             #pragma vertex vert_shadow
276.             #pragma fragment frag_shadow
277.             #pragma target 2.0
278.             #pragma multi_compile_shadowcaster
279.             #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
280.             #include "UnityCG.cginc"
281.             #include "UnityLightingCommon.cginc"
282.             #include "UnityStandardBRDF.cginc"
283.             #include "Lighting.cginc"
284.             #include "AutoLight.cginc"
285.             #include "UnityShadowLibrary.cginc"
286.             struct v2f_shadow
287.             {
288.                 V2F_SHADOW_CASTER_NOPOS UNITY_POSITION(pos);
289.                 float2  uv : TEXCOORD1;
290.                 UNITY_VERTEX_OUTPUT_STEREO
291.             };
292.             v2f_shadow vert_shadow(appdata_tan v)
293.             {
294.                 v2f_shadow o;
295.                 v.vertex.xyz += GetWindNoise(v.texcoord, v.vertex);
296.                 UNITY_SETUP_INSTANCE_ID(v);
297.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
298.                 TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
299.                 o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
300.                 return o;
301.             }
302.             float4 frag_shadow(v2f_shadow i) : SV_Target
303.             {
304.                 fixed4 texcol = tex2D(_MainTex, i.uv);
305.                 clip(texcol.a - _Cutoff);
306.                 SHADOW_CASTER_FRAGMENT(i)
307.             }
308.             ENDCG
309.         }
310.     }
311. }

 

Okay, so the issue you're running across isn't the issue you think you're having. If you click on the error in the console you'll get a little bit more information. Unfortunately the line number being shown is totally bogus, so that's probably sending you on a wild goose chase.

They key bit is this info from the error:

Basically the shadow caster pass isn't compiling ... the forward pass functions. The issue is you're using the Unity 4.0 style forward lighting macros, and they break if they're included in a shadow caster pass.

 

Even though I'm likely gonna stick with what now works, I'm inclined now to ask though. What are the lighting macros that I'm supposed to be using?

 

Generally speaking you don't want to shove everything into the CGINCLUDE block, as tempting as it is. It can work in some specific cases, but you have to be very careful when it comes to any of the forward lighting macros because they're complicated and are all designed only to work in the forward passes. They break (as you were experiencing) when they show up in another pass. Apart from very heavy handedly boxing everything in

#if defined(UNITY_PASS_passtype)

preprocessor blocks, the other solution is to use separate files that hold common code for the forward passes.

Code (csharp):

1. Shader "Unlit/TreeBranch"
2. {
3.     Properties
4.     {
5.         _Color("Color", Color) = (1,1,1,1)
6.  
7.         _MainTex("Base (RGB) Transparency (A)", 2D) = "white" {}
8.  
9.         _BumpMap("Normalmap", 2D) = "bump" {}
10.  
11.         //note to self: this has to be named _Cutoff or else shadows won't work well
12.         _Cutoff("Alpha Cutoff", Range(0.0, 1.0)) = 0.5
13.  
14.         _Smoothness("Smoothness", Range(0.0, 1.0)) = 0.5
15.  
16.         _VertexAnimationMask("Vertex Animation Mask", 2D) = "white" {}
17.  
18.         [Header(Vertex Animation)]
19.         _NoiseTex("Noise Texture", 2D) = "white" {}
20.         _WindAmount("Wind Amount", Range(0.0, 1.0)) = 0.5
21.         _WindSpeed("Wind Speed", Vector) = (1.0, 1.0, 1.0, 1.0)
22.         _WindScale("Wind Scale", Vector) = (1.0, 1.0, 1.0, 1.0)
23.     }
24.  
25.     CGINCLUDE
26.     #include "UnityCG.cginc"
27.     #include "UnityLightingCommon.cginc"
28.     #include "Lighting.cginc"
29.     #include "AutoLight.cginc"
30.     #include "UnityStandardBRDF.cginc"
31.  
32.     #include "HLSLSupport.cginc"
33.     #include "UnityShadowLibrary.cginc"
34.  
35.     sampler2D _MainTex;
36.     sampler2D _NoiseTex;
37.     float4 _MainTex_ST;
38.  
39.     sampler2D _BumpMap;
40.     sampler2D _VertexAnimationMask;
41.  
42.     //note to self: this has to be named _Cutoff or else shadows won't work well
43.     float _Cutoff;
44.  
45.     float _Smoothness;
46.  
47.     float4 _Color;
48.  
49.     float4 _WindSpeed;
50.     float4 _WindScale;
51.     float _WindAmount;
52.  
53.     float Noise3D(sampler2D noiseTex, float3 pos) //moved to textured noise
54.     {
55.         float p = floor(pos.z);
56.         float f = (pos.z - p);
57.  
58.         float invNoiseRes = 1.0 / 32;
59.         float zStretch = 40.0f * invNoiseRes;
60.         float2 coord = pos.xy * invNoiseRes + (p * zStretch);
61.         float2 nise = float2(tex2Dlod(noiseTex, float4(coord.x, coord.y, 0, 0)).x, tex2Dlod(noiseTex, float4(coord.x + zStretch, coord.y + zStretch, 0, 0)).x);
62.  
63.         float final = saturate(lerp(nise.x, nise.y, f));
64.         return final;
65.     }
66.  
67.     struct appdata
68.     {
69.         float4 vertex : POSITION;
70.         float2 uv : TEXCOORD0;
71.         float4 tangent : TANGENT;
72.         float3 normal : NORMAL;
73.     };
74.  
75.     #ifndef UNITY_PASS_SHADOWCASTER
76.     struct v2f_base
77.     {
78.         float4 pos : SV_POSITION;
79.  
80.         float2 uv : TEXCOORD0;
81.         float4 posWorld : TEXCOORD1; //world space position
82.  
83.         half3 tspace0 : TEXCOORD2; //tangent space 0
84.         half3 tspace1 : TEXCOORD3; //tangent space 1
85.         half3 tspace2 : TEXCOORD4; //tangent space 2
86.  
87.         LIGHTING_COORDS(5, 6)
88.         UNITY_FOG_COORDS(7)
89.     };
90.  
91.     v2f_base vert_base(appdata v)
92.     {
93.         v2f_base o;
94.  
95.         float2 uvCoords = TRANSFORM_TEX(v.uv, _MainTex);
96.         float vertexMask = tex2Dlod(_VertexAnimationMask, float4(uvCoords, 0, 0)).r;
97.  
98.         float3 vertexPosition = UnityObjectToClipPos(v.vertex).xyz;
99.         float4 worldPosition = mul(unity_ObjectToWorld, v.vertex);
100.  
101.         worldPosition.xyz *= _WindScale.xyz * _WindScale.w;
102.         worldPosition.xyz += float3(sin(_WindSpeed.x * _Time.x), sin(_WindSpeed.y * _Time.y), sin(_WindSpeed.z * _Time.z)) * _WindSpeed.w;
103.  
104.         float noise = Noise3D(_NoiseTex, worldPosition.xyz);
105.         noise = saturate(noise);
106.         noise *= vertexMask;
107.         noise *= _WindAmount;
108.  
109.         v.vertex.xyz += noise;
110.  
111.         o.pos = UnityObjectToClipPos(v.vertex);
112.         o.uv = uvCoords;
113.  
114.         //define our world position vector
115.         o.posWorld = worldPosition;
116.  
117.         //the world normal of the mesh
118.         half3 worldNormal = UnityObjectToWorldNormal(v.normal);
119.  
120.         //the tangents of the mesh
121.         half3 worldTangent = UnityObjectToWorldDir(v.tangent.xyz);
122.  
123.         // compute bitangent from cross product of normal and tangent
124.         half tangentSign = v.tangent.w * unity_WorldTransformParams.w;
125.         half3 worldBiTangent = cross(worldNormal, worldTangent) * tangentSign;
126.  
127.         // output the tangent space matrix
128.         o.tspace0 = half3(worldTangent.x, worldBiTangent.x, worldNormal.x);
129.         o.tspace1 = half3(worldTangent.y, worldBiTangent.y, worldNormal.y);
130.         o.tspace2 = half3(worldTangent.z, worldBiTangent.z, worldNormal.z);
131.  
132.         UNITY_TRANSFER_FOG(o, o.pos);
133.  
134.         TRANSFER_VERTEX_TO_FRAGMENT(o);
135.  
136.         return o;
137.     }
138.  
139.     fixed4 frag_base(v2f_base i) : SV_Target
140.     {
141.         //--------------------------- VECTORS -------------------------------
142.         float2 uv = i.uv.xy;
143.         float4 worldPosition = i.posWorld;
144.         float3 viewDirection = normalize(_WorldSpaceCameraPos.xyz - i.posWorld.xyz);
145.         float3 lightDirection = normalize(_WorldSpaceLightPos0.xyz);
146.         float3 halfDirection = normalize(viewDirection + lightDirection);
147.  
148.         //--------------------------- NORMAL MAPPING -------------------------------
149.         //sample our normal map
150.         float4 normalMap = tex2D(_BumpMap, uv);
151.         normalMap.xyz = UnpackNormal(normalMap);
152.  
153.         //calculate our mesh normals with the normal map into consideration
154.         float3 normalDirection;
155.         normalDirection.x = dot(i.tspace0, normalMap.xyz);
156.         normalDirection.y = dot(i.tspace1, normalMap.xyz);
157.         normalDirection.z = dot(i.tspace2, normalMap.xyz);
158.         normalDirection = normalize(normalDirection);
159.  
160.         //--------------------------- ADDITIONAL VECTORS ------------------------------- (some of these vectors need to take normals into account)
161.         float3 reflectionDirection = reflect(-viewDirection, normalDirection);
162.         float NdotL = dot(normalDirection, lightDirection);
163.         float NdotV = dot(normalDirection, viewDirection);
164.         float LdotH = dot(lightDirection, halfDirection);
165.         float HdotV = dot(halfDirection, viewDirection);
166.  
167.         //--------------------------- TEXTURES -------------------------------
168.         float4 albedo = tex2D(_MainTex, i.uv);
169.         float4 specular = float4(0, 0, 0, 0);
170.         float oneMinusReflectivity = 0.0f;
171.         float smoothness = _Smoothness;
172.  
173.         //alpha cutoff
174.         clip(albedo.a - _Cutoff);
175.  
176.         //--------------------------- LIGHTING -------------------------------
177.         float atten = LIGHT_ATTENUATION(i); // Light attenuation + shadows.
178.         float3 ambientColor = ShadeSH9(half4(normalDirection, 1));
179.  
180.         //_LightColor0
181.  
182.         float perceptualRoughness = SmoothnessToPerceptualRoughness(smoothness);
183.         float diffuseLighting = DisneyDiffuse(NdotV, NdotL, LdotH, perceptualRoughness);
184.         diffuseLighting = max(0.0f, diffuseLighting);
185.  
186.         float4 finalColor = float4(0, 0, 0, albedo.a);
187.  
188.         //--------------------------- COMBINE LIGHTING -------------------------------
189.         //finalColor.rgb += diffuseLighting * atten;
190.         finalColor.rgb += _LightColor0.rgb * atten;
191.         finalColor.rgb += ambientColor;
192.         finalColor.rgb *= albedo.rgb;
193.  
194.         // apply fog
195.         UNITY_APPLY_FOG(i.fogCoord, finalColor);
196.  
197.         return finalColor;
198.     }
199.     #endif
200.  
201.     struct v2f_shadow
202.     {
203.         V2F_SHADOW_CASTER;
204.         float2  uv : TEXCOORD1;
205.         UNITY_VERTEX_OUTPUT_STEREO
206.     };
207.  
208.     v2f_shadow vert_shadow(appdata_base v)
209.     {
210.         v2f_shadow o;
211.         UNITY_SETUP_INSTANCE_ID(v);
212.         UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
213.         TRANSFER_SHADOW_CASTER_NORMALOFFSET(o)
214.         o.uv = TRANSFORM_TEX(v.texcoord, _MainTex);
215.         return o;
216.     }
217.  
218.     float4 frag_shadow(v2f_shadow i) : SV_Target
219.     {
220.         fixed4 texcol = tex2D(_MainTex, i.uv);
221.         clip(texcol.a - _Cutoff);
222.  
223.         SHADOW_CASTER_FRAGMENT(i)
224.     }
225.  
226.     ENDCG
227.  
228.     SubShader
229.     {
230.         Tags
231.         {
232.             "Queue" = "AlphaTest"
233.             "RenderType" = "TransparentCutout"
234.             "IgnoreProjector" = "True"
235.             "DisableBatching" = "LODFading"
236.         }
237.  
238.         LOD 100
239.  
240.         //BASE PASS
241.         Pass
242.         {
243.             Tags {"LightMode" = "ForwardBase"}
244.  
245.             CGPROGRAM
246.             #pragma vertex vert_base
247.             #pragma fragment frag_base
248.  
249.             // make fog work
250.             #pragma multi_compile_fog
251.  
252.             #pragma multi_compile_fwdbase
253.             #pragma fragmentoption ARB_precision_hint_fastest
254.  
255.             #include "UnityCG.cginc"
256.             #include "UnityLightingCommon.cginc"
257.             #include "UnityStandardBRDF.cginc"
258.             #include "Lighting.cginc"
259.             #include "AutoLight.cginc"
260.  
261.             ENDCG
262.         }
263.  
264.         //FORWARD ADD PASS
265.         Pass
266.         {
267.             Tags {"LightMode" = "ForwardAdd"}
268.  
269.             Blend One One
270.  
271.             CGPROGRAM
272.             #pragma vertex vert_base
273.             #pragma fragment frag_base
274.  
275.             // make fog work
276.             #pragma multi_compile_fog
277.  
278.             #pragma multi_compile_fwdadd_fullshadows
279.             #pragma fragmentoption ARB_precision_hint_fastest
280.  
281.             #include "UnityCG.cginc"
282.             #include "UnityLightingCommon.cginc"
283.             #include "UnityStandardBRDF.cginc"
284.             #include "Lighting.cginc"
285.             #include "AutoLight.cginc"
286.  
287.             ENDCG
288.         }
289.  
290.         //SHADOW CASTER PASS
291.         Pass
292.         {
293.             Tags {"LightMode" = "ShadowCaster"}
294.  
295.             CGPROGRAM
296.             #pragma vertex vert_shadow
297.             #pragma fragment frag_shadow
298.  
299.             #pragma target 2.0
300.             #pragma multi_compile_shadowcaster
301.             #pragma multi_compile_instancing // allow instanced shadow pass for most of the shaders
302.  
303.             #include "UnityCG.cginc"
304.             ENDCG
305.         }
306.     }
307. }

(Also removes the

#pragma alphatest:_cutoff

lines, which don't do anything but produce warnings. Those only work as part of

#pragma surface

lines)

 

Who knows!

Unity's own shaders often flip flop between the supposed "Unity 5.0" ones, "Unity 4.0" ones, and just hard coding stuff in

#if

checks. It's a mess. Realistically most of them "work" still, or recursively end up calling the other anyway. Most of the complication comes from having to support shadow maps, light maps, shadow masks, and light cookies all at the same time. Often times you can simplify things by just sticking to the "SHADOW" related macros and ignoring the "LIGHT" ones.

 

Yeah... saying it's a mess I think is really an understatement. Just felt like Jumanji the whole way through when going through the countless .cginc files from the built-in shaders.