Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Trying to get correct shadows on custom standard shader with clip(), addshadow doesnt work correctly

Discussion in 'Shaders' started by Steven-1, Jun 15, 2020.

  1. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    455
    I made a simple shader that is just the standard shader with added functionality to fade out by clipping using using a dotted pattern.

    See here:
    Capture4.PNG Capture3.PNG

    It works correctly, but the shadows are incorrect.
    I remembered I had to add "addshadow" to the "#pragma surface"-line, but that doesn't make it correct either.
    You can see the shadows of objects behind the object with this shader.

    see this:
    Capture.PNG

    while it should look like this:
    Capture2.PNG

    Anyone know what to do to fix this?


    full shader:
    Code (CSharp):
    1. Shader "Custom/Hideable"
    2. {
    3.     Properties
    4.     {
    5.         _Color ("Color", Color) = (1,1,1,1)
    6.         _MainTex ("Albedo (RGB)", 2D) = "white" {}
    7.         _Glossiness ("Smoothness", Range(0,1)) = 0.5
    8.         _Metallic ("Metallic", Range(0,1)) = 0.0
    9.         _VisibilityF("Visibility", Range(0,1)) = 1.0
    10.         _Mask("Visibility Mask", 2D) = "gray" {}
    11.     }
    12.     SubShader
    13.     {
    14.         Tags { "RenderType"="Opaque" }
    15.         LOD 200
    16.  
    17.         CGPROGRAM
    18.         // Physically based Standard lighting model, and enable shadows on all light types
    19.         #pragma surface surf Standard fullforwardshadows addshadow
    20.  
    21.         // Use shader model 3.0 target, to get nicer looking lighting
    22.         #pragma target 3.0
    23.  
    24.         sampler2D _MainTex;
    25.         sampler2D _Mask;
    26.  
    27.         struct Input
    28.         {
    29.             float2 uv_MainTex;
    30.             float4 screenPos;
    31.         };
    32.  
    33.         half _Glossiness;
    34.         half _Metallic;
    35.         fixed4 _Color;
    36.         half _VisibilityF;
    37.  
    38.         // Add instancing support for this shader. You need to check 'Enable Instancing' on materials that use the shader.
    39.         // See https://docs.unity3d.com/Manual/GPUInstancing.html for more information about instancing.
    40.         // #pragma instancing_options assumeuniformscaling
    41.         UNITY_INSTANCING_BUFFER_START(Props)
    42.             // put more per-instance properties here
    43.         UNITY_INSTANCING_BUFFER_END(Props)
    44.  
    45.         void surf (Input IN, inout SurfaceOutputStandard o)
    46.         {
    47.             // Albedo comes from a texture tinted by color
    48.             fixed4 c = tex2D (_MainTex, IN.uv_MainTex) * _Color;
    49.             o.Albedo = c.rgb;
    50.             // Metallic and smoothness come from slider variables
    51.             o.Metallic = _Metallic;
    52.             o.Smoothness = _Glossiness;
    53.             o.Alpha =  c.a;
    54.  
    55.             IN.screenPos.x *= 16.0 / 9.0;
    56.             float2 screenUV = 80 * IN.screenPos.xy / IN.screenPos.w;
    57.             clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    58.         }
    59.         ENDCG
    60.     }
    61.     FallBack "Standard"
    62. }
    63.  
     
    Last edited: Jun 15, 2020
  2. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    455
    Anyone?
     
  3. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    455
    No one?
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    There are probably two things happening here.

    First I'm going to point out in the second to last image, what you're seeing "through" the opaque wall aren't shadows, that's screen space ambient occlusion. It's true ambient occlusion is an approximation of indirect shadows, but for rendering the distinction is important because shadows and ambient occlusion are done very differently. We'll come back to that point in a bit.


    Unity’s Surface Shaders have a long time bug that the
    screenPos
    Input
    isn’t properly set for the shadowcaster pass. You need to calculate the screen position yourself using a vertex function and a custom
    Input
    property. Otherwise in the shadow caster pass, which is used both for casting shadows and for calculating the camera depth texture, will be acting as if the
    IN.screenPos
    is all zeros. This is bad for a number of reasons, not the least of which is that the
    IN.screenPos.xy / IN.screenPos.w
    line is now dividing by zero. You may even have an error / warning on your shader if you select it saying there is a divide by zero.

    Code (csharp):
    1. #pragma surface surf /* stuff */ vertex:vert
    2.  
    3. struct Input {
    4.   /* stuff */
    5.   // can't be named screenPos because Unity will override it
    6.   float4 customScreenPos;
    7. };
    8.  
    9. // custom vertex function to calculate the screen pos
    10. void vert (inout appdata_full v, out Input o)
    11. {
    12.   o.customScreenPos = ComputeScreenPos(UnityObjectToClipPos(v.vertex));
    13. }
    14.  
    15. // in the surf function
    16. float2 screenUV = IN.customScreenPos.xy / IN.customScreenPos.w;
    17. // proper aspect ratio correction rather than the hard coded 16:9 aspect you had
    18. screenUV.x *= _ScreenParams.x / _ScreenParams.y;
    19. screenUV *= 80.0;
    However, there's a problem with this. The screen position is calculated from the current projection being used for rendering, and for shadowmaps that's the projection from that light, not from the camera, so the UVs won't match between the shadow and the camera. There is a way to fix that, but there's also the question of if you even want the shadows being cast to be affected by the visibility clipping. I suspect you don't. To fix that you'll want to do the
    clip()
    only in the case of the shadow caster pass being used during the camera depth texture rendering and not during shadow map rendering. There's no really clean way to do this in Unity's built in rendering paths. The closest would be to do something like this:
    Code (csharp):
    1. // in the surf function
    2. #ifdef SHADOWS_DEPTH
    3. if (!any(unity_LightShadowBias)) {
    4.   // most likely the camera depth texture
    5.   clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    6. }
    7. #endif
    If you're using and spot lights with zero bias, this code will also run, but you shouldn't be using spot lights with no bias...


    And after all that ... it might still be broken. This is getting back to how I mentioned ambient occlusion works very differently from shadows. Some AO works purely from the camera depth texture, in which case the above should mostly fix the problem. The problem is if you're using an ambient occlusion post processing that uses the camera depth normals texture, which apart from looking like it should be just like the camera depth texture, is actually rendered in a completely different way that doesn't use the shadow caster pass. If doing the above doesn't fix the problem then it means you'll need to use a custom
    RenderType
    tag, and modify the Internal-DepthNormalsTexture.shader to add a pass that uses that tag that does the custom visibility clipping you're doing. That shader can be found in the built in shaders you can download, or here:
    https://github.com/TwoTailsGames/Un...rcesExtra/Internal-DepthNormalsTexture.shader

    I would recommend you use the Frame Debugger (under Window > Analysis > Frame Debugger) to step through how things are being rendered. It might give you a better understanding of where the issue is coming from.



    There's also one other easy fix. Switch to using deferred rendering and almost all of these issues go away. You don't even need the
    addshadow
    on the
    #pragma surface
    line since you most likely want the shadows to be opaque anyway.
     
    Steven-1 likes this.
  5. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    455
    Thanks a lot for the extensive reply.

    Oh I know, I was referring to the fact that the left side is entirely dark except for the hole where the tunnel ends. sorry yeah, I guess the image isn't super clear.

    You're right that I don't want the clipping to affect the shadow being cast.

    I tried it and it works!
    But like you expected, the ambient occlusion acts a bit weird as a result. It renders AO-shadow in the shape of the pattern (but at a different scale) on top of the mesh.

    I'm gonna experiment with it a bit and decide what I'm gonna do with it.

    Thanks for the help!

    Edit: you know what, the ao-shadow isn't very noticeable, so it's fine like this.
    I might switch to deffered at some point though, but I'm just not familiar with the impact of it.
    (the 4-lights limit in forward is a bit annoying, but other than that, it's fine, so I dunno if it's worth it)
     
    Last edited: Jun 21, 2020
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Yep, the AO issue is one you'll have to live with. Any screen space ambient occlusion post process is going to be dependent on what you're rendering to the screen. In this case you're rendering a wall that's filled with holes, so that's what the SSAO is going to be seeing too. You can do some hacks where the AO sees a solid wall, but then nothing inside gets AO. Or you can hack it so the exterior wall isn't seen at all and ... then you get what you already had. The only solution to this is to not use SSAO. You have to use a true volumetric approach to AO, either using a voxelized scene (like UE4) or mathematical SDF shape based manual AO like many games from the Xbox 360 / PS3 era, like Saints Row 3, Remember Me, or in a more complex way Last of Us. Or you just live with it because most people won't notice anything wrong.
     
  7. osmotic_simon

    osmotic_simon

    Joined:
    Jun 3, 2020
    Posts:
    3
    Sorry to necro this thread, but is it possible to make this work with point light shadows?
    Shadows being cast by point lights get the clipping effect too, unlike shadows from directional lights which work fine.

    Thanks in advance!
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    The above bit of example code with
    #ifdef SHADOWS_DEPTH
    should already fix the problem for point shadows too.
     
  9. osmotic_simon

    osmotic_simon

    Joined:
    Jun 3, 2020
    Posts:
    3
    Is this how the example code should be implemented into the shader?

    Code (CSharp):
    1. Shader "Custom/Hideable"
    2. {
    3.     Properties
    4.     {
    5.         _Color("Color", Color) = (1,1,1,1)
    6.         _MainTex("Albedo (RGB)", 2D) = "white" {}
    7.         _Glossiness("Smoothness", Range(0,1)) = 0.5
    8.         _Metallic("Metallic", Range(0,1)) = 0.0
    9.         _VisibilityF("Visibility", Range(0,1)) = 1.0
    10.         _Mask("Visibility Mask", 2D) = "gray" {}
    11.     }
    12.         SubShader
    13.     {
    14.         Tags { "RenderType" = "Opaque" }
    15.         LOD 200
    16.  
    17.         CGPROGRAM
    18.         // Physically based Standard lighting model, and enable shadows on all light types
    19.         #pragma surface surf Standard fullforwardshadows addshadow vertex:vert
    20.  
    21.         // Use shader model 3.0 target, to get nicer looking lighting
    22.         #pragma target 3.0
    23.  
    24.         sampler2D _MainTex;
    25.         sampler2D _Mask;
    26.  
    27.         struct Input
    28.         {
    29.             float2 uv_MainTex;
    30.             float4 customScreenPos;
    31.         };
    32.  
    33.         // custom vertex function to calculate the screen pos
    34.         void vert(inout appdata_full v, out Input o)
    35.         {
    36.             UNITY_INITIALIZE_OUTPUT(Input, o);
    37.             o.customScreenPos = ComputeScreenPos(UnityObjectToClipPos(v.vertex));
    38.         }
    39.  
    40.         half _Glossiness;
    41.         half _Metallic;
    42.         fixed4 _Color;
    43.         half _VisibilityF;
    44.  
    45.         // Add instancing support for this shader. You need to check 'Enable Instancing' on materials that use the shader.
    46.         // See https://docs.unity3d.com/Manual/GPUInstancing.html for more information about instancing.
    47.         // #pragma instancing_options assumeuniformscaling
    48.         UNITY_INSTANCING_BUFFER_START(Props)
    49.             // put more per-instance properties here
    50.         UNITY_INSTANCING_BUFFER_END(Props)
    51.  
    52.         void surf(Input IN, inout SurfaceOutputStandard o)
    53.         {
    54.             // Albedo comes from a texture tinted by color
    55.             fixed4 c = tex2D(_MainTex, IN.uv_MainTex) * _Color;
    56.             o.Albedo = c.rgb;
    57.             // Metallic and smoothness come from slider variables
    58.             o.Metallic = _Metallic;
    59.             o.Smoothness = _Glossiness;
    60.             o.Alpha = c.a;
    61.  
    62.             float2 screenUV = IN.customScreenPos.xy / IN.customScreenPos.w;
    63.             // proper aspect ratio correction rather than the hard coded 16:9 aspect you had
    64.             screenUV.x *= _ScreenParams.x / _ScreenParams.y;
    65.             screenUV *= 80.0;
    66.  
    67.             #ifdef SHADOWS_DEPTH
    68.                 if (!any(unity_LightShadowBias)) {
    69.                     // most likely the camera depth texture
    70.                     clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    71.                 }
    72.             #endif  
    73.         }
    74.         ENDCG
    75.     }
    76.     FallBack "Standard"
    77. }
    Because the above doesn't work for me, nothing gets clipped at all.
    However, if I additionally put the following code to the end of the surface shader, directional shadows work fine but point lights don't.

    Code (CSharp):
    1. #ifndef SHADOWS_DEPTH
    2.         clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    3. #endif
    ...which looks like this:
    upload_2021-6-17_23-37-20.png

    But maybe this is not how you meant to implement the code?
    I am on Unity 2021.1.9f1 btw!
     
  10. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Ah, yeah, you also need:
    Code (csharp):
    1.             #if defined(SHADOWS_DEPTH) && defined(UNITY_PASS_SHADOWCASTER)
    2.             if (!any(unity_LightShadowBias)) {
    3.                 // most likely the camera depth texture
    4.                 clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    5.             }
    6.             #endif
    7.             #if !defined(UNITY_PASS_SHADOWCASTER)
    8.             clip(_VisibilityF - tex2D(_Mask, screenUV).r);
    9.             #endif
     
    osmotic_simon likes this.
  11. osmotic_simon

    osmotic_simon

    Joined:
    Jun 3, 2020
    Posts:
    3
    Thank you so much bgolus! You are my hero :)