Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question How to cast (maybe recieve)sprite shadows without discarting alpha pixels with cutout shader?

Discussion in 'Shaders' started by InkaTorque, Nov 20, 2020.

  1. InkaTorque

    InkaTorque

    Joined:
    Sep 4, 2014
    Posts:
    22
    Hi , everybody !

    I´m working on a 2.5D game that has movement on all axes and uses hand drawn sprites ( not pixel art , this is very important).

    I managed to make our sprites cast an recieve shadows by making our shader render in the Geometry queue and using an alpha cutoff parameter so that the shadows show the correct sillouete of the sprite. This worked and generated a really cool level of polish in our project but it has one big disadvantage : The alpha pixels that are below the cutout threshold are not rendered at all. Since our art is hand drawn and has alpha gradients , this is not aceptable because we lose quality , sprites look jagged and pixelated on the edges due to the pixels getting discarded. Here is an image explainng the problem

    AlphaDiscard (2).png

    I would really like to know if there is any way to cast shadows without discarding those alpha pixels. In my complete ingnorance of shaders I was thinking of a shader with multiple passes or using a second sprite renderer in each object to with the cutout shader but not rendering anything at all so that it only casts shadows but that seems really overkill. The ability to cast shadows is really cool and I don't want to lose it.

    The project uses forward renderer and uses de default/basic render from Unity ( not using URP or HDRP)

    Also : Is there any way my sprites could also recieve shadows without having to use the geometry queue? Recieving shadows would be the extra quality touch the game would need to look really good.

    Here is the shader I´m currently using
    Code (CSharp):
    1. Shader "Custom/CharactersV4Players"{
    2.     Properties
    3.     {
    4.         [PerRendererData] _MainTex("Texture", 2D) = "white" {}      
    5.         _EffectColor1("Effect Color", Color) = (1,1,1,1)
    6.         _Crossfade("Fade", float) = 0
    7.         _FlashColor("Flash Color", Color) = (1,1,1,1)
    8.         _FlashAmount("Flash Amount",Range(0.0,1.0)) = 0
    9.         _Cutoff("Alpha Cutoff", Range(0,1)) = 0.9
    10.      
    11.     }
    12.  
    13.     SubShader
    14.     {
    15.         Tags
    16.         {
    17.             "Queue" = "Transparent"
    18.             "IgnoreProjector" = "True"
    19.             "RenderType" = "TransparentCutOut"
    20.             "PreviewType" = "Plane"
    21.             "CanUseSpriteAtlas" = "True"
    22.         }
    23.  
    24.      
    25.         Cull Off
    26.         Lighting Off
    27.         ZWrite Off
    28.         Blend SrcAlpha OneMinusSrcAlpha
    29.  
    30.  
    31.         CGPROGRAM
    32.             #pragma surface surf Lambert alpha:blend fullforwardshadows addshadow alphatest:_Cutoff  
    33.  
    34.             #pragma target 3.0
    35.      
    36.             struct Input {
    37.                 fixed2 uv_MainTex;
    38.                 fixed4 color : COLOR;
    39.             };
    40.  
    41.             sampler2D _MainTex;
    42.             fixed4 _EffectColor1;
    43.             fixed _Crossfade;
    44.             fixed4 _FlashColor;
    45.             float _FlashAmount;
    46.  
    47.             void surf(Input IN, inout SurfaceOutput o)
    48.             {
    49.                 fixed4 col = tex2D(_MainTex, IN.uv_MainTex);
    50.                 fixed4 returnColor = lerp(col, col * _EffectColor1, _Crossfade) * _EffectColor1.a + col * (1.0 - _EffectColor1.a);
    51.  
    52.                 o.Albedo = returnColor.rgb * IN.color.rgb;
    53.                 o.Alpha = col.a * IN.color.a;
    54.  
    55.                 o.Albedo = lerp(o.Albedo,_FlashColor.rgb,_FlashAmount);
    56.              
    57.             }
    58.         ENDCG
    59.     }
    60.     Fallback "Legacy Shaders/Transparent/Cutout/VertexLit"
    61. }
    62.  
     
  2. InkaTorque

    InkaTorque

    Joined:
    Sep 4, 2014
    Posts:
    22
    Any ideas , guys?
     
  3. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    When using Unity’s built in renderer, transparent queue materials cannot receive shadows. There are work arounds that allow you to extract the main directional light’s shadows with a script and access them during the forward base pass with a custom shader, but there is no solution for additional shadow casting lights.

    For casting transparent shadows, the closest you can get is dithered shadows like Unity’s Standard shader does. Unity’s shadow maps cannot support true transparent shadows. Really, neither can almost any real time shadowing technique used by any game engine. There are some techniques out there for doing them, but to use them in Unity would require replacing Unity’s entire lighting and shadowing system, from scratch. You couldn’t use any of the built in systems. At that point you’re better of switching to one of the SRPs, not because either of them support transparent shadows either, but because it would be easier to modify. Not easy mind you, just easier. But, if you’re asking the questions you are here, that’s going to be a task way beyond your current skill or knowledge level.

    So the “easy” hack is you can change your shader to use the Standard shader as the fallback shader to use its shadow caster pass, though by default it will still be the opaque version of the shadow. You also need to remove the
    addshadow
    from your shader as that’s generating a shadow caster pass already. To enable the dithered transparent shadow, the shader needs to have the
    _ALPHABLEND_ON
    keyword enabled. The easiest way to do that is to add a property to your shader that looks like this:
    Code (CSharp):
    1. [Toggle(_ALPHABLEND_ON)] _ALPHABLEND_ON (“Enable Dithered Shadows”, Float) = 0.0
    Then make sure you enable it for every material. That or you’ll need to create a custom shadow caster pass in your shader manually.
     
    Last edited: Nov 27, 2020
  4. InkaTorque

    InkaTorque

    Joined:
    Sep 4, 2014
    Posts:
    22
    Hi , Sorry for answering so late but I had given up on this topic. Thank you so much for your answer , I really appreciate it.

    So if I understand you correctly the shader would end up looking like this?

    Code (CSharp):
    1. Shader "Custom/CharactersV4Players"{
    2.     Properties
    3.     {
    4.         [PerRendererData] _MainTex("Texture", 2D) = "white" {}  
    5.         _EffectColor1("Effect Color", Color) = (1,1,1,1)
    6.         _Crossfade("Fade", float) = 0
    7.         _FlashColor("Flash Color", Color) = (1,1,1,1)
    8.         _FlashAmount("Flash Amount",Range(0.0,1.0)) = 0
    9.         _Cutoff("Alpha Cutoff", Range(0,1)) = 0.9
    10.     [Toggle(_ALPHABLEND_ON)] ALPHABLEND_ON("Enable Dithered Shadows", Float) = 0.0
    11.  
    12.  
    13.     }
    14.     SubShader
    15.     {
    16.         Tags
    17.         {
    18.             "Queue" = "Transparent"
    19.             "IgnoreProjector" = "True"
    20.             "RenderType" = "TransparentCutOut"
    21.             "PreviewType" = "Plane"
    22.             "CanUseSpriteAtlas" = "True"
    23.         }
    24.  
    25.         Cull Off
    26.         Lighting Off
    27.         ZWrite Off
    28.         Blend SrcAlpha OneMinusSrcAlpha
    29.         CGPROGRAM
    30.             #pragma surface surf Lambert alpha:blend fullforwardshadows alphatest:_Cutoff
    31.             #pragma target 3.0
    32.  
    33.             struct Input {
    34.                 fixed2 uv_MainTex;
    35.                 fixed4 color : COLOR;
    36.             };
    37.             sampler2D _MainTex;
    38.             fixed4 _EffectColor1;
    39.             fixed _Crossfade;
    40.             fixed4 _FlashColor;
    41.             float _FlashAmount;
    42.             void surf(Input IN, inout SurfaceOutput o)
    43.             {
    44.                 fixed4 col = tex2D(_MainTex, IN.uv_MainTex);
    45.                 fixed4 returnColor = lerp(col, col * _EffectColor1, _Crossfade) * _EffectColor1.a + col * (1.0 - _EffectColor1.a);
    46.                 o.Albedo = returnColor.rgb * IN.color.rgb;
    47.                 o.Alpha = col.a * IN.color.a;
    48.                 o.Albedo = lerp(o.Albedo,_FlashColor.rgb,_FlashAmount);
    49.          
    50.             }
    51.         ENDCG
    52.     }
    53.     Fallback "Standard"
    54. }
    If so , I have some more questions:
    1. Are the _Cutoff keyword in the properties section and the alphatest:_Cuofff on the pragma section still necessary?
    2. Should I change my renderType to just "Transparent"?
    3. The shadow shows up if _ALPHABLEND_ON is disabled and goes away if it is enabled ( tested by positioning a plane behind the sprites) . Is this the expected behaviour?
    Also , the produced shadow is not that accurate compared to the shape of the sprite and changing the value of the _Cutoff property discards pixels frrom the sprite but the shadows remains the same shape. Is there a way to control the cutoff value just for the shadow and not for the main sprite? ( This is the effect I wanted to achieve , I don´t know if my question was very clear about that).

    Here is an example of how the shadow looks now.
    Again , thank you for taking the time to answer this question.
     
  5. InkaTorque

    InkaTorque

    Joined:
    Sep 4, 2014
    Posts:
    22
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    No. Remove them from your shader.

    You can. But it doesn't really matter unless you're using post processing that makes use of the depth normal texture, which for a mainly 2D game is highly unlikely. Basically only used for some SSAO and post process outlines.

    The shader needs a
    _Color
    property. It doesn't have to use it, or even have it exposed, but it needs to exist in the properties list and have an alpha of 1.0 otherwise the shader will get the default value, which is
    float4(0.0, 0.0, 0.0, 0.0)
    . While your shader doesn't use the color property, the Standard shadow pass does.

    However there's an easier option I realized later. Instead of using the Standard shader as the Fallback, use the "Particles/Standard Surface" shader instead. This has the benefit of not needing the "hidden"
    _Color
    , and properly supports use of the vertex alpha, which Sprite components use to pass the color to the shader, and which the "regular" Standard shader does not do at all.
    Code (csharp):
    1. Fallback "Particles/Standard Surface"
    Then you'll be able to get a partially transparent shadow, like this:
    upload_2020-12-7_13-55-5.png
    It's not perfect, as it only has 16 levels of opacity, and requires you use soft shadows else you'll get even more obvious artifacts, like this:
    upload_2020-12-7_13-55-42.png
    But that's as good as it gets.
     
  7. InkaTorque

    InkaTorque

    Joined:
    Sep 4, 2014
    Posts:
    22
    @bgolus thank you so much !!!! It worked !!!!!!!! and shadows look awesome . I really appreciate your support.

    I have one more question , though : Where can I learn ( books , videos , etc ) about the work arounds that allow to obtain the main directional light's shadows and use them? The only source for shadow casting in our game is the directional light so having that feature is completely feasable . I would really like to learn and if you could point me in the right direction it would be awesome.

    Thank you once again for your help.
     
  8. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329