Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Sprite Renderer Mask on Specific Sorting Layer

Discussion in '2D' started by joaobsneto, Sep 23, 2015.

  1. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
    Hi guys,

    I'm making a 2D game that zombies come out of the ground. The visuals are similar to Plants Vs. Zombies. There are 4 lanes that spawn zombies. I'm having trouble to find a good solution for that. There are 3 sprites that I'm working it: the background, the tomb and the zombie. I could simply get piece of the ground image and put above the zombie, but that would not be a great solution, because I want to change the position of the tombs and change the background image. I would have to slice the background for each situation. The obvious solution was to mask the zombie part that is beneath the ground. I found some mask shader on the Internet, and for all that worked on Unity 5, they masked everything that was above the mask. I could not configure the shader to select the sprites that I wanted to mask or to specify any parameter that limited the layer that the mask would affect. Here are the images of the game to illustrate what I'm saying:

    This is two zombies getting out of the tomb. The Tomb on the center of the image is on the Sorting Layer "Tomb1", the zombie on the Sorting Layer "Zombie1", the tomb beneath in on the Sorting Layer "Tomb2", and the zombie "Zombie2".

    When I apply the mask at the Sorting Layer "Zombie1", it masks everything above, including the "Tomb2" and "Zombie2". My question is if it is possible to limit the mask to have effect only in the Sorting Layer that the mask is included. The shader that I'm using is this:
    Code (CSharp):
    1.  Shader "Custom/MaskTest" {
    2.      Properties
    3.      {
    4.          _MainTex ("Base (RGB) Alpha (A)", 2D) = "white" {}
    5.          _Cutoff ("Base Alpha cutoff", Range (0,.9)) = .5
    6.      }
    7.  
    8.      SubShader {
    9.          Tags {"Queue" = "Transparent+1"}
    10.           Offset 0, -1
    11.          ColorMask 0
    12.          ZWrite On
    13.          Pass
    14.          {
    15.              AlphaTest Greater [_Cutoff]    
    16.              SetTexture [_MainTex] {
    17.                  combine texture * primary, texture
    18.              }
    19.          }
    20.      }
    21. }
    I found that there is an Alpha Version of Unity with Mask Component to SpriteRenderer, but I don't have the Pro License, and this game will be release before december (the month that this feature will be released, according to Unity Road Map). I appreciate for reading this far.
     
  2. sandboxed

    sandboxed

    Unity Technologies

    Joined:
    Apr 6, 2015
    Posts:
    95
    We have SpriteMasking coming with Unity 5.4 (https://unity3d.com/unity/roadmap).. ie is March.

    Here is what you can do if you want to try things on your own.
    For Masks, create two versions of a shader.. one to mark the mask's presence, other to remove the mask after its done.
    1) Create a shader that increments the stencil buffer whenever the mask is rendered. This shader discards a pixel if the mask pixel has alpha above a pre-selected cutoff (something like 0.2).
    2) Create a shader (or variation of the first) that decrements the stencil buffer whenever the mask is rendered. Also discards pixel if alpha is above the pre-selected cutoff.

    For maskables (zombies), modify the Defaults-Shader available in Unity to use the stencil buffer and draw only if the stencil value > 0 (or whatever ref value you start with for the masks)
    Encapsulate zombies in the same layer with Masks (start with ON, end with OFF for the layer)
     
    joaobsneto likes this.
  3. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
    Thank you very much!! It worked!!! The keyword that I was missing was "Stencil". I didn't know the existence of this buffer and its use for 2D games. I found this forum thread: http://forum.unity3d.com/threads/is-it-possible-to-clip-a-sprite-with-another-sprite.254144/ and with a little googling and your tutorial I could make it work.
    I'll just check the shaders and share with you all.
     
  4. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
    Hi! Here are the shaders that solve this "problem". First, the shader for the SpriteRenderer of the objects that will masked.
    Code (CSharp):
    1. Shader "Sprites/Stencil Draw In Mask"
    2. {
    3.     Properties
    4.     {
    5.         [PerRendererData] _MainTex("Sprite Texture", 2D) = "white" {}
    6.         _Color("Tint", Color) = (1,1,1,1)
    7.         [MaterialToggle] PixelSnap("Pixel snap", Float) = 0
    8.     }
    9.  
    10.     SubShader
    11.     {
    12.         Tags
    13.         {
    14.             "Queue" = "Transparent+1"
    15.             "IgnoreProjector" = "True"
    16.             "RenderType" = "Transparent"
    17.             "PreviewType" = "Plane"
    18.             "CanUseSpriteAtlas" = "True"
    19.         }
    20.  
    21.         Cull Off
    22.         Lighting Off
    23.         ZWrite Off
    24.         Blend One OneMinusSrcAlpha
    25.  
    26.         Pass
    27.         {
    28.             Stencil
    29.             {
    30.                 Ref 0
    31.                 Comp Equal
    32.             }
    33.  
    34.         CGPROGRAM
    35.             #pragma vertex vert
    36.             #pragma fragment frag
    37.             #pragma multi_compile _ PIXELSNAP_ON
    38.             #include "UnityCG.cginc"
    39.  
    40.             struct appdata_t
    41.             {
    42.                 float4 vertex   : POSITION;
    43.                 float4 color    : COLOR;
    44.                 float2 texcoord : TEXCOORD0;
    45.             };
    46.  
    47.             struct v2f
    48.             {
    49.                 float4 vertex   : SV_POSITION;
    50.                 fixed4 color : COLOR;
    51.                 half2 texcoord  : TEXCOORD0;
    52.             };
    53.  
    54.             fixed4 _Color;
    55.  
    56.             v2f vert(appdata_t IN)
    57.             {
    58.                 v2f OUT;
    59.                 OUT.vertex = mul(UNITY_MATRIX_MVP, IN.vertex);
    60.                 OUT.texcoord = IN.texcoord;
    61.                 OUT.color = IN.color * _Color;
    62.                 #ifdef PIXELSNAP_ON
    63.                 OUT.vertex = UnityPixelSnap(OUT.vertex);
    64.                 #endif
    65.  
    66.                 return OUT;
    67.             }
    68.  
    69.             sampler2D _MainTex;
    70.             sampler2D _AlphaTex;
    71.             float _AlphaSplitEnabled;
    72.  
    73.             fixed4 SampleSpriteTexture(float2 uv)
    74.             {
    75.                 fixed4 color = tex2D(_MainTex, uv);
    76.                 if (_AlphaSplitEnabled)
    77.                     color.a = tex2D(_AlphaTex, uv).r;
    78.  
    79.                 return color;
    80.             }
    81.  
    82.             fixed4 frag(v2f IN) : SV_Target
    83.             {
    84.                 fixed4 c = SampleSpriteTexture(IN.texcoord) * IN.color;
    85.                 c.rgb *= c.a;
    86.                 return c;
    87.             }
    88.         ENDCG
    89.         }
    90.     }
    91. }
    And the shader for the mask:

    Code (CSharp):
    1. Shader "Sprites/Stencil Mask"
    2. {
    3.     Properties
    4.     {
    5.         [PerRendererData] _MainTex("Sprite Texture", 2D) = "white" {}
    6.         _Color("Tint", Color) = (1,1,1,1)
    7.         [MaterialToggle] PixelSnap("Pixel snap", Float) = 0
    8.         [MaterialToggle] ShowTexture("Show Texture", Float) = 0
    9.     }
    10.  
    11.         SubShader
    12.         {
    13.         Tags
    14.         {
    15.             "Queue" = "Transparent"
    16.             "IgnoreProjector" = "True"
    17.             "RenderType" = "Transparent"
    18.             "PreviewType" = "Plane"
    19.             "CanUseSpriteAtlas" = "True"
    20.         }
    21.  
    22.         Cull Off
    23.         Lighting Off
    24.         ZWrite Off
    25.         Blend One OneMinusSrcAlpha
    26.  
    27.         Pass
    28.         {
    29.             Stencil
    30.             {
    31.                 Ref 1
    32.                 Comp always
    33.                 Pass replace
    34.             }
    35.         CGPROGRAM
    36.             #pragma vertex vert
    37.             #pragma fragment frag
    38.             #pragma multi_compile _ PIXELSNAP_ON
    39.             #pragma multi_compile _ SHOWTEXTURE_ON
    40.             #include "UnityCG.cginc"
    41.  
    42.             struct appdata_t
    43.             {
    44.                 float4 vertex   : POSITION;
    45.                 float4 color    : COLOR;
    46.                 float2 texcoord : TEXCOORD0;
    47.             };
    48.  
    49.             struct v2f
    50.             {
    51.                 float4 vertex   : SV_POSITION;
    52.                 fixed4 color : COLOR;
    53.                 half2 texcoord  : TEXCOORD0;
    54.             };
    55.  
    56.             fixed4 _Color;
    57.  
    58.             v2f vert(appdata_t IN)
    59.             {
    60.                 v2f OUT;
    61.                 OUT.vertex = mul(UNITY_MATRIX_MVP, IN.vertex);
    62.                 OUT.texcoord = IN.texcoord;
    63.                 #ifdef SHOWTEXTURE_ON
    64.                 OUT.color = IN.color * _Color;
    65.                 #else
    66.                 OUT.color = IN.color * _Color;
    67.                 OUT.color.rgb = 0;
    68.                 #endif
    69.                 #ifdef PIXELSNAP_ON
    70.                 OUT.vertex = UnityPixelSnap(OUT.vertex);
    71.                 #endif
    72.  
    73.                 return OUT;
    74.             }
    75.  
    76.             sampler2D _MainTex;
    77.  
    78.             fixed4 frag(v2f IN) : SV_Target
    79.             {
    80.                 fixed4 c = tex2D(_MainTex, IN.texcoord) * IN.color;
    81.                 if (c.a < 0.2) discard;
    82.                 c.rgb *= c.a;
    83.                 c.a = 0;
    84.                 return c;
    85.             }
    86.         ENDCG
    87.         }
    88.     }
    89. }
    I created a copy of this mask shader changing the shader name to "Sprites/Stencil Mask Restore" and Stencil Ref value to 0. Then, I created the materials for the shaders mentioned. Finally, set up the scene as @sandboxed explained. First the mask sprites, then the sprites to be masked and finally the restore mask sprite. I put the last one as a child of the mask sprite, to easily fit one over the other, and set the appropriate material.
    Here is the result, thanks to @sandboxed
     
    e140games, dizez5, sandboxed and 2 others like this.
  5. sandboxed

    sandboxed

    Unity Technologies

    Joined:
    Apr 6, 2015
    Posts:
    95
    Cool!!!
    With SpriteMasking coming to Unity in 5.4, you will not need to bother about setting up the order of mask and maskables. Also masks will be able to affect multiple layers.
     
  6. leandro_nw

    leandro_nw

    Joined:
    Jul 17, 2012
    Posts:
    8
    Just wanted to thank you for posting the shaders code, it works great and saved me a lot of trouble :)
     
  7. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
  8. Oruji

    Oruji

    Joined:
    Nov 5, 2012
    Posts:
    14
    Thanks for sharing the shader.

    Could anyone point me in the direction of getting this to work with alpha blending?
     
  9. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
    I think you just have to copy and paste the shader that you want. Just add this part to the shader and follow those steps.
    1. Stencil
    2. {
    3. Ref 0
    4. Comp Equal
    5. }
     
  10. BadSeedProductions

    BadSeedProductions

    Joined:
    Dec 26, 2014
    Posts:
    137
    Nope. Hopefully 5.5. Masking multiple layers is a must, please Unity let's not wait until 6.
     
    RemDust likes this.
  11. e140games

    e140games

    Joined:
    Aug 2, 2012
    Posts:
    3
    joaobsneto - Thanks, Thanks, Thanks !
     
  12. WesleyNR

    WesleyNR

    Joined:
    Mar 7, 2015
    Posts:
    3
    Thank you for posting the shaders, joaobsneto, and for everyone who helped! :)
     
  13. dfpdev

    dfpdev

    Joined:
    Jul 12, 2016
    Posts:
    1
    Hi, I used this shader and it is working but it doesn't have soft edges. Is it possible to use the alpha channel to get soft edged blending rather than the all or nothing threshold?
     
  14. joaobsneto

    joaobsneto

    Joined:
    Dec 10, 2009
    Posts:
    152
    I don't think so... but I'm not a shader expert. I think that stencil doesn't support alpha blending. Am I right @sandboxed ?
     
  15. Zinov

    Zinov

    Joined:
    Jul 20, 2015
    Posts:
    38
    5.4 is out and it seems like there is no masking for SpriteRenderer still.
     
  16. zero_null

    zero_null

    Joined:
    Mar 11, 2014
    Posts:
    159
    yes no masking still.
     
  17. Kurius

    Kurius

    Joined:
    Sep 29, 2013
    Posts:
    412
    In the Unity Roadmap it says...
    @sandboxed what are "Workflow concerns"?
    When do you think Sprite Masking will be available?
    Thank you
     
  18. LazyKnight

    LazyKnight

    Joined:
    Jul 28, 2017
    Posts:
    7
    Another solution I'm using, with unity 2017.1's SpriteMask feature:

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. namespace DCGame.UnityFeaturePatch
    4. {
    5.   [RequireComponent(typeof(SpriteMask))]
    6.   public class SpriteMaskRangeFixer : MonoBehaviour
    7.   {
    8.     private static int index = 0;
    9.     public SpriteRenderer[] targets;
    10.     public SpriteMask mask;
    11.  
    12.     private void Start()
    13.     {
    14.       mask = GetComponent<SpriteMask>();
    15.  
    16.       foreach (var target in targets)
    17.       {
    18.         target.sortingOrder = index + 2;
    19.         target.sortingLayerName = mask.sortingLayerName;
    20.       }
    21.       mask.isCustomRangeActive = true;
    22.       mask.backSortingOrder = index + 1;
    23.       mask.frontSortingOrder = index + 2;
    24.  
    25.       index += 2;
    26.  
    27.       if (index >= 1000)
    28.       {
    29.         index = 0;
    30.       }
    31.     }
    32.   }
    33. }
    34.  
     
unityunity