Search Unity

Stencil mask of stencil mask

Discussion in 'Shaders' started by Sir_Rosenfeld, May 6, 2020.

  1. Sir_Rosenfeld

    Sir_Rosenfeld

    Joined:
    Jan 9, 2019
    Posts:
    3
    Hi everyone,

    I am making a top-down 2D game and I'd like to be able to display elements under those conditions. I am using stencil masks to block elements.
    I'd like to be able to cutout a stencil mask before applying it.

    Another way to put it is :
    Mesh A and Mesh B are stencil masks and a Mesh C.
    Mesh A displays whatever is under it (usual stencil mask)
    Mesh B displays what is under it ONLY where Mesh C and Mesh B intersects. (and it doesn't interfere with Mesh A, Mesh A still displays no matter what)

    I was thinking of cutting out Mesh B through some intermediary stencil shader but I don't know how to do it.


    My code is the default Stencil Mask/Object shader

    Mask :
    Code (CSharp):
    1.         Tags { "RenderType"="Opaque" "Queue"="Geometry-100" }
    2.         ColorMask 0
    3.         ZWrite off
    4.         LOD 200
    5.  
    6.         Stencil {
    7.             Ref 1
    8.             Pass replace
    9.         }
    Object :
    Code (CSharp):
    1.         Tags { "RenderType"="Opaque" }
    2.         LOD 200
    3.  
    4.         Stencil {
    5.             Ref 1
    6.             Comp equal
    7. }

    Would really appreciate some help :)

    Thanks !
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,352
    Stencils take some mind bending to realize their full utility.

    The stencil buffer is a byte per pixel buffer, and you can use it either as the straight byte value (0-255) or as a bit wise operation where you’re setting and checking the bits. The reason why this is useful is you can have one stencil use
    Ref 1
    , and a second use
    Ref 2
    with both using
    Pass Replace
    . With no other stencil settings whichever is the last of those two to render is what the stencil buffer has. But if you use
    WriteMask 1
    (00000001) and
    WriteMask 2
    (00000010) respectively each shader will only write to the first and second bit of the byte. This means any anywhere both of them render both meshes render the stencil will hold the value 3 (00000011). Then you can have your object check
    Ref 3
    an
    Comp Equal
    and it’ll only show in those area where both overlap.
     
    _eternal and tomekkie2 like this.