Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice

Problems with silhouette overlay shader

Discussion in 'Shaders' started by Waliactico, Oct 17, 2015.

  1. Waliactico

    Waliactico

    Joined:
    Jan 16, 2013
    Posts:
    91
    Hello, everyone. I'm trying to make a shader to use when characters are behind walls, to draw a transparent silhouette (blending a solid color with the wall).
    The problem that I have is that my characters are modelled with more than 1 mesh, so even if I make the shader to avoid rendering, and thus blending, hidden parts, when it renders another mesh from the same character, the blending inevitably occurs, as you can see in the picture attached. The hierarchy of my model also looks like in the other picture.

    hierarchy.png sil1.png

    Now, to the core of the problem. I thought I could write the stencil buffer, and then the other meshes of the same character would ignore the parts already drawn, but that doesn't work if there are other characters overlapping that character, they would not render the silhouette, resulting in fake visual depth order, like this:

    sil3.png

    This is the shader to get this result:

    Code (CSharp):
    1. SubShader {
    2.             Tags { "Queue" = "Transparent+100" "IgnoreProjector"="True" "RenderType"="Transparent" }
    3.             LOD 100
    4.             ColorMask RGB
    5.             Lighting Off
    6.      
    7.             Pass {
    8.                //the first time the silhouette is rendered, writes 10 to the stencil. Other passes won't render, because stencil doesn't pass (Comp NotEqual)
    9.                 Stencil {
    10.                     Ref 10
    11.                     Comp NotEqual
    12.                     Pass Replace
    13.                 }
    14.          
    15.                 ZTest Greater  //to draw only behind walls
    16.                 ZWrite Off
    17.                 Blend SrcAlpha OneMinusSrcAlpha
    18.              
    19.                 CGPROGRAM
    20.                 #pragma vertex vert
    21.                 #pragma fragment frag
    22.                 #include "UnityCG.cginc"
    23.              
    24.                 uniform float4 _ColorOverlay;
    25.                
    26.                 struct vertexOutput { float4 pos: POSITION; };
    27.              
    28.                 vertexOutput vert ( appdata_base v ) {  
    29.                     vertexOutput output;
    30.                     output.pos = mul(UNITY_MATRIX_MVP, v.vertex);
    31.                     return output;
    32.                 }
    33.                 float4 frag ( vertexOutput input ) : COLOR {
    34.                     float4 lColor     = _ColorOverlay;                  
    35.                     lColor.a = 0.5;                  
    36.                     return lColor;
    37.                 }
    38.                 ENDCG
    39.             }
    40. }
    Then what I tried was to "reset" the stencil buffer after rendering the mesh, with an extra pass like this:
    Code (CSharp):
    1. Pass {
    2.                 ZTest Always
    3.                 Colormask 0
    4.                 ZWrite Off
    5.              
    6.                 Stencil {
    7.                     Ref 10
    8.                     Comp Equal
    9.                     Pass Zero
    10.                 }
    11.             }
    ...but then, what I got is the first result I told. That would be with the 2 characters:

    sil2.png

    So... what can I do to get the characters with all its meshes in a flat color, and in the correct depth order?

    Any help would be appreciated, shader gurus! Thanks for your time!
     
    Last edited: Oct 18, 2015
  2. Waliactico

    Waliactico

    Joined:
    Jan 16, 2013
    Posts:
    91
  3. Wompipomp

    Wompipomp

    Joined:
    Aug 31, 2014
    Posts:
    24
    Hi,

    I don't know if this is possible because of how the render pipeline works.
    The first object drawn is the wall as it is opaque. Afterwards the transparent objects are drawn in back to front order so the first object drawn is the furthest away which operates first on the stencil buffer. So the pixel of the green object is already blended with the wall before the red object is drawn. Deleting the green pixel also results in deleting the wall pixel.

    The best thing I think is to use opaque silhouettes.
    The other possibility would be to change the render order of the transparent objects so that the stencil buffer can be used.
    e.g. use one shader with Transparent+10 and one with Transparent+100.
    But this is static and I don't know if you can change the render order on the fly. Possibly with Material.renderQueue.

    Cheers Mark