Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Question regarding backface occlusion, and per-object depth testing

Discussion in 'Shaders' started by mcurtiss, Jun 19, 2020.

  1. mcurtiss

    mcurtiss

    Joined:
    Nov 29, 2012
    Posts:
    26
    I have this edge case scenario where I want an object's backfaces to be occluded depth-wise by its own frontfaces, but to *not* be occluded by frontfaces from other objects.

    Here's a simple visual example of what I want to achieve:

    Two intersecting cubes:



    Two intersecting cubes with simple plane clip to reveal internal backfaces (rendered magenta):



    Below is the desired effect where backfaces render in front of front-faces of other objects, but behind self-frontfaces (this is photoshopped).



    I'm having trouble figuring out how to manipulate the depth buffer/z-testing/stencils to get this effect. Any guidance would be appreciated.

    It would be trivial if I could query depth buffer for just the object's self, but I don't think that's possible.
     
  2. mcurtiss

    mcurtiss

    Joined:
    Nov 29, 2012
    Posts:
    26
    On the off chance anyone from the future has a similar problem:

    The other day while in the shower I realized that I could abstract the 'rule' I want to achieve: Only render pixels that have more backfaces than frontfaces. So we can easily generate mask by increasing the stencil buffer value on backfaces and decreasing it when rendering frontfaces (ZTest Always for both front and backfaces). We can access this mask in another shader by checking if the stencil buffer value is >= 1 (if there are equal number of frontface/backface writes, the value will be 0. If there is an exposed backface, the value will be >= 1).

    E.G:

    BackfacePass:


    Code (CSharp):
    1.        
    2.             Cull Front
    3.             ZTest Always
    4.             ZWrite On
    5.             Stencil
    6.             {
    7.                 Comp Always
    8.                 Pass IncrSat
    9.             }
    FrontFacePass:

    Code (CSharp):
    1.  
    2.            Cull Back
    3.             ZTest Always
    4.             ZWrite On
    5.  
    6.             Stencil
    7.             {
    8.                 Comp Always
    9.                 Pass DecrSat
    10.             }
    Draw to StencilMask in another pass/shader:

    Code (CSharp):
    1.  
    2.         Stencil{
    3.             Ref 1
    4.             Comp LEqual
    5.         }
     
    Last edited: Jun 23, 2020
  3. mcurtiss

    mcurtiss

    Joined:
    Nov 29, 2012
    Posts:
    26
    I should also add, that shader passes are not necessarily rendered in order, depending on the material's position in the render queue. It seems that 2500 and below queues are seen as Opaque objects, for these Objects Unity can assume that the pass order shouldn't matter in total (per object the order always stays 1,2,3) and thus can try to optimize the process of how the passes are sorted for the GPU vs in what order to render the objects (closest to camera first). So if you want the passes to be executed in the proper order, set render Queue on material to be 2501+.

    Important to point out, because out of order passes can affect stencil read/writes for opaque objects.