Search Unity

When is the stencil buffer cleared?

Discussion in 'Shaders' started by MuseGames, Oct 30, 2014.

  1. MuseGames

    MuseGames

    Joined:
    Mar 19, 2008
    Posts:
    98
    This doesn't seem to be answered anywhere in the documentation: at what point(s) in Unity's rendering process is the stencil buffer cleared?

    I'm using forward rendering, if that makes a difference. The specific case that is failing for me: using a RenderWithShader() call to write to the stencil buffer, then attempting to read those values in a Graphics.Blit() call immediately afterwards.
     
  2. MuseGames

    MuseGames

    Joined:
    Mar 19, 2008
    Posts:
    98
    Bumping to see if anyone knows an answer...
     
  3. CHPedersen

    CHPedersen

    Joined:
    Mar 2, 2011
    Posts:
    63
    I won't claim to know how Unity implemented it, but in general, it's pretty standard to clear it alongside the depth and frame buffers. :) I'm guessing Unity does it there, too.
     
  4. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    1,520
    It's actually a pretty good question. In for example DirectX, the stencil and depth buffer isn't cleared by default. You have to do that yourself at the start of a new frame if you want to have it cleared. I think this way is the standard for desktop graphics.

    Mobile devices with tile based deferred rendering don't have a full screen stencil and depth buffer, so there is no option to keep the contents between frames in this case.

    To keep things in line between desktop and mobile, Unity probably clears the stencil buffer at the start of the frame for desktop graphics.

    But that is not something you would notice in your example. So, what are your stencil settings?
     
  5. MuseGames

    MuseGames

    Joined:
    Mar 19, 2008
    Posts:
    98
    I suppose that if CHPedersen's guess is correct, then it would also be cleared at the start of any Blit() call. But that seems unnecessarily restrictive, as it makes it basically impossible to use the stencil buffer in post-process effects.

    But, actually, that doesn't seem to be the case. I've set up a small project to show off my issue, and in doing so discovered that it seems to be failing other than where I thought at first. Namely: RenderWithShader doesn't appear to *write* to the stencil buffer. Objects with my stencil-write shader attached to a normal material are actually masked properly in the Blit stage...

    EDIT: see below for what I think is actually going on

    Demonstration project attached.
     

    Attached Files:

    Last edited: Nov 10, 2014
  6. imaginaryhuman

    imaginaryhuman

    Joined:
    Mar 21, 2010
    Posts:
    5,834
    From what I recall, when the camera applies a `clear` of the buffers, this will include all buffers unless Unity has a setting to not do some.. e.g. depth-only does not clear the color buffers. I presume the camera therefore always clears the stencil buffer each frame, but I'm not certain. It's something you could set up an experiment to test.
     
  7. MuseGames

    MuseGames

    Joined:
    Mar 19, 2008
    Posts:
    98
    I've discovered a few inconsistencies that I think together explain most of my confusion over the issue:

    1) Every RenderTexture has its own stencil buffer. This is implied by the documentation (RenderTexture.SupportsStencil) but I was uncertain in light of other issues.
    2) The stencil buffer in use is determined by the value of RenderTexture.active. This is the buffer being written to or read from by all stencil operations. In particular, the active stencil buffer is *not* affected by setting Camera.targetTexture, which might be a bug (this behavior certainly is not documented anywhere that I've found.)
    3) As far as I can tell, the stencil buffer(s) are cleared when a camera renders, if and only if its clear flags are set to clear the depth buffer. However, explicitly calling GL.Clear() right before Render(WithShader) doesn't affect the stencil buffer.