Search Unity

Stencil Test Weirdness

Discussion in 'Shaders' started by thebarryman, Aug 11, 2020.

  1. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    I am drawing a few different passes with stencil operations that go something like -


    Stencil test always, pass: replace 1
    Stencil test == 1, pass: increment
    Stencil test == 2, pass: increment


    Stencil test <= 1, pass: keep

    ^ this last pass only draws over pixels with stencil value == 1. If I change it to Stencil test == 0, it will draw over the other pixels I want, but not the ones containing 1. I get the same result if I change the stencil test to < 2.

    Has anyone else encountered this issue? I'm using Unity 2019 with URP.
     
  2. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    I was able to get around this by changing the passes to the following -

    Pass 0
    Test: Always, Ref: 255, Pass: Replace
    Pass 1
    Test: Equal, Ref: 255, Pass: Decrement
    Pass 2
    Test: Equal, Ref: 254, Pass: Decrement
    Pass 3
    Test: GEqual, Ref: 255, Pass: Keep

    For whatever reason, testing >= 255 passes when the stencil value == 0, but <= 1 does not. Weird, but I'm OK with it!

    I am running into another issue that might be related though. If do the following -

    Pass 0
    Test: Always, Ref: 255, Pass: Replace
    Pass 1
    Test: LEqual, Ref: 255, Pass: Decrement

    Drawing multiple objects in pass 1 on top of each other fails for any stencil value < 255. Basically, it acts as an Equal test instead of an LEqual test.

    I should note that rather than explicitly setting these values in shader code, I am passing the comparisons in from UnityEngine.Rendering.CompareFunction. Should this be expected to fail?
     
    Last edited: Aug 12, 2020