Search Unity

Using stencil buffer to hide/reveal objects, but hidden objects still receive shadows

Discussion in 'Shaders' started by viveadam, Sep 11, 2020.

  1. viveadam

    viveadam

    Joined:
    May 4, 2020
    Posts:
    5
    Hey all, so I'm using the stencil buffer portal effect based on the Ronja tutorial here: https://www.ronja-tutorials.com/2018/08/18/stencil-buffers.html

    Everything works fine, except the invisible items are still receiving shadows, like so:


    What's stranger is it ignores turning off "receives shadows". It looks the same whether you have it checked or not, but if you set the object's 'stencil reference' to 0 (which makes it visible without the portal) suddenly turning off "receives shadows" starts working:


    Is there any reason why it not being rendered would cause the shadows to be visible, ignoring 'receive shadows' being turned off? And if so, is there anything I can do to the shader to make it ignore shadows?

    Thanks!
     

    Attached Files:

  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,348
    Checking "receive shadows" off won't actually fix it, the phantom shadow receiving will still happen.

    The problem is Unity uses a screen space shadow texture that is generated by casting shadows onto the camera depth texture. Even if you turn receive shadows off on a renderer, all that does is change the shader to ignore that texture, but if that object is hidden (say, by stencils) it'll still be receiving shadows in the screen space shadow texture. That's because it's still rendered to the camera depth texture.

    You can't really stencil an object out of the camera depth texture, since Unity doesn't respect the draw order or stencils properly when it's generating it. So the only thing you can do is remove any object you want to be hidden from the camera depth texture entirely. The easiest way to do that is by moving them into the transparent queue, but for stencils that causes all sorts of other problems.