Question Stencil Buffer/Shader works in viewport, but is bugged during play?

Hi everyone, I was making my stencil shader code according to user aldonaletto's answer in this thread;

https://answers.unity.com/questions...4.1431465914.1595946491-1704241803.1588690360

And it all seems to work well in the viewport, as you can see, the blue room behind is being obscured by my invisible wall



However, once I pressed play the invisible wall keeps drawing what the camera sees onto itself. Which produces this weird effect:



Any suggestions or solutions for this will be very appreciated!

Unity Version 2018.4.22f1

 

That Unity Answer is from 2012, and some things have changed since then. The main issue is how the skybox and frame buffer clear is handled.

The frame buffer clear is how the game ensures that the previous frame's contents aren't still visible when the game renders the next frame. There's kind of two ways to handle clearing for a new render frame. You can either clear the entire contents of the frame buffer, both the color and the depth/stencil buffer, or you can clear just the depth/stencil buffer and assume the new frame will entirely replace the contents of the color data. The reason to do the second method is because there's some cost to clearing the frame buffer, so if you only clear the depth it's a little faster.

Why this matters is since Unity 5.0 they changed the skybox from rendering as the first thing that the game renders to rendering between the opaque queue range and the transparent queue range. They did this because the default proceedural skybox was more expensive than a basic texture based skybox, so it would be faster to only render it in the pixels it's needed. By rendering the opaque objects first they can fill in the depth buffer and then render the sky to only the areas of the screen no opaque objects have set the depth for. The result is the main cost of the skybox is only in the pixel where it would be visible.

But there's an underlying assumption here. That "render the skybox after the opaque" assumes any object that renders to the depth buffer also renders to the color buffer. The depth shader you're using to occlude the objects behind it isn't rendering a color value, so now you're seeing the previous frame's content that wasn't cleared!

The scene view still uses the old style of rendering the skybox first, hence why it works.


There are a few options.
One option is to render your skybox manually with a material queue of 0 and use a solid color or depth only clear on your camera. This can be done by attaching a box to the camera with a script or shader that keeps it aligned to the world. Or just a really, really big box. Depending on the skybox shader you're using you may need to either write a new completely custom shader, or provide your own mesh, as the skybox meshes Unity uses aren't something you can access from script or as an asset, and there's more than one mesh that the engine is automatically choosing based on the shader name.

Another option is to have the "Invisible Mask" shader draw the skybox on it's surface, but that only works if the skybox is all you need behind it.

The easiest option is to move it to a transparent queue, like

"Transparency-1"

(2999), and then render your hidden objects using a later transparent queue.