HDRP Stencil ref and stencil buffer

Is there any clean way of working with the stencil buffer in HDRP, or is such a thing coming soon? Would be fantastic if it was part of the shader graph.

I'm making a worm-hole kind of effect which should appear as a tunnel when looking into it, but not from the sides, but this has proven incredibly difficult in HDRP.

 

There are two stencil bits reserved for users but there's no built-in way to set these bits individually from SG. You need custom stencil passes to set them.

I wish Unity would expose them same was as current decal and ssr stencil bits on the material settings.

 

Yeah, having it on the material would be great. Do you think that's something that will come along, or are they pushing for custom passes to be the new norm for that?

 

You can basically see the existing userbits here:
https://github.com/Unity-Technologi...tion/Runtime/RenderPipeline/HDStencilUsage.cs
https://github.com/Unity-Technologi...Runtime/RenderPipeline/HDStencilUsage.cs.hlsl

Here's a naive example for setting the hdrp stencil on a custom pass: https://cdn.discordapp.com/attachme...95847590445058/CustomStencilPass.unitypackage

Package has custom stencil pass shader + mat + prefab, just set the material's stencil mask to whatever you need to set and also set the custom passes layer value to something that holds objects you want to tag with the shader. Stencil shader here is modified from some stencil test I found on SRP repo.

Do note that it sets the whole stencil for the selected material, meaning it will override built-in stencil values as well (including SSR, decal etc stencil values).

As additional note, I think I forgot to enable gpu instancing on the material so you may want to do that if you use this for many objects.

 

I have no idea if this is on Unity's plans, but like mentioned, would love to have it built-in as it wouldn't be much extra code from Unity's side.

 

Thanks for sharing this, I've been looking for a way to work with the stencil buffer in HDRP for a while, to be able to make portals, etc

I tried your example in unity 2019.4 and can't get it to work though, is there any special setting required? I can't see the custom pass in the frame debugger either, so I'm not sure where the issue is...

could you elaborate a bit on how you went about using it ? thanks!

 

I'm also interested in using this, I'm hoping to be able to have hidden objects that only become visible when viewed through another object.

I'm not sure if that is similar to what this does, but I don't really understand the instructions given either.

Is it possible for you to upload a very simple scene for how that package can be used rz_0lento?

I'm looking to achieve what is described here, but in HDRP using VR, does anyone know if it is possible?
https://answers.unity.com/questions/1656794/help-converting-stencil-shader-in-hdrp.html

 

It's possible that unitypackage doesn't work anymore on recent HDRP, it's been a while since I did it. I also don't have regular shaders that show the results, I've only used this myself for custom postprocessing (to tag objects for customized TAA). But if I remember right there should be some debug view shader for stencil buffer on Graphics repo on github.

I'll check it out.

 

You can find the relevant info here on the Graphics repo stencil test scene:
https://github.com/Unity-Technologi.../2x_Lighting/2208_And_2209_Reflection_Stencil
(See both StencilShowShader and StencilWriteShader)

Check especially scene 2208.

 

Thanks for your help!

I took those shaders from the Graphics repo and have got it working in my VR scene so that an Object with StencilWriteShader is only visible when I look at it through an object with StencillShowShader.

The final piece I need for it to be usable for my game is that I need the StencilShowShader to be partially transparent with an alpha level, so that I can see that object easily.

I think I just need an extra pass on the shader to render the StencilShowShader material itself, but when I try to add it it only renders in my left eye.

I then followed this code to make my new pass support single instance rendering,but for some reason adding the line UNITY_DECLARE_SCREENSPACE_TEXTURE breaks my shader - https://docs.unity3d.com/Manual/SinglePassInstancing.html

Anyway, thank you for your advice, it has set me on hopefully the right path!