Shader help: Conditionally writing to stencil buffer based on TextMeshPro color

Hello Unity community,

I'm trying to write a shader that will allow me to write TextMeshPro text to the stencil buffer wherever the text would normally draw a sprite on the screen.

Here is what my text looks like before any additional work. There is the TextMeshPro object (the white text) and then there is a planar surface behind the text which is reflecting some lights.



The intended effect is that the white text become a stencil that is used by the planar surface, so that the planar surface will only be drawn where what is currently the white text will be drawn. The white text will not be drawn at all, it will only be used as a stencil.

I wrote a simple shader for the text that just writes to the stencil buffer, which is then used by a shader on the planar surface, but because the TextMeshPro object's geometry is actually much broader than the geometry of the sprites it renders, the effect is not quite what I am looking for. See below:



Does anyone have any advice on how I can get this effect working? I don't know if I am barking up the wrong tree, but if I could conditionally write to the stencil buffer based on what color would normally be drawn to a pixel, I think I could make this work. I have no idea if that is possible though.

Any help is super appreciated! Thank you.

 

The stencil is written anywhere the shader is rendered to the screen. When a shader is using alpha blending, the entire mesh is still getting rendered to the screen even in the areas it's transparent. To avoid that you need to use an alpha tested or alpha to coverage shader.

Alpha to coverage requires MSAA to work properly, otherwise it's identical to alpha testing (on PC). So instead you can try adding something like this to a custom TMP shader.

clip(color.a - 0.001);

Except here's, the weird thing, that snippet of code was copied from the default TMP shader. The functionality is already there, just inside of a #if that's never ever enabled. I believe you can enable this by using the debug inspector to add

UNITY_UI_ALPHACLIP

to the material's ShaderKeywords.

 

@bgolus Thank you so much! I was able to get the effect working and it looks great! For some reason the HLSL `clip` command never really came up during my reading of the Unity documentation.