Resolved Text outline effect for text

Hello there.

I wanted to add opaque font outline to the TMP's pixel font outline shader. So my implementation idea was simple:

• in first pass i render TMP's pixel glyph:

Code (CSharp):

1. Pass
2. {
3.     Stencil
4.     {
5.         Ref [_Stencil]
6.         Comp [_StencilComp]
7.         Pass [_StencilOp]
8.         ReadMask [_StencilReadMask]
9.         WriteMask [_StencilWriteMask]
10.     }
11.     ColorMask[_ColorMask]
12.    
13.     CGPROGRAM
14.    
15.     float4 _MainTex_TexelSize;
16.     uniform float4 _OutlineColor;
17.    
18.     v2f vert(appdata_t i)
19.     {
20.         float4 vert = i.vertex;
21.         vert.xy += (vert.w * 0.5) / _ScreenParams.xy;
22.         float4 vPosition = UnityPixelSnap(UnityObjectToClipPos(vert));
23.         fixed4 faceColor = i.color;
24.  
25.         v2f o;
26.         UNITY_INITIALIZE_OUTPUT(v2f,o);
27.         o.vertex = vPosition;
28.         o.color = faceColor;
29.         o.texcoord0 = i.texcoord0;
30.         o.texcoord1 = TRANSFORM_TEX(UnpackUV(i.texcoord1), _FaceTex);  
31.            
32.         float2 pixelSize = vPosition.w;
33.         pixelSize /= abs(float2(_ScreenParams.x * UNITY_MATRIX_P[0][0], _ScreenParams.y * UNITY_MATRIX_P[1][1]));
34.  
35.         float4 clampedRect = clamp(_ClipRect, -2e10, 2e10);
36.         o.mask = float4(vert.xy * 2 - clampedRect.xy - clampedRect.zw, 0.25 / (0.25  + pixelSize.xy));
37.  
38.         return o;
39.     }
40.  
41.     float4 frag(v2f i) : COLOR
42.     {
43.         fixed4 c = tex2D(_MainTex, i.texcoord0);
44.         c = fixed4(tex2D(_FaceTex, i.texcoord1).rgb * i.color.rgb, i.color.a * c.a);
45.         half2 m = saturate((_ClipRect.zw - _ClipRect.xy - abs(i.mask.xy)) * i.mask.zw);
46.         c *= m.x * m.y;
47.         return c;
48.     }
49.     ENDCG
50. }

• in second pass i render outline with using stencil buffer to prevent overlapping opaque pixels:

​

Code (CSharp):

1. Pass
2. {      
3.  
4.     Stencil{
5.         Ref [_Stencil]
6.         Comp Equal
7.         Pass IncrSat
8.     }
9.  
10.     CGPROGRAM
11.  
12.     float4 _MainTex_TexelSize;
13.     uniform float4 _OutlineColor;
14.    
15.     v2f vert(appdata_t i)
16.     {
17.         v2f o;
18.         UNITY_INITIALIZE_OUTPUT(v2f,o);
19.         o.vertex = UnityPixelSnap(UnityObjectToClipPos(i.vertex));
20.         o.color = i.color;
21.         o.texcoord0 = i.texcoord0;
22.         return o;
23.     }
24.  
25.     float4 frag(v2f i) : COLOR
26.     {
27.         float4 c = tex2D(_MainTex, i.texcoord0) * i.color;
28.  
29.         float2 texelSize = _MainTex_TexelSize * _Outline;
30.         texelSize = float2(1.0 / 256.0, 1.0 / 256.0) * _Outline;
31.        
32.         float pL = tex2D(_MainTex, i.texcoord0 + texelSize * float2(-1, 0)).a;
33.         float pR = tex2D(_MainTex, i.texcoord0 + texelSize * float2(1, 0)).a;
34.         float pU = tex2D(_MainTex, i.texcoord0 + texelSize * float2(0, 1)).a;
35.         float pD = tex2D(_MainTex, i.texcoord0 + texelSize * float2(0, -1)).a;
36.  
37.         half outline = saturate(pL + pR + pD + pU);
38.  
39.         float4 outlineColor = lerp(0, _OutlineColor, outline);
40.  
41.         clip(outlineColor.a - 0.01);
42.  
43.         outlineColor = lerp(_OutlineColor, 0, c.a);
44.        
45.         return outlineColor;
46.     }
47.  
48.     ENDCG
49. }

So all was alright until I found that my shader affects now on Unity masking mechanics:

​

I tried to find any solution that might prevent this problem, but I couldn't, now I have no idea. Can know how to handle this? I'm new in shader writing.
Thank's in advance.

 

Unity's masking system uses stencils. You're writing to the stencil buffer too, so it's interfering with the masking system.

My recommendation would be to use a hard coded stencil ref that's unlikely to be used by Unity's own masking system. Though this will mean you cannot use a mask to limit outlined text.

Code (csharp):

1. Stencil {
2.     Ref 128
3.     Comp NotEqual
4.     Pass Replace
5. }

 

Thanks for your reply!
I remarked Unity UI uses one type of masks: 0....01 -> if there's content in first mask, 0...11 -> if there's content in second mask which nested in first one, etc. But I don't understand how Unity UI responds to rest stencil values. I would guess rest values should using in user's shader code to implement own logic within Unity UI system.

 

In my project it's important outlined text to be masked,so when shader has stencil > 0 next code will work correctly:

Code (CSharp):

1. Stencil{
2.     Ref [_Stencil]
3.     Comp Equal
4.     Pass DecrSat
5. }

I thinking now about what to do when stencil buffer value is equal to zero.

 

Actually, I think there is a way to solve this with

DecrSat

, but not in the way you’re thinking.

Keep your existing pass as is using

_Stencil

and

IncrSat

, but add another pass that does the

DecrSat

.

Code (CSharp):

1. Stencil {
2.     Ref [_Stencil]
3.     Comp Greater
4.     Pass DecrSat
5. }
6.  
7. ColorMask 0
8.  
9. float4 vert(appdata_t i) : SV_Position
10. {
11.     return UnityPixelSnap(UnityObjectToClipPos(i.vertex));
12. }
13.  
14. fixed4 frag() : SV_Target { return 0.0; }

That third pass should decrement the stencil value back to what it was before. And because Unity’s masking system will never draw an object with a

_Stencil

value greater than the max value that is already in the stencil buffer from masks, there shouldn’t be any other issues. Unity’s masks do something similar after drawing all child objects under them, resetting the stencil to a lower value. The only issue I can think of is if you have 8 chained layers of masks, the

_Stencil

reference value will be

255

, so

IncrSat

won’t do anything. So don’t have that many masks in a single hierarchy.

 

Thanks for you reply again!
But in your code snippet there shouldn't be

Less

instead of

Greater

? As I understood comparison function works as follows:

refValue compFunction bufferValue

. It seems replacing

Greater

function to the

Less

works well as you explained.

 

Oh, yep, you’re right. It’s comparing the ref to the stencil, not the stencil to the ref.

 

Can I ask you about one moment? I have not enough understanding how shader applies to the TMP meshes: per pass for every glyph(mesh)(i.e performing 1st pass from shader for every mesh, next 2nd pass, etc.) or all glyphs batched in one mesh and then shader applies to it? Thanks in advance!

 

If a TMP mesh is using a single font / sprite atlas for all of the glyphs, and you’re not using rich text to change the material mid line, it’s drawn as a single mesh with multiple glyphs in it. As far as Unity’s rendering system is concerned each TMP component is a single mesh.