Stencil comparisons don't work on Android?

I have a pair of shaders that work together. The first one writes on the stencil and the second one checks the value of the stencil and gets drawn if an equal comparison is true.

Everything works fine in the editor but on the Android build the stencil comparison doesn't work. It acts as if the comparison was always true.

It's a weird issue but any help or insight would be much appreciated

 

Stencil support is optional on GLES 2.0 Android hardware, so some simply don’t support it, or it’s disabled by default. Worse, there might be a bug with Unity that they simply don’t even enable it on gles 2.0 devices.

It should work on gles 3.0 devices, but it can be optionally disabled for performance reasons, and Unity might have a bug where it is not enabling it.

In GLES you can choose between 16 bit and 24 bit depth buffers, only 24 bit supports stencils, but comes with a decent performance penalty, hence defaulting to 16 bit.

 

So basically for what I understand I can't rely on any stencil dependat shader since some devices won't even support it right?

I tried both GLES 2.0 and GLES 3.0 tried tweaking all the build settings that I could think of and no luck what so ever. Unity bug or not it looks like I better think something else. Thanks for the heads up.

The objective of the shader was creating some shadows. This shadows are quads with a texture that only gets drawn on surfaces that write to the stencil. With this setup it was waaaaay cheaper than with shadowBlobProjectors or super low res shadowmaps. Do you know of an alternative way of creating any shadow-like effect? I really appreciate your help, in the past I've learned a lot of your answers on other posts and now you have already helped me directly.

 

I've use destination alpha on Android devices to do stencil-like effects. Instead of having your objects' shaders write to the stencil to mark what pixels should or shouldn't have the shadows render on, you can have them either write 0.0 or 1.0 as their alpha. Then the shadow quad needs to use a blend mode that makes use of DstAlpha or OneMinusDstAlpha.

However for shadows I usually just handle it from script and manually 'clip' the shadow quads to the colliders. Really just scale them / fade them when close to an edge so they don't hang off too badly.

 

That's a great idea! I haven't been succesful though... I attatched an image with my current results. On the left the shadow texture with regular blending (Blend SrcAlpha OneMinusSrcAlpha) and on the right what the setup should look like only that I can't get the alpha of the texture right(Blend SrcAlpha DstAlpha).

I will also add the code here just in case:

Code (CSharp):

1. Shader "Unlit/shadow"
2. {
3.     Properties
4.     {
5.         _MainTex ("Texture", 2D) = "white" {}
6.         _Color("Color", Color) = (1,1,1,1)
7.         _Alpha("Alpha", Range(0.0, 1.0)) = 1.0
8.     }
9.     SubShader
10.     {
11.          Tags { "Queue" = "Transparent" "RenderType" = "Transparent" "IgnoreProjector" = "True" }
12.  
13.         Pass
14.         {
15.             ZWrite Off
16.             Blend SrcAlpha DstAlpha
17.             CGPROGRAM
18.             #pragma vertex vert
19.             #pragma fragment frag
20.             #pragma multi_compile_instancing
21.  
22.             #include "UnityCG.cginc"
23.  
24.             struct appdata
25.             {
26.                 float4 vertex : POSITION;
27.                 float2 uv : TEXCOORD0;
28.                 UNITY_VERTEX_INPUT_INSTANCE_ID
29.             };
30.  
31.             struct v2f
32.             {
33.                 float2 uv : TEXCOORD0;
34.                 float4 vertex : SV_POSITION;
35.             };
36.  
37.             sampler2D _MainTex;
38.             float4 _MainTex_ST;
39.             fixed4 _Color;
40.             float _Alpha;
41.  
42.             v2f vert (appdata v)
43.             {
44.                 UNITY_SETUP_INSTANCE_ID(v);
45.                 v2f o;
46.                 o.vertex = UnityObjectToClipPos(v.vertex);
47.                 o.uv = TRANSFORM_TEX(v.uv, _MainTex);
48.                 return o;
49.             }
50.  
51.             fixed4 frag (v2f i) : SV_Target
52.             {
53.                 fixed4 col = tex2D(_MainTex, i.uv) * _Color;
54.                 return col;
55.             }
56.             ENDCG
57.         }
58.     }
59. }
60.  

 

You're going to have to really think about and understand how the Blend (and BlendOp) works.

For example, the traditional alpha blending of Blend SrcAlpha OneMinusSrcAlpha is also assuming the implicit default of BlendOp Add. That results in a blend that's doing the equivalent of:

Code (csharp):

1. // constant ───┰────────────────────┒
2. outColor = SrcColor * SrcAlpha + DstColor * OneMinusSrcAlpha;
3. // BlendOp Add ───────────╫────┘                   ║
4. //          ╔═════════════╝                        ║
5. // Blend ScrAlpha OneMinusSrcAlpha ════════════════╝

You can change what you're multiplying the SrcColor and DstColor by, and how they're being combined to be either add, subtract, reverse subtract (dst - src), min and max.

In the case of a black shadow and your shader, you're really doing this:
outColor = (0,0,0,0) * (shadow texture alpha) + DstColor * (1 - shadow texture alpha);

The first part of that cancels itself out; 0 * anything = 0, and 0 + something = something. So the important portion can be simplified down to just:
outColor = DstColor * (1 - shadow texture alpha);

So how can you recreate the above, or at least get close? Unfortunately, there's no multiply blend op (well, there can be in OpenGL, but it's not that useful), but you can get close with:
outColor = DstColor - shadow texture alpha;╗
╔══════════════════════════════════════════╝
outColor = DstColor * One - SrcColor * One;

or:
BlendOp ReverseSubtract
Blend One One

// fragment shader
return fixed4(col.aaa, 1.0);

One step further, you can add in the destination alpha:
BlendOp ReverseSubtract
Blend DstAlpha One

outColor = DstColor * One - SrcColor * DstAlpha;

And there you go. Anywhere the destination alpha is 1.0 will show the shadow. Everywhere it's 0.0 won't.

 

You really helped further understand this whole alpha blending thing. But unfortunately this setup doesn't work and I think I understand why (I may be wrong though).

Let's just assume that the ground is white and that the shadow texture has no gradient and that its alpha is either 1 or 0. And that everypixel on the screen that the shadow can be casted on has an alpha of 1 and everything else has an alpha of 0. In this case if we do:
BlendOp RevSub
Blend One One

This would mean that for every pixel of the shadow that is overlapped with the white surface we'll have one of this 2 cases depending on the alpha of the shadow texture:
1.Black parts of the shadow texture: outColor = (white, 1) * One - (0,0,0,1) * One; => outColor = (white, 0); //Which means that we ain't drawing anything since outColor's alpha is 0 and the white below will be the thing visible.

2. Transparent parts of the shadow texture: outColor = (white, 1) * One - (0,0,0,0) * One; => outColor = (white, 1); //Which means that we paint white to the already white pixel

If we try with the other setup(with the same setup) it won't work for the same reason:
BlendOp RevSub
Blend DstAlpha One

The problem is that the (shadow texture color * whatever SrcFactor we choose) = (0,0,0,1) or (0,0,0,0) and this gives no room to achieve the desired result unless it can be multiplied.

All of this stuff is very new for me and my brain hurts but I think that this is whats going on and this is the reason it won't work. Also I don't understand your previous reply from the point you say "One step further" and maybe that's the reason I'm failing miserably.

 

I think you missed this line:

The color is ignored. We're instead outputting the alpha as the color in my above example. However, you're not wrong that this does not match alpha blending. It's more like a color burn blend in Photoshop.

Here's an example that does use color:

Code (CSharp):

1. Shader "Unlit/DstAlphaBlobShadow"
2. {
3.     Properties
4.     {
5.         _Color ("Shadow Color", Color) = (1,1,1,1)
6.         [NoScaleOffset] _MainTex ("Shadow (A)", 2D) = "white" {}
7.     }
8.     SubShader
9.     {
10.         Tags { "Queue"="Transparent-499" "RenderType"="Transparent" }
11.         LOD 100
12.  
13.         Pass
14.         {
15.             BlendOp RevSub
16.             Blend DstAlpha One
17.             ZWrite Off
18.             Cull Off
19.  
20.             CGPROGRAM
21.             #pragma vertex vert
22.             #pragma fragment frag
23.  
24.             #pragma multi_compile_fog
25.            
26.             #include "UnityCG.cginc"
27.  
28.             struct appdata
29.             {
30.                 float4 vertex : POSITION;
31.                 float2 uv : TEXCOORD0;
32.             };
33.  
34.             struct v2f
35.             {
36.                 float4 pos : SV_POSITION;
37.                 float2 uv : TEXCOORD0;
38.                 UNITY_FOG_COORDS(1)
39.             };
40.  
41.             fixed4 _Color;
42.             sampler2D _MainTex;
43.            
44.             v2f vert (appdata v)
45.             {
46.                 v2f o;
47.                 o.pos = UnityObjectToClipPos(v.vertex);
48.                 o.uv = v.uv;
49.                 UNITY_TRANSFER_FOG(o, o.pos);
50.                 return o;
51.             }
52.            
53.             fixed4 frag (v2f i) : SV_Target
54.             {
55.                 fixed4 col = tex2D(_MainTex, i.uv);
56.                 col.a *= _Color.a;
57.                 col.rgb = (fixed3(1,1,1) - _Color.rgb) * col.a;
58.  
59.                 UNITY_APPLY_FOG_COLOR(i.fogCoord, col, fixed4(0,0,0,0));
60.                 return col;
61.             }
62.             ENDCG
63.         }
64.     }
65. }

 

I completely missed that line. But when you highlighted it everything clicked in my head. I really understand it now! And it works in-game too!

You have no idea how much I learned just by reading to your replies and using it to bang my head against this thing. For real thank you, thank you, thank you. Keep being awesome and see you around

 

Whether it's true per the spec, it's certainly true many older GLES 2.0 Android devices did not have a working stencil buffer. Enabling 24 bit depth and stencil wouldn't error, but it also wouldn't actually do anything. So maybe "was treated as optional" would be more accurate.

There are a lot of features early gles 2 phones treated as "optional"...

 

So, continuing this old-ish conversation, since I am considering of using stencil for a mobile project and I'm wondering if I'm going to be opening a new can of weird incompatibility worms:

Barring any unforeseen issues or bugs you are unaware of, stencils on mobile devices that are currently supported by 2018.4, should "just work", right?

 

I'am no sure if this is a bug or me doing something wrong, even in the editor unity 2019.1.f1 i have a weird behavior.
on the Google Pixel 2 it cuts it out like there was no ZTest at all.

Code (CSharp):

1.  
2. Shader "HioVR/Mask OneZLess"
3. {
4.  
5.     Properties
6.     {
7.             [IntRange] _StencilRef("Stencil Reference Value", Range(0,255)) = 1
8.     }
9.        
10.     SubShader
11.     {
12.         Tags { "RenderType"="Opaque" "Queue"="Geometry-1" }
13.         ColorMask 0
14.         ZWrite off
15.              
16.         Pass
17.         {
18.             //Cull Back
19.             ZTest Always
20.        
21.                 Stencil
22.                 {
23.                     Ref[_StencilRef]
24.                     //    Ref 1
25.                     Comp always
26.                     Pass replace
27.                    
28.                  }
29.        
30.             CGPROGRAM
31.             #pragma vertex vert
32.             #pragma fragment frag
33.            
34.             struct appdata
35.             {
36.                 float4 vertex : POSITION;
37.             };
38.             struct v2f
39.             {
40.                 float4 pos : SV_POSITION;
41.             };
42.             v2f vert(appdata v)
43.             {
44.                 v2f o;
45.                 o.pos = UnityObjectToClipPos(v.vertex);
46.                 return o;
47.             }
48.             half4 frag(v2f i) : COLOR
49.             {
50.                 return half4(1,1,0,1);
51.             }
52.            
53.             ENDCG
54.         }
55.     }
56. }

And my characers have standard unity shader on an i use this pass:

Code (CSharp):

1.             Tags {  "Queue" = "Geometry-1" "RenderType" = "Opaque" "PerformanceChecks" = "False" }
2.             LOD 300
3.             Zwrite on
4.                
5.             // ------------------------------------------------------------------
6.             //  Base forward pass (directional light, emission, lightmaps, ...)
7.             Pass
8.             {
9.                 Name "FORWARD"
10.                 Tags { "LightMode" = "ForwardBase" }
11.  
12.             Stencil
13.             {
14.                 Ref 1
15.                 Comp notequal
16.                 Pass keep
17.                 ZFail keep
18.             }
19.  
20.                 Blend[_SrcBlend][_DstBlend]
21.                 ZWrite[_ZWrite]
22.  
23.                 CGPROGRAM
24.                 #pragma target 3.0

 

ZTest Always means never do ZTest. The ZTest setting is for saying under which situations should the ZTest succeed and allow the fragment to render. Always means it’ll always succeed, thus always render.

The differences you’re seeing a purely down to render order. ZTest isn’t involved at all. Sometimes the character closer to you is rendering before the mask, sometimes after. If it renders before, then it isn’t affected by it and renders normally. If it renders after then it is affected by it. Unity’s sorting isn’t perfectly consistent with the distance to the camera because it’s also sorting by material to try to improve rendering performance. If you absolutely need a specific rendering order, like when using stencils, then you’ll need to manage that manually.

 

Hey guys. I have the exact same problem on URP and Android. In the editor everything seems to work just fine. But as I build on Android for Oculus Quest 2 the stencils are not working. Textures etc. are visible and fine but every material passes the Stencil. Has anyone found a solution?

 

Hi,
I have a strange URP stencil problem on Android.
Everything works well in Editor and in WebGL.
On Android everything works well when stencils are applied as RenderObjects override.
When the stencils are coded inside shader code, the meterial doesn't show up at all.
That happens in Unity 2020.2.2 and also in 2021.1.3
- - - - - - - - - - - - - - - - - - - - - - -
update

I had just to replace all the occurrences of UsePass, like:

UsePass "Universal Render Pipeline/Complex Lit/SHADOWCASTER"

with the actual pass code.