Vertex shader normalized 2D vertex position on unit circle mesh produces wrong normalization result

Hi

I'm having a weird problem.
I've generated a unit circle mesh by defining the vertices using this code:

Code (CSharp):

1. Vector3[] vertices = new Vector3[sides + 1];
2. vertices[0] = Vector3.zero;
3. for(int i = 0; i < sides; i++)
4. {
5.     float angle = i * 2 * Mathf.PI / sides;
6.     vertices[i + 1] = new Vector3(Mathf.Cos(angle), Mathf.Sin(angle), 0);
7. }
8.  

The reason I'm showing you the vertex position generation code is because the problem I'm having only happens when applying the shader on this particular mesh. And the problem is with normalizing xy components of object space position in a vertex shader.

Code (CSharp):

1.  
2. normalize(input.pos.xy); // THIS DOES NOT WORK (produces close to zero values)
3. normalize(input.pos).xy; // THIS WORKS
4.  

the whole test shader code is this:

Code (CSharp):

1. Shader "VertexShaderNormalizePositionTest"
2. {
3.     SubShader
4.     {
5.         Pass
6.         {
7.             CGPROGRAM
8.  
9.             #pragma vertex vertex_shader
10.             #pragma fragment fragment_shader
11.            
12.             struct VS_Input //vertex shader input
13.             {
14.                 float4 pos   : POSITION;
15.             };
16.            
17.             struct VS_Output
18.             {
19.                 float4 pos  : SV_POSITION;
20.                 float2 data : TEXCOORD0;
21.             };
22.            
23.             VS_Output vertex_shader(VS_Input input)
24.             {
25.                 VS_Output output;
26.  
27.                 output.pos = UnityObjectToClipPos(input.pos);
28.                 output.data = normalize(input.pos.xy); // THIS DOES NOT WORK
29.                 //output.data = normalize(input.pos).xy; // THIS WORKS
30.  
31.                 return output;
32.             }
33.            
34.             fixed4 fragment_shader(VS_Output input) : COLOR
35.             {
36.                 return float4(input.data, 0, 1);
37.             }
38.  
39.             ENDCG
40.         }
41.     }
42. }

I've looked into difference in compiled code and this is the diff:



Can someone explain to me what is going on?

 

If

normalize(input.pos).xy

works then what's the problem?

 

Finding a workaround is not a problem. The problem is understanding why the workaround is even needed, as it could imply a possible bug in the shader compiler.

I tried experimenting a little further and if I pass the pos.xy to fragment shader and use

Code (CSharp):

1. normalize(pos.xy)

in the fragment shader, then it also works, but it fails when used in a vertex shader.

Few more things that also work in vertex shader:

Code (CSharp):

1. normalize(float3(input.pos.xy, <insert any value othar than zero here>))

Code (CSharp):

1. normalize(input.pos.xyw).xy

Code (CSharp):

1. normalize(float2(1,0))

The last one is just the most simple test to see if normalize on float2 even works.

So the question remains, why does it fail only on a unit circle vertices?

 

normalize(pos.xy)

is actually the correct way to go. I'm not sure why you'd be getting "close to zero values", but the reason that you're getting different values is because pos is not a float3, it's a float4. Which turns out to be rather important. Position values are identified by a 1 in the w field, and the vector will be normalized as a float4, meaning that it will take the 1 into account when doing so, affecting the values of x and y.

 

I took your shader, unmodified, and put it on a unit circle mesh I have (imported from an fbx, not generated in editor) and I get this:


That is the exact result I expect to see. Is that not what you're seeing?

If this is what you're seeing, what were you expecting to see?
If this isn't what you're seeing, can you post a screenshot?

 

I agree that points are represented as (x,y,z,1), but that means that constructing a vector pointing from origin point (0,0,0,1) to the 2D point represented in 3D as (x,y,0,1) would result in a vector (x,y,0,0) and so there should be no difference when calculating the normalized version of (x,y) or (x,y,0,0).

 

This the result for "normalize(input.pos).xy" ... this is what I expect to see, but ...


This is for "normalize(input.pos.xy)" ... clearly something is wrong

 

Oh, wait, I missed this:

Yep, you can't do this with normalize().

2D or 3D is irrelevant here. You're normalizing the mesh position as Unity sends the data. That vertex position data is always a float4 with the w component == 1.

That center vertex is float4(0,0,0,1) and normalize(float2(0,0)) produces a NaN, which then propagates out to the interpolated values. You cannot normalize an zero length vector in a shader, and the built in function offers no protection against that. When you do normalize(v.pos).xy you're doing normalize(v.pos.xyzw).xy which for that center vertex is normalize(float4(0,0,0,1)).xy, which does not produce a NaN since it is not zero length, but also isn't remotely correct for what you're trying to do.

The solution is to use what's known as a "safe normalize". Unity provides one for float3 values in the UnityStandardUtils.cginc file, and it looks like this:

Code (csharp):

1. inline float3 Unity_SafeNormalize(float3 inVec)
2. {
3.     float dp3 = max(0.001f, dot(inVec, inVec));
4.     return inVec * rsqrt(dp3);
5. }

This code is almost exactly what normalize() does in the shader. The last highlighted line in your compiled shaders is that dot product, after that should be a rsqrt and a mul. The problem is the dot product of a zero length vector is zero, and an rsqrt of that produces a NaN. So a safe normalize just limits the output of the dot product to a small, but arbitrary non zero number.

It works on the mesh I tested with because I did not have a center vertex.

 

I can't thank you enough
I knew I was missing something ... never thought of NaN getting interpolated to other vertices (I thought it would just result as black color in the middle of the circle if something goes wrong with calculation)

 

@bgolus curious, because my code worked as expected using his code essentially verbatim.