Floating point precision in Unity 5 vs Unity 4.6

I am having an issue with some float values.

Here is the example

Code (CSharp):

1.         Vector3 center = new Vector3(294.5f,0f,75.5f);
2.         Vector3 start = new Vector3(294.5f,0f,78.1f);
3.         float value = Vector3.Dot(center, center) + Vector3.Dot(start, start) - 2 * Vector3.Dot(center, start) - (.5f * .5f);
4.         Debug.Log(value);

In Unity 5, the result is 6.515625
In Unity 4.6, the result is 6.509992

How can I make it so that Unity 5 gets the same result as Unity 4.6?
In Unity 4.6, if I put (float) in front of all the Vector3.Dot methods, then it returns the same result as in Unity 5, so how would I do the reverse for Unity 5 to get what Unity 4.6 has?

Also, why is this happening? I'm assuming Unity 5 has a new version of C# and it is doing something in the back-end, but what exactly is it doing?

Ive also noticed Unity 5's Physics casts (raycast, spherecast...) return different values than in 4.6
So many times I see a very small float value representing 0 instead of just 0 like in Unity 4.6

 

My first guess would be because the Unity 5 Editor is 64 bit

 

These differences are miniscule. Why are they breaking your code?

You should never be comparing two floats with "==", for this exact reason. Float point calculation is going to vary slightly from environment to environment and system to system.

Use Mathf.Approximatly(), instead:

Code (CSharp):

1. float f1 = 6.078127;
2. float f2 = 6.077951;
3.  
4. bool equal = Mathf.Approximatly(f1,f2); //true

 

Worth noting that, if the change to 64 bit is the cause, then building to any 64-bit platform (e.g. iOS) would have resulted in the same differences.

 

I think the problem might have to do with CapsuleCasts. In the past, when a CapsuleCast hits something, like a big box, the hitInfo.Point would prefer to be at the center forward direction of the CapsuleCast. However, in Unity 5, it seems this is no longer the case, which means my hitInfo.Point is changing a lot more, which might be opening up to more floating point imprecision that I did not see before since the hitInfo.Point was relatively stationary.
I can somewhat confirm this is my issue as I was able to make the Unity 4.6 test show similar bad results as the Unity 5 test scene I have.

So the problem in my original post may indeed be very minuscule and was just a distraction to the overall real problem.

 

To make Unity 5.0 behave like 4.x follow these simple steps (Windows)

• Go to the control panel
• Select Add or Remove programs
• Locate Unity
• Click Uninstall
• Download Unity 4.x from here
• Run the installer

Aside from that you are out of luck.

But seriously, requiring accuracy on the 7th significant figure is pretty much the limit for a 32 bit float, especially when you have 20-30 arithmetic operations going on. You could use a 64 bit float.

Have you considered simplifying your math? Write out the entire operation in full and you will see there is a lot of duplicate work going on. Simplifying your arithmetic will improve accuracy. Especially if you can avoid some of the big intermediate numbers.

 

Perhaps you or someone can take a look at the code and let me know what can be done for more accuracy =)

Here is all the code..
Put the "TestCastsProblems" script on a gameobject, place a cube with a collider in front so the capsulecast can hit it. When you move the cube up and down, you will notice the capsule that is spawned to represent the capsulecast will start to shake. If you make the cords of the cube and the gameobject that has this script on it whole numbers, then the shake will stop, but if there are more floating points, things get messy. I don't know what to do.

Spoiler: Click for Codes

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class TestCastsProblems : MonoBehaviour
5. {
6.     GameObject capsule;
7.     float radius = .5f;
8.     float distance = 100;
9.  
10.     void Start()
11.     {
12.         if(GetComponent<Collider>())
13.         {
14.             gameObject.GetComponent<Collider>().enabled = false;
15.         }
16.         Debug.DrawRay(transform.position, transform.forward * distance, Color.blue, 100000);
17.         Debug.DrawRay(transform.position, transform.up * distance, Color.blue, 100000);
18.  
19.         MeshTestCreate(ref capsule, Vector3.zero, PrimitiveType.Capsule);
20.     }
21.  
22.     void Update()
23.     {
24.         Vector3 castOrigin = transform.position;
25.         Vector3 bottomPoint = castOrigin - (transform.up * radius);
26.         Vector3 topPoint = castOrigin + (transform.up * radius);
27.      
28.         RaycastHit hitInfo;
29.         Physics.CapsuleCast(bottomPoint, topPoint, radius, transform.forward, out hitInfo, distance);
30.  
31.         if(hitInfo.collider)
32.         {
33.             capsule.transform.position = ExtensionsPhysics.CapsuleCastCenterOnCollision(bottomPoint, topPoint, radius, transform.forward, hitInfo.point);
34.             Debug.Log(capsule.transform.position.x + " " + capsule.transform.position.y + " " + capsule.transform.position.z);
35.         }
36.     }
37.  
38.     void MeshTestCreate(ref GameObject newGameObject, Vector3 position, PrimitiveType type)
39.     {
40.         newGameObject = GameObject.CreatePrimitive(type);
41.         newGameObject.transform.position = position;
42.         newGameObject.GetComponent<Collider>().enabled = false;
43.     }
44. }

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public partial class ExtensionsPhysics
5. {
6.     public static Vector3 CapsuleCastCenterOnCollision(Vector3 bottomPoint, Vector3 topPoint, float radius, Vector3 directionCast, Vector3 hitPoint)
7.     {
8.         Vector3 origin = ExtensionsVector3.Vector3CenterPoint(bottomPoint, topPoint);
9.         Vector3 storedOrigin = origin;
10.      
11.         Vector3 bottomTopPointDirection = (topPoint - bottomPoint).normalized;
12.         float originHitPointDistance = Vector3.Distance(origin, hitPoint);
13.  
14.         Vector3 endPoint = origin + (directionCast * originHitPointDistance);
15.  
16.         float originHitPointHeight = ExtensionsVector3.ProjectDistance(origin, hitPoint, bottomTopPointDirection);
17.  
18.         float height = Vector3.Distance(bottomPoint, topPoint) + (radius * 2);
19.         float heightRadiusDifference = (height / 2) - radius;
20.  
21.         bool isTop = ExtensionsVector3.IsInDirection(bottomTopPointDirection, origin, hitPoint);
22.         if(originHitPointHeight > heightRadiusDifference)
23.         {
24.             if(isTop)
25.             {
26.                 origin += bottomTopPointDirection * heightRadiusDifference;
27.                 endPoint += bottomTopPointDirection * heightRadiusDifference;
28.             }else{
29.                 origin -= bottomTopPointDirection * heightRadiusDifference;
30.                 endPoint -= bottomTopPointDirection * heightRadiusDifference;
31.             }
32.          
33.         }else if(isTop)
34.         {
35.             origin += bottomTopPointDirection * originHitPointHeight;
36.             endPoint += bottomTopPointDirection * originHitPointHeight;
37.         }else{
38.             origin -= bottomTopPointDirection * originHitPointHeight;
39.             endPoint -= bottomTopPointDirection * originHitPointHeight;
40.         }
41.  
42.         Vector3 capsuleCastCenter = ExtensionsPhysics.SphereLineIntersect(origin, endPoint, hitPoint, radius);
43.         capsuleCastCenter = storedOrigin + capsuleCastCenter;
44.      
45.         return capsuleCastCenter;
46.     }
47. }

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public partial class ExtensionsVector3
5. {
6.     public static Vector3 Vector3CenterPoint(params Vector3[] points)
7.     {
8.         Vector3 centerPoint = Vector3.zero;
9.  
10.         foreach(Vector3 point in points)
11.         {
12.             centerPoint += point;
13.         }
14.  
15.         return (centerPoint / points.Length);
16.     }
17. }

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public partial class ExtensionsVector3
5. {
6.     public static float ProjectDistance(Vector3 startPoint, Vector3 secondPoint, Vector3 direction)
7.     {
8.         return Vector3.Distance(startPoint, Project(startPoint, secondPoint, direction));
9.     }
10.  
11.     public static Vector3 Project(Vector3 startPoint, Vector3 secondPoint, Vector3 direction)
12.     {
13.         return startPoint + Vector3.Project(secondPoint - startPoint, direction);
14.     }
15. }

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public partial class ExtensionsPhysics
5. {
6.     public static Vector3 SphereLineIntersect(Vector3 lineStartPoint, Vector3 lineEndPoint, Vector3 sphereCenterPoint, float sphereRadius)
7.     {
8.         Vector3 startPointEndPointDifference = lineEndPoint - lineStartPoint;
9.      
10.         float a = Vector3.Dot(startPointEndPointDifference, startPointEndPointDifference);
11.         float b = 2f * Vector3.Dot(startPointEndPointDifference, lineStartPoint - sphereCenterPoint);
12.         float c = Vector3.Dot(sphereCenterPoint, sphereCenterPoint) + Vector3.Dot(lineStartPoint, lineStartPoint) - 2f * Vector3.Dot(sphereCenterPoint, lineStartPoint) - (sphereRadius * sphereRadius);
13.      
14.         float delta = b * b - (4f * a * c);
15.  
16.         //Find the two intersections.
17.         if (delta > 0)
18.         {
19.             float SquareRootDelta = Mathf.Sqrt(delta);
20.          
21.             float intersect2 = (-b - SquareRootDelta) / (2f * a); //We only need the second hit but for the first hit you do ____ float intersect1 = (-b + SquareRootDelta) / (2 * a); ___ result is ___ storedCapsuleCastStartPointCenter + ((startPointEndPointDifference * intersect1))
22.  
23.             //Local intersect vector
24.             Vector3 intersectPoint = startPointEndPointDifference * intersect2;
25.  
26.             return intersectPoint;
27.         }
28.         //If nothing intersected
29.         return Vector3.zero; //cant return null with vector3 so we return a zero value
30.     }
31. }

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public partial class ExtensionsVector3
5. {
6.     public static bool IsInDirection(Vector3 direction, Vector3 startPoint, Vector3 targetPoint)
7.     {
8.         Vector3 targetLocalPoint = targetPoint - startPoint;
9.         return IsInDirection(direction, targetLocalPoint);
10.     }
11.  
12.     public static bool IsInDirection(Vector3 direction, Vector3 targetLocalPoint)
13.     {
14.         if (Vector3.Dot(direction, targetLocalPoint) > 0) return true;
15.         return false;
16.     }
17. }

 

No, because the datatypes are exactly the same. In fact in a way it's sort of the opposite of 64-bit now, where it looks like they fixed the compiler in Unity 5 so it keeps the results consistently as 32-bit floats across the calculation, whereas previously it preferred to unnecessarily promote them to doubles. I haven't tested it but it might have a bit of a performance improvement because of that. See this code for example:

Code (csharp):

1.         Vector3 center = new Vector3(294.5f,0f,75.5f);
2.         Vector3 start = new Vector3(294.5f,0f,78.1f);
3.      
4.         float value1 = Vector3.Dot(center, center) + Vector3.Dot(start, start);
5.         float value2 = 2 * Vector3.Dot(center, start);
6.         float value3 = (.5f * .5f);
7.      
8.         float valueA = Vector3.Dot(center, center) + Vector3.Dot(start, start) - 2 * Vector3.Dot(center, start) - (.5f * .5f);
9.         float valueB = value1 - value2 - value3;
10.         Debug.Log(valueA + " " + valueB);

Run that in Unity 4 (A = 6.509992, B = 6.515625) compared to Unity 5 (A and B = 6.515625). I'd probably classify that as a bugfix; I don't think you can get the Unity 4 behavior back (but it's undesirable anyway).

--Eric

 

The code doesn't quite compile, missing ExtensionsVector3.IsInDirection. But I faked that.

I couldn't duplicate your problem at the origin. The capsule sits nicely against the cube as long as its in the sphere cast area. At about 200 units out it starts to shake. So it does look like floating point error.

What's your use case for this? Because that's a huge amount of code just to make a capsule move to a cube. But you may have a use case where all that vector gymnastics is justified?

 

Sorry, I added the IsInDirection code to my other post.

After your comment, I decided to test this all at the origin. Sure enough, at the origin, all seems to work as intended. Try moving the gameobject that has the "TestCastsProblems" script to cords 294, 0, 78 and the box to cords 294, 0, 74.
All would work well there too, except if you move either the gameobject or the cube so that their cords is not a whole value anymore.
This video shows what I see.
www.youtube.com/watch?v=b25UmHLBOZ4


Is there a way to reduce this floating point error?

The reason for all of this is to get the CapsuleCast center on collision, the "TestCastsProblems" script was just made to debug things. If you move the box, it will curve nicely around the capsule. I use this for my custom character controller collision. After I set a desired position to move, I shoot a capsulecast, and then if it collides, I just get its center on collision and move my character to that point instead of the original target point.

 

Unity, nor C#/Mono/.Net control the implementation of the floating point system used. This is often system dependent, and even hardware specific (most modern computers have a floating point unit in the processor, if not the OS will simulate it). The only consistency is that they follow the IEEE-754 standard:
http://en.wikipedia.org/wiki/IEEE_floating_point

What this means is that you may be able to expect consistency across a single system, but never expect consistency between systems.

You won't every be running Unity5 and Unity4.6 code side by side... so you'll never run into a situation where you need to compare floats from Unity5 and Unity4.6 on the same system.

Where as floats coming from other systems should NEVER be expected to be accurate. Regardless of unity version. So really, there's no point in trying to get the same exact results in Unity5 as you did in Unity 4.6.

IF for some reason you need to develop a system that is determinate like that... you should NOT be using floats. Instead you should probably go with a fixed point number system. No fixed point number data type exist in .Net/mono by default, you'll have to implement it yourself.

As for the why.

Eric5h5 already described it.

Specifically in your code, it's because you call Vector3.Dot. In Unity 4.6 this is the implementation of that method:

Code (csharp):

1.  
2.     public static float Dot(Vector3 lhs, Vector3 rhs)
3.     {
4.       return (float) ((double) lhs.x * (double) rhs.x + (double) lhs.y * (double) rhs.y + (double) lhs.z * (double) rhs.z);
5.     }
6.  

Note the case from float to double during the calculation.

In Unity5 they've since removed that cast from float to double and back. Float error is no longer introduced as a result.




IF YOU WANTED...

You could simulate unity 4.6 Dot product behaviour in Unity5 by having a version of the dot product method that operates it the same way (using doubles). And vice versa, you can make a Unity4.6 implementation of the Dot product that acts like the Unity5 system by not casting to double.

Why you would do this though? Pffff... no idea. I wouldn't even bother.

 

I'm sure there is a far simpler way to achieve this. Let me do some playing.

Some thoughts for fixing your problem in the meantime. Floating precision is mostly an issue of combining big numbers and small numbers. As indicated by @lordofduct the dot product multiples the vector components together, then adds them.

So to step through the dot product case at 1000,1000,1000

Vector3.Dot(centre,centre) = 1000 * 1000 + 1000 * 1000 + 1000 * 1000
= 30,000,000

As a rule of thumb floats are precise to 7 digits. So this is stored as 3.000,000e8

Now if you subtract some equally large float, say 30,000,014 (stored as 3.000,001e8)

30,000,000 - 30,000,004 = 3.000,000e8 - 3.000,001e8
= -10

Obviously wrong. This is the problem.

Moving to using doubles will help with the problem. There is a performance hit from casting, but its minimal. Try casting everything to a double, doing all of your calculations as doubles, then casting back to a float before returning the value. You'll need to implement your own double version of the dot product.

 

How much more would doubles help? With floats, at about 200 things fail, so with doubles at about 400 things will fail?
If that is the case, then I would need to probably redesign things to work in a more local world space so the numbers stay small.

EDIT - I changed the CapsuleCenterOnCollision method so that it does things in a local space and I think this has fixed the issue. I will be testing for bugs.
There would still be floating point error when the CapsuleCast hitInfo.Point is very far, but for my purposes this wont be an issue since collisions will be very very close.

Here is the code if you would like to test as well.

Spoiler: Click for Code

Code (CSharp):

1.     public static Vector3 CapsuleCastCenterOnCollision(Vector3 bottomPoint, Vector3 topPoint, float radius, Vector3 directionCast, Vector3 hitPoint)
2.     {
3.         Vector3 worldOrigin = ExtensionsVector3.Vector3CenterPoint(bottomPoint, topPoint);
4.         Vector3 localOrigin = Vector3.zero;
5.         Vector3 localHitPoint = hitPoint - worldOrigin;
6.  
7.         Vector3 bottomTopPointDirection = (topPoint - bottomPoint).normalized;
8.         float originHitPointDistance = Vector3.Distance(localOrigin, localHitPoint);
9.  
10.         Vector3 localEndPoint = localOrigin + (directionCast * originHitPointDistance);
11.  
12.         float originHitPointHeight = ExtensionsVector3.ProjectDistance(localOrigin, localHitPoint, bottomTopPointDirection);
13.  
14.         float height = Vector3.Distance(bottomPoint, topPoint) + (radius * 2);
15.         float heightRadiusDifference = (height / 2) - radius;
16.  
17.         bool isTop = ExtensionsVector3.IsInDirection(bottomTopPointDirection, localOrigin, localHitPoint);
18.         if(originHitPointHeight > heightRadiusDifference)
19.         {
20.             if(isTop)
21.             {
22.                 localOrigin += bottomTopPointDirection * heightRadiusDifference;
23.                 localEndPoint += bottomTopPointDirection * heightRadiusDifference;
24.             }else{
25.                 localOrigin -= bottomTopPointDirection * heightRadiusDifference;
26.                 localEndPoint -= bottomTopPointDirection * heightRadiusDifference;
27.             }
28.          
29.         }else if(isTop)
30.         {
31.             localOrigin += bottomTopPointDirection * originHitPointHeight;
32.             localEndPoint += bottomTopPointDirection * originHitPointHeight;
33.         }else{
34.             localOrigin -= bottomTopPointDirection * originHitPointHeight;
35.             localEndPoint -= bottomTopPointDirection * originHitPointHeight;
36.         }
37.  
38.         Vector3 capsuleCastCenter = ExtensionsPhysics.SphereLineIntersect(localOrigin, localEndPoint, localHitPoint, radius);
39.         capsuleCastCenter = worldOrigin + capsuleCastCenter;
40.      
41.         return capsuleCastCenter;
42.     }

 

But the perfect precision answer is 6.51 (someone check my math for me, been a while since I've done vectors by hand). So somewhere in the upgrade of the dot product mix repeatability increased, but the total error increased.

A double has twice as many digits of precision as a float. With a float you have about seven digits of precision. So at 200 out your dots come in around 4,000.000. After a subtraction down to single digits you can expect precision of 0.000. Anything further digits are meaningless.

With doubles you at 200 you get about 4,000.000,000,000. Do the subtraction and you get 0.000,000,000. Cast it back to a float and you still have all the precision a float can possibly hold. In theory this lets you get out to about 60,000 away from the origin. Unity's physics system well and truly breaks down before that point. Google the Space Kraken for some videos.

 

That would also fix the issue.

 

So rule of thumb, do all things in a local space, but if even in local space the float gets to large, move to doubles. I'm glad I upgraded to Unity 5 to find all these issues I woulda had =). Thank you all for your help!

 

Well... technically not. But for all intents and purposes you could just guess around 2 times the precision.

Rant ahead for anyone interested...



Here's the first important thing to consider. Floating point values are stored as binary, so we get X digits of precision in binary.

Single Float:

http://en.wikipedia.org/wiki/Single-precision_floating-point_format

Double Float:

http://en.wikipedia.org/wiki/Double-precision_floating-point_format

(note - these images would probably be rotated on most machines depending on format of the memory... big-endian/little-endian and whatnot).

The mantissa/fraction there is the actual sig value range. But we have to consider the fact that the mantissa actually gets 1 extra bit of precision because there is always an implied leading 1. In binary ALL non-zero values have a leading 1 digit, we can just drop it, zero is represented in sort of special case.

So a single gets 24 (23 explicit) binary digits of precision

A double gets 53 (52 explicit) binary digits of precision.

So a double technically gets 2x +5 more binary digits of precision.

To translate that to decimal sig value range isn't exactly easy... because it's not really a sig value range, but instead a value range that's dependent on the exponent of the floating point.

A single will get approximately 7 decimal sig values. Though a lot of times you can get 8 (2^25 - 1 == 33,554,431 after all), but on some weird occasions you might only get 6.

A double will result in approximately 16 to 17 decimal sig values (2^54 - 1 == 18,014,398,509,481,984). Though again there are weird moments that you get 15.

The biggest issue is when you're storing fractional values. Whole numbers convert to and from binary rather nicely. Fractions on the other hand... they can act REALLY weird. This is because your fractions are binary... each digit of the fraction is 1/(2^n) summed up. It's kind of hard to build a 10's when 10 has very few even (base 2) factors.

10 -> 1,2,5,10

It's actually why before the west finally got Arabic numerals from India (they're only called Arabic because they came through the middle east via Arab merchants), our number systems weren't often base 10.

Instead we used base 12, and base 4, and base 60 systems.

This is why our clocks are broken into 12. It's why we say "eleven" or "twelve" instead of "one teen" or "two teen". It's why we have 360 degrees on a circle (6 60's). And the Babylonians writing system for numbers was base 60.

12 -> 1,2,3,4,6,12
60 -> 1,2,3,4,5,6,10,12,15,20,30,60

You can make a lot of even fractions out of that.

Anyways, 2's to 10's, not enough fractions to build 10's out of with 2's.

What ends up happening is that numbers that only have 1 digit of precision in decimal grow into huge sig value ranges in binary.

for instance
0.1dec is 0.0001100110011001100110011001100110011...

Where other values that might be long in decimal are very short in binary relatively speaking.

0.625dec is 0.101
0.9375dec is 0.1111
0.15625dec is 0.00101 (from the image above...)

Note that usually in whole numbers the binary value is exhorbitantly larger in sig values...

255 = 11111111
63 = 11111

Guessing/anticipating fractional sig value range (and error for that matter) can be near impossible for your general programmer.

In these cases you can't really say that you get "X digits of precision", as opposed to the whole numbers. You could write a 30+ digit decimal fraction that just happens to be a perfect factor of 2 and fit in the sig value range of a single float.

For example (2^24 -1) / 2^24 ~= 0.999999940395355224609375...(some more digits I don't know) which would fit perfectly in the 24 digits of sig value range in a single floating point (every mantissa value is 1).

Where as another time you could write a single digit precision and it results in an infinite series in binary (like 0.1) and you get NO precision whatsoever.

 

Yeah, I kind of skipped over the binary stuff. The base 10 approximations are in the right ball park though.

The absolute precision (error? I feel like I may be mixing terms here) of a float isn't that hard to calculate. You simply convert to a base ten number, flip the last bit of the mantissa, convert to base ten again, then subtract the two base ten numbers from each other. The result is the maximum error.

Most floating point issues are not due to this error, after all, its typically a very tiny fraction of the original number. The problem comes because of the property of errors with arithmetic. If you add or subtract two numbers you add their errors. With floats the error becomes larger as the number becomes larger. So you add a relatively large number to a relatively small number, and the small number can disappear entirely into the error of the large number.

Sorry if my terms are coming across mixed. I'm from a chemical engineering background, rather then a computer science background. It also means I may have things completely wrong. But I'm reasonably confidentish.

 

Hey everybody. I just found out that there's a difference between how my "LevelGeneration" is working in the Unity Editor(64bit) and in a Standalone or Android Version. Does that mean that the behaviour in a 64bit - IOS Version will be different from the Android Version? That's would be bad, because I have to rely on the fact that a Level generated with a "Seed" will be the same everywhere.

So what can I do to make sure the results the same everywere?
Btw, if I make a 64bit Standalone Build the LevelGeneration is the same as in the Editor.

 

Its probably not floating point precision. This will be due to different implementations of random on the different platforms. Building your own random number generator should solve the problem

 

Ok thanks. I will look into that! I was using Mathf.PerlinNoise until now. Will try to find more informaton about this. Sorry if this was unrelated.