Showcase Floating point Inaccuracy in a surprise location

It is found Inside the mesh of the unity editors plane object, all values but 1 return whole That is index 6,

A nice bit of a find here if I say so myself. What we do here is get the component mesh filters plane. Get vertices, and then iterate through the verts and ask to list the different X values of their positions and sort that list. The result is always 0.9999999 so it has something to do with the model itself.

I suppose it could be considered a bug.

 

Nice catch, but here's a fun fact: if this was in meters, this discrepancy would be 10 times smaller than the width of the most common bacteria (1-2 microns). I'm sure Unity team has better things to do

 

Please educate yourself on floating point arithmetics: https://0.30000000000000004.com/

 

Actually, integer 1 is perfectly representable with IEEE-754 as it is equivalent to 0x3f800000 in memory.
This issue has nothing to do with the fp arithmetics, and is indeed wrongly set in the mesh itself, but as I said, this is not to be seen as error, though it does offend OCD which is common among devs.

More info:
The eponymous number of that URL is represented with 0x3e99999a * and you can see that the repeating 9 gets abruptly truncated in the end, and this is where that rounding error comes from. 0.3 is simply irrational in this format, and can't be sanitized in a way that is compatible with the decimal system without truncation or rounding off. This is not the case with 1.

* edit: btw I'm not even sure how is that site getting that value, because it's very easy to verify that .1f + .2f is approximately 0.300000011921 (or 0x3e99999a) in C#.

Code (csharp):

1. Console.WriteLine($"{.2f + .1f:F16}"); // 0.300000011921

And even if you get 0.300000041723 (0x3e99999b) that's still far less zeros in between.
(This is in agreement with multiple 32-bit IEEE-754 sites I've tested.)

 

That page assumes double precision floats which is often the standard in many programming languages like Javascript and even in C#. The value

0.1

is a double precision float. See this site. Decreasing the mantissa by 1 (changing the least significant bit) would bring you below 0.3 and the next larger value is

0.300000000000000044409

.

Have a look at the following values. Those are the next larger and the next lower values of 0.1, 0.2 and 0.3. The issue with this classic FP arithmetic issue is that the closest number to 0.1 is slightly over 0.1. The same is true for 0.2. The error is smallest when we pick the slightly larger values. Even though 0.2999999.... would be closer to 3 that the result, adding 0.1 and 0.2 them would give you the number

0.300000000000000016653

that's slightly closer to 0.300000000000000044409. Actually just by a single bit closer ^^. That's why the result is not the optimal approximation of 0.3

Code (CSharp):

1. // 0.1
2. 0.100000000000000005551  // closest value
3. 0.0999999999999999916733
4.  
5. // 0.2
6. 0.200000000000000011102 // closest value
7. 0.199999999999999983347
8.  
9. // 0.3
10. 0.300000000000000044409
11. 0.299999999999999988898 // closest value
12.  

 

@Bunny83 I considered it was a double, however the examples (on that site) clearly show single precision. Originally, I wanted to make my point across that it was a double, but then I also couldn't recreate the value at all. Probably I did something wrong, but hey, at least the value is explained (it definitely looks like a double precision error).