Burst compiler: deterministic floats

I was just reading the ECS docs and noticed that the Burst compiler can create deterministic code. Would this also apply to engine code e.g Vector3, path finding, physics? I’d like to try making an RTS game and it sounds like networking with deterministic lock step simulation is currently being worked on.

 

I love the ambition to have deterministic floating point operations on all platforms. However as this seems to be quite some time in the Future, I think it would be great if Unity.Math could include some fixedpoint types in the mean time. I know that there are lots of fixedpoint libraries for c# out there but the beauty of having it integrated would be that it could have 100% consistent naming and methods with float,float2,float3,float4 and once the support for deterministic floats is there it would be super easily refactorable.from fixed to float.

 

@PaulNK When they announce that their floats are deterministic, this should automatically include structs like Vector2, Vector3, etc, as these are just using floats.

@AurelWu I totally agree with you. I would love to see a standard fixed point math library for c#. I read into this topic once for quite a while, and I came to the conclusion that implementing a fixed point math library is the safest way in terms of determinism. 1+1 will always be 2. period. It's not just 100% safe, it's very fast too, including the conversion from fixed to float.

I am curious how unity wants to achive floating point determinism. It's a heavilly discussed topic, and there are lots of problems to be solved / factors to be terminated. Cpu architecture, operating system, compiler settings. I'm not a pro on this topic but I think they will do something like the IEEE 754 strict mode for the compiler (which will bring a significant performance drawback for all float calculations) but even with this, there is a chance that eventual thirdparty libraries / assemblies which do not implement that setting can blow the whole thing up.

As I understand, a 100% guarantee is not possible, depending on your usecase. For example If you stick to Windows 10 on 64bit only, it's very unlikley that you ever run into problems with float determinism, even with different cpu architectures, but still no guarantee. There are some games out there which rely on floating point determinsim but you have to be very careful if you do so.

Regarding lockstep simulation, I would run that whole thing in a separate c# assembly (and thread too) anyway, because you accidentally could bypass the lockstep command wrapper and access / manipulate values directly which will lead to async sim states. You won't be able to benefit from unity libraries like the new job system, at least not in the simuation (still in the game itself) but that shouldn't matter at all. The simulation runs in a slower tickrate and if you are going the datadriven design all the iterations will be very fast due to hardware prefetching. I also like the clean separation of 3D engine and simulation, you will never mix up things and it stays very maintainable.

 

Greetings, I am researching about Determinism in Unity, and came across this topic.

We are creating an RTS based on what is available to us back then.
Starting from Fixed Point based float, vector2/3, maths, and physics.

We need a clear yes/no to decide on our next steps.

1. Will burst compiler provide cross platform determinism?
2. Will we be able to enjoy the benefits of ECS, Jobs, and Burst with our own custom fixed point library?
3. When is ECS planned to ready for production? Will it be deterministic when ECS is out of preview?
4. Is the new DOTS Unity Physics deterministic?

Thank you!

 

Thank you for the clear answer, it definitely helped us decide how to proceed our next steps.

DOTS seems to be everything we are looking for, given all the benefits of safe multi threading and Burst compiler level optimization, it is unfortunate that we already have invested a lot in the foundation code built for determinism from scratch. It became pointless to have a deterministic library when everything built with Burst will already be deterministic.

However, with these news, I can't help but look forward to the future of Unity with passion, I wish you all a good future ahead!

Thank you.

 

I guess it depends on what you are simulating. If you really need deterministic / accurate / complex 3D physics, I think you should dare the switch to floats.

Despite mass unit / lockstep sims, you will need cross-platform determinism rather for stuff like replay-features in racing games, for example. As for mass unit / lockstep sims, depending on your maximum unit count to be simulated, I would do custom 2D based / faked physics anyway (collision detection, steering, pathfinding based on 2D maps, etc), as doing this with 3D methods will cost you much more performance. But if you really need accurate 3D physics in your mass unit / lockstep sim for whatever reason, then floats using unity is the solution.

I also would wait for the final solution of unitys float determinism, I guess there will be a performance drawback (even if it's very fast with all the nice unity performance features) when running the burst method in deterministic mode (at least I guess there will be compiler options), so again, depending on your needs, your fixed point math solution (using burst, too, of course) may be even faster. Also it's not clear yet what operations exactly will be deterministc.

Remember, only your simulation needs to be deterministic, not the render stuff, at least if you are not creating some kind of scientific simulation where the visual stuff matters too.

 

Hi,

Now I'm in a similar situation of @Gekigengar and I'm wondering if the cross platform floating point determinism of the burst compiler is now ready? if it's not yet ready, approximately when it should be ready? and, what changes are necessary to be able to use a fixed point math library in the DOTS workflow to achieve determinism?

Any help will be really helpful
Thanks!

 

Coincidentally I spent the end of last week working on a fixed point library for something, you're welcome to have a look and use it as a starting point.

https://gitlab.com/tertle/com.bovinelabs.fix32

• 32 bit
  
• works with burst
• it's far from complete, wasn't intended for the public (at least yet)
  • no documentation
    
  • it's only got methods i needed implemented atm (no sqrt, sin/cos/tan yet)
• based off the popular fix64 library https://github.com/asik/FixedMath.Net
• you can change number of decimal bits and it should work fine
  • public const int DecimalBits = 12 in fix.cs
    
  • note as it's only 32bit decimal places are limited. the default 12 only leaves a range of +- 500k
• includes a fix2 (float2) and a mathfix with a few methods
  
• unlike the library it's based off, will throw exceptions if an overflow occurs (rather than return an overflow value)
• includes tests
• can turn off overflow checks with FIXMATH_NO_OVERFLOW
• can turn off rounding with FIXMATH_NO_ROUNDING

 

@Joachim_Ante: do you know if there is any plan to develop a new DOTS-based version of Unity's 2D Physics engine as well? Thank you

 

Is there any progress on this? It's been 2 years since this was proposed and I haven't found any other updates on this feature.

 

It was discussed on GitHub recently, see this issue.

 

I'm starting to think fixed point is the the way to go more these days when determinism is a requirement. I think Unity could maybe get deterministic floats eventually (although based on recent posts it doesn't sound like it's a focus anymore?) but I can see it being dropped in later years due to continued dev maintenance cost & keeping up with evolving hardware. Or maybe I'm wrong and someone knows better?

 

Haha, I was the one who filed that issue. Commenting here for a bit more visibility.

 

For those who are looking for a working solution without needing compiler support one of my fellow devs found this repo https://github.com/RomanZhu/FixedPoint-Sharp, which supports a Unity.Mathematics style use of Q48.16 fixed point values (fp, fp2, fp3, fp4),seems to be well tested, and seems to be optimized for performance (plenty of manual inlining and other micro-optimizations). It however currently lacks an equivalent Matrix4x4 definition and related operations and obviously doesn't have Burst compiler support (no SIMD optimizations).

This does however mean needing to build atop of it the other related systems sometimes needed in deterministic simulations (physics, animation ,etc) and would require bifurcating existing systems to support it.

 

Thanks for sharing @james7132. That may come in use in a year or so if there's still no Unity mathematical determinism around.

 

I just read the Unity 2021 Official Roadmap; unfortunately, there's no reference to DOTS cross-platform floating point determinism.

Is this feature eventually abandoned?

 

That's excellent news, thanks!

 

Cant wait to see a cross platform deterministic compiler even it is a very early access

 

It's been over 3 years since the start of this post, and still no sign of this feature landing in any near or even far future of Burst.

Things outside of Unity has also moved quite a bit since then. There is now a Rust-based physics engine called Rapier (https://rapier.rs/) that purports to provide deterministic results on all platforms that are IEEE 754-2008 compliant (read: pretty much every desktop processor since the early 2010s). A redditor dug into it's implementation (https://www.reddit.com/r/Unity3D/comments/lkxb9d/crossplatform_deterministic_physics_with_unity/) and found that the use of Rust's port of musl C's libm provided in-software floating point operations that are guaranteed to be IEEE 754 compliant in a cross-platform way. They then took the existing physics package and made it work across a desktop Editor, WebGL, and Android.

This of course incurs a 4x performance loss due to the failure to vectorize and other compiler optimizations, about on par with using fixed point calculations, but it allows for operating on a wide range of scales and retains binary compatibility with other floating point computations and thus does not incur conversion costs when entering and exiting deterministic code. This performance loss is unavoidable for cross platform determinism, as SIMD instructions hardware implementations may not be identical (Intel vs AMD) nor IEEE 754 compliant.

Even if it may not be possible for all of Unity's supported platforms, it'd be nice to at least have FloatMode.Deterministic just substitute all floating point operations with these "soft floats" instead. Since Burst requires going through an IL2CPP step, it seems trivial to introduce this step: create a C++ type for soft floats and just use it in place of float during C++ codegen. Perhaps fail compilation when the target platform is not IEEE 754-2008 compliant or add a runtime initialization check for it.

Additionally, a define flag for changing the default FloatMode.Default to Deterministic instead of Strict may be needed as developers will need to fork existing packages just to update the FloatModes on system jobs (as the redditor above did) to get the desired determinism.

 

Thanks for the swift response!

Non-determinism as a result of multithreaded code or use of non-determinsitic data structures is a given, but developers can work around that, but not the compilation results provided by Burst. Getting some version of FloatMode.Determinstic out would already make a world of difference for those who were developing or experimenting with Burst with the expectation that the feature would land soon. I certainly have developed everything else needed short of actually having determinsitic floats while waiting for the feature to land. If it takes another year or three to have this feature land, I think the time would have been better spent using a different engine or developing my own from scratch.

 

First of all thanks for the update on the current status and priorities, it's incredibly useful to know what's happening currently and in the near future!

I'm basically just commenting to hopefully bump up floating point determinism a tiny notch on the priorities list since we're also waiting for it and it's pretty important to us.
We have a bit of experience with determinism in Unity due to the lockstep multiplayer mode in our game Parkitect, and as james7132 says non-determinism in some parts is usually not a big problem and can be worked around.
Having just deterministic floats alone would be huge already.

 

I typed out a response and deleted already but what the hell, here's the jist of it again.

+1 What James7132 said

This thread is about float determinism in burst, not about all of Unity engine being deterministic. Games have been launched and developed before in Unity working around the other issues you mentioned using fixed point. The issues you mentioned sound like a different topic to the thread title, unless I'm ignorant of something here?

That said I'm glad you are prioritising debugging and iteration time as these are the biggest pain points when using Unity for me currently. I've already switched to fixed point maths & physics and it's actually nicer in a way as I have more control of things now.

Appreciate you responding with an update, xoofx.

 

What James7132 +1000 I can work around any other level of non-determinism. I am in the middle of developing a large scale RTS. I can even write my own physics but I really need floating point determinism.

Fixed point is kind of meh trigonometric functions suffer. When I started working with ECS this feature seemed imminent. Now I have this bad feeling that floating point determinism may never happen.

 

From what I've seen of most fixed libraries, they just precompute all values up to a certain precision to avoid this issue. Think the issue I had with a fixed library was Sqrt.

 

I have had problems with trig functions being jagged enough that stuff does not look smooth.

 

For those interested, I have been experimenting with offloading computations to a compiled Rust binary, since Rust's f32 type is guaranteed to be IEEE-754 2008 compliant on all platforms, utilizing nalgebra and libm to cover operations not covered by the spec.

I'm getting bitwise identical results on every platform that I'm testing on so far (just Windows, Mac, and Linux right now), though I have yet to try to integrate this with a Burst compiled job, and I'm not sure how performant it is given the potential FFI interface it has to go through.

With this I might move towards just using Unity only as a rendering/audio/authoring tool, and just move all game simulation logic and state to a compiled Rust plugin, or just move off of Unity entirely.

For those looking for a pure C# solution, the aforementioned soft-floats should work, though you might need to fork all of Unity's packages and just replace everything with sfloat.

 

Maybe this can be somehow helpful to the Unity team to implement determinism in DOTS (Physics), or to other devs looking for an alternative:

https://www.reddit.com/r/Unity3D/co...hysics_with_unity/?utm_source=amp&utm_medium=

 

Software floats would be great. I am not sure if they plan to go that way or not.

 

When I was looking for an alternative to float for determinism, I saw this post among others. As far as I remember, it has a repository of reworked Unity physics? If this is so and I am not confusing anything (because about six months ago I tried several similar solutions), then when tested in the editor, this library was 2-20 times slower (with different operations) than float. And that's acceptable, I think determinism is worth it.

However, the build profiling showed the following:
Common operations (+, -, etc.) are 2-10 times slower than using float
Some operations, such as square root, are several thousand times slower than float.
These numbers are roughly the same (or worse) for all the libraries I've tested.

Vector normalization is a common operation that uses square root (or inverse square root, but that seems to be outside the scope of such libraries). Slowing down this operation ten thousand times is too significant.

These are the results of tests half a year ago, everything could have changed, and I could be wrong somewhere.

 

Thank you so much for sharing your experience! I totally agree with what you said.
Only thing I hope is that there could be a significant margin of improvement, especially because (IIRC) some operations (IE: division) had been implemented by the aforementioned developer himself, and it was his first experience with such things.

 

If software floats are ten thousand times slower they are not viable. Determinism has been done other ways for a long time.

 

I have tested the latest ECS build with burstcompiler, surprisingly, it is very different from my previous test back in 2018. Currently, I were able to get same results test via Arm64 and X86-64 architecture through IL2CPP burst compiler, PC editor will skip il2cpp using mono burst compiler, thus still have different results. What is the exact status on this, is there any loop hole that I might run into? Also what if I had a json file with bunch of floats, then read those floats into ECS during runtime and using those floats modify physics force. Would this affect determinstic result?

 

Hello, as of today, I see dots physics 1.0 is released. Is there any updates regarding deterministic physics in unity generally?

version 1.0 documentation mentions determinism everywhere, which is great

 

There is a cross-platform physical engine library，it's stable
https://github.com/sam-vdp/bepuphysics1int

 

Looks dead last commit was on Mar 1, 2018. Fixed points are a bad solution you only see appearing in C# code bases.

 

Can you elaborate? What to you suggest instead for deterministic math?

 

What I am attempting to convey is that in C++ or Rust people use deterministic floating points, not fixed points. It is only in C# where we see a return to fixed points. Because of the limits of managed code. Bust is native code so we should not have to rely on fixed points.

Fixed points bring tons of issues that are very hard to solve.

 

I see, thanks for clarifying.

Correct me if I'm wrong but wouldn't that mean that, unless 100% of gameplay code can be bursted, fixed point is still required for determinism in c#?

 

Yes your sim code needs to be bursted. Nonsynchronous code does not have to be. If you are using ECS physics it should not be a big issue.