Vector3 and other structs optimization of operators

No, you were not harsh at all. I am happy we had this discussion.

So on my side, as a conclusion, I get that the IL based micro-optimizations in FRB do not translate to the c++ world, probably because of the differences between .Net's JIT and the c++ compilers. My limited knowledge about compilers, and unwillingness to spend a huge amount of time learning about them right now, makes me accept that FRB should for now stay compatible only with Mono builds. Would happily follow any findings you make about this subject.

 

"Burst" il2cpp makes sense as it's easier to optimize that than C++ compiler. It goes beyond simple math operations but also about what I've written about in Unity.Mathematics thread (and what's written on https://jacksondunstan.com/?s=il2cpp). I wish there was a callback to do it ourselves, something like "IIl2CppProcessor.OnAfterConvertRun" as "before" version already exists https://docs.unity3d.com/ScriptReference/Build.IIl2CppProcessor.OnBeforeConvertRun.html. That would allow us to also quickly check if C++ compiler is doing proper optimizations. Only way of doing it I've found is to make 2 builds, one with "Create Visual Studio Solution" then modify C++, build and combine with first one.

 

The main issue we have is mono anyway, everything is a function call. and it just adds up.

Another issue is that mathf is having a lot of function calls aswell, and nothing is being inlined, so it's hell.
I recently pointed this out here: where I use as example a raycast.
https://forum.unity.com/threads/is-using-burst-as-simple-as-checking-a-box.899684/#post-5915207

Another point is that I made a copy of all the mathf class and manually inlined every function, and it gave a 4x speedup average on my math heavy stuff. that is significant.

Because mathf is implemented in such a way that a single math call could mean clamping, adding and multiplying vectors, doing calls to mathf other functions that in turn call other functions. it's just sad really. I doubt even il2cpp inlines it because its all without any aggressive inlining markers.

This could be easy solved by unity, just fix up the mathf class, would take 1 day work. and benefit everybody using it, which is well.. everyone.

 

Was it in editor or in a Mono build?

 

Yes, almost. I guess it may not be the case for mono.

It simply would search and replace generated c++ before "c++ to assembly" compilation. For ex. when I've found that Unity.Mathematics.math.sin is ~15 times slower than Math.Sin in il2cpp:

I could simply regex replace:

Code (CSharp):

1. math_sin_m7CB1E8DF153CC3E22A6C366F3F5F98571CF77BA9((2.435f), /*hidden argument*/NULL);

to

Code (CSharp):

1. sinf(2.345f)

I guess could also remove all "IL2CPP_RUNTIME_CLASS_INIT(Mathf_tFBDE6467D269BFE410605C7D806FD9991D4A89CB_il2cpp_TypeInfo_var);" to make Mathf faster.

 

@JoshPeterson if I understand it correctly, was the point of the entire previous discussion simply finding out whether a "constructor is more expensive for C++ compared to creating a struct directly"? I mean wouldn't this be a huge issue of the IL2CPP due to the fact that users often use their own structs even if they don't use the Vector3 library? I don't really understand why this issue could stay ignored anymore as low proprity if it's so straightforward that everyone who inspects the code sees the issue immediately. As was also mentioned here, Editor performance matters too, and the Unity profiler now displays the performance loss (which is sometimes arguably even worse than a negligible temporary assembler performance issue that'll get fixed on its own anyway, if it's guaranteed to cost every single new Unity developer at least few hours of their time, because everyone profiling vectors will eventually find out about this).

Regarding the point number 2: "Can the Vector3 C# implementation be improved to provide better performance?", could you please also try to address the following points I previously mentioned?

* Vector3.Distance internally uses the expensive constructor (due to C# struct behavior)
* Operators like - also use the expensive constructor, and they are further required by methods like "Vector3.SqrMagnitude(b - a)". Also is there a reason why it would be hard to add a new high-performance overload with parameters that cover the most popular use-case, in a way such that the user doesn't have to do the "square" operation manually, making the code harder to read and more error-prone if expressions are complex?
* Regarding function call, C# now finally supports an "in" (readonly ref) modifier of the parameter, such as "in Vector3 a" instead of "Vector3 a", and it seems to reduce the function call overhead by 10% - 50%, as the struct isn't unnecessarily copied. Is there any reason why all the similar math-heavy operations cannot simply utilize this easy improvement? Shouldn't passing a reference be an obvious path forwrad in all performance-critical parts of the code, providing additional free safety and self-documentation about the intention? There may also be more useful features of the latest C# version.
* The well-known micro-optimization of replacing Distance by "SqrMagnitude(b - a) > greaterThan * greaterThan" could benefit from all of the previous improvements, but on top of that I'd like to ask, is there a reason why the more direct expression like "Distance(a, b) > greaterThan" cannot be also automatically compiled to the Sqr variant, so that people can prefer a more declarative approach? For example, I was already met with a backlash from Mirror developers regarding this micro-optimization in their code, as they already had a negative experience, and now they prefer "clean code", while this definitely looks like a hack < https://github.com/vis2k/Mirror/pull/1514#discussion_r384335611 >. So as long as there are similar deterministic use-cases (for example where the partial results aren't being passed around anywhere in the code), is there a reason why Unity doesn't try to automatically replace the well known patterns within IL2CPP? This may be also related to things like LINQ and foreach, which for some reason still provide less performance than their other equivalent. And let's not forget about stuff like constant strings, which could be also easily replaced by their reference, yet this isn't happening either and the objects are being created anew. If gaining performance like this isn't a priority of Unity whatsoever, could you please at least point me to some alternative option, like an example of how to extend the IL2CPP myself? I'm sure the future of C# lies in more intelligent compilers

 

This should be simplified but I see Unity doesn't want us to write C++. I guess we could make il2cpp.exe wrapper to add "IIl2CppProcessor.OnAfterConvertRun" callback ourselves.

 

Quote of the day

 

That's fine for me, let us (and plugins like Frame Rate Booster) take care about changes. Being able to inject custom c++ easily is very powerful in terms of performance. I'd rather have this option than wait for Unity to fix il2cpp, issue with static constructors was marked as "Postponed" almost a year ago. https://issuetracker.unity3d.com/is...tialized-and-il2cpp-runtime-class-init-checks I guess it wouldn't be fixed for a long time if not posts in Unity.Mathmetics thread.

 

@JoshPeterson would it be possible to move all Vector classes into a package that gets installed via the package manager? or would this not be possible due to some internal optimizations?

I'd really like to see some improvements to some of the operator methods as mentioned in this thread but there's most likely a breaking change when using precompiled third-party code. The way I see it moving this to the package manager would allow switching to an updated version if the project permits it. More details here.

 

@Hurri04 You could attempt to rewrite the class using Cecil too!

 

I'm pretty sure that this would be against Unity's TOS. And even if it wasn't, the breaking change I mentioned would cause further errors anywhere an implicit V3->V2 cast operator is used in the internal Editor UI code. Changing these to explicit cast operators is something a Unity dev has to do.

The crux of my issue with the V3->V2 cast operator is that it really shouldn't be implicit but fixing this means a breaking change. The way I see it the package manager presents the perfect solution with the possibility to replace even very core systems which were kept from being updated (to keep sustaining support for legacy systems) by offering newer package versions to the users.

The main things I'm concerned about with this approach (because I don't have deep enough insight into these parts) are these:

• Are there any conversions, optimizations, or other interactions happening between C# side and C++ side which would prevent moving the code to a package?

• And is the assembly/package available (can it be loaded) before it is needed by the internal Editor UI code when starting up Unity? What happens when the package manager is used to change the package version to a newer/older one? Can the assembly simply be reloaded in place of the old one? Or would it require restarting Unity?

Other than the breaking change I described I'm not really seeing any further problems arising on the user side, since the other suggested changes are essentially "just" syntactic sugar (but even so, I think they'd be really useful to modernise this part of the engine, comparable to e.g. the new responsive Editor UI design).

 

No its not against their TOS at all to change code. Other plugins do this, they do this and we have done this too.

 

https://unity3d.com/legal/terms-of-service/software
My understanding from 2.6 (General Restrictions) is that this might not be allowed because of this being internal code?

 

Hmm, hooking up a thing to prevent Vector3/2 implicit casts would get rid of a lot of bugs.

Do you know if Cecil allows us to remove things? Like the mentioned cast definition?

 

I'm not sure, you might ask JB Evain who made it and also worked with Unity.

 

From the reddit AMA (I think I also read about this in a Unity blog post but can't find it anymore right now):

This seems to fit exactly what I described. So if Vectors could be turned into such a core package (with the option to upgrade to a newer/preview version with the suggested changes) that'd be really awesome!

 

Haven't had time to download and test it yet myself but in today's blog post about the new 2021.2 beta it says:

Does this include anything from this thread?

 

Excellent! I came searching for similar information about the beta's notes thanks all.

 

Hi @JoshPeterson

Thanks for having considered the optimization discussed in this thread to be part of Unity. That's what I wanted from the start, but never managed to convince Unity to do so, until your intervention.

Maybe I misunderstood something, but it doesn't seem like the inlining was included.
I downloaded 2021.2.0b1, then opened Unity\Hub\Editor\2021.2.0b1\Editor\Data\Managed\UnityEngine.dll using ILSpy, then went to the Vector3 multiplication operator, and saw no change. As reference, here is how it should look like:
https://forum.unity.com/threads/vector3-and-other-structs-optimization-of-operators.477338/

Can you clarify the situation?
Thanks

 

Any updates over here?

 

None from me. I haven't checked the latest beta though.

 

2021.2b11 it's still using

new Vector3

instead of modification of the vector. I also don't see any aggressive inlining attributes added, but maybe they've made that change in their compiler somewhere else.

 

Any chance to backport these optimizations to e.g. 2020.3. LTS? Would surely be very helpful in the VR space with such high frame rates and almost zero tolerance.

 

I completely understand about introducing bugs.

I don't have any benchmarks but in VR every microsecond counts! It's not like with regular displays, where a bit of lag or low FPS can be ignored - in VR your brain literally thinks you are poisoned and makes you want to vomit or gives you a headache if your game/app can't consistently render @ 72/90/120/144Hz (on both eyes), granted some people are less sensitive, but personally whenever a game stutters even slightly I need to take a break, sometimes for the rest of the day.

I will try to get some benchmarks using the OVR Metrics Tool i.e. outside of the Editor, though I've never done benchmarking before so I'll have to read up first.

 

Which parts of Unity tend to be a bit slower for convenience ?

 

Hi

First, my own experience: As stated here, "On my other asset, Curvy Splines, I got also a 10% increase for operations like mesh generation and splines cache building."
Then, the experience of users of Frame Rate Booster:

• In this twitter thread, @dnlbrgs speaks about the impact of said optimizations on his game Pawarumi. From a discussion I had with him at the time of that tweet, the guy seemed nice and available to talk. So maybe contacting him will bring you the real life information you are looking for.
• In the reviews page of FRB, there are some people who expressed the asset bring big improvements for their projects. Maybe asking them can be helpful. The one review I saw that gave actual numbers said: "Got a solid + 10-20% FPS depending on areas"

For this optimization, I really don't see how it can go wrong, as long as basic non regression tests are done to make sure that the new implementations actually do the needed operations. I hope I am not saying this out of ignorance of how wrong such a simple and isolated change can go wrong

 

I get your point, which I find fair enough.

 

I absolutely don't see a way I'd be using high performance DOTS any time soon, so improvements to the performance of current Unity are a big deal for me! I'd happy take a few float rounding changes. Heck, quite used to it as a game dev choosing lower precisions.

 

Indeed! I see performance upgrades as actual *features* that deserve top billing in release notes, as should anyone really so long as it's not a regression fix.

 

But that behavior was exclusive to mono, the conversion to 64-bits did not happen in IL2CPP builds. This meams that behavior was actually inconsistent between the editor and the vast majority of build targets, where IL2CPP is the only available runtime.

 

I think this would be a great opportunity* to revisit the idea of putting Vectors into a package so these changes could be released as a new version, giving users the freedom to test if it works for them or going back to the previous version if it does not, while also giving you a lot more feedback.

In the spirit of "Extension Everything" for C# I'd like to see something akin to "Package Everything" for Unity, making the engine even more modular and allowing to replace these modules with different versions.

One huge step in that direction would be the possibility to have git dependencies in custom packages imo, since this would allow for powerful dependency-chaining. Sadly that topic also has gotten a bit stale over there.

*) For future versions. I agree that it's probably not a good idea to backport such a new feature into an LTS version now. But it would've been nice to have, for exactly this use case

 

Is this improvement also inlining our custom functions that are operating only with Mathf and vectors ?
And similar functions but in nested chain ?