Unity Future .NET Development Status

I wasn't suggesting we ditch the C# for C++ but having both would be nice. Godot for example has a variety of choices.

 

Accessing and moving things around between C++ land and C# land so it can be consumed by user code has a lot of considerations and even some non trivial performance issues. That's one of the reasons behind the effort to "bring" part of the engine from C++ to C# (along with ease of use ofc).

Supporting multiple languages directly in an engine is usually a poor idea that ends with divided engine resources, incompatibilities, feature disparity, divided documentation, etc, etc, etc. As for C++ in particular, you can write native plugins and just add the dll to unity. Ofc u will need to write C# wrappers but code generators should help make it semi automatically (?)

 

We just going to pretend this doesn't already happen in Unity often more frequently than in UE and Godot ? lol

 

The issue here is that IL2CPP is pretty much terrible at generating C++, it generates code so bad that even LLVM gives up on optimizing it from what I've seen. So in fact, when we get CoreCLR, it'll most likely have much better performance than IL2CPP.

 

I use IL2CPP so people don't easily dotpeek all my source code. Even if some one very determined can with IL2CPP it is a lot of effort that most won't try.

 

That's add a whole lot far more amount of work to do for unity

 

Last year, I adopted and updated a benchmarking suite which compares the performance of Unity's various compilation methods against C++ and CoreCLR in various benchmarks. You can find it here: https://github.com/XJDHDR/BurstBenchmarks/tree/master/benchmark_results_2022-10-20

My opinion overall is that, while there are 1 or 2 scenarios where C++ is beneficial, CoreCLR for the most part is either on par with C++ or not much worse. And for performance critical scenarios where you do need more, Burst and C++ DLLs can give you that.
And I was disappointed with IL2CPP. There were 2 benchmarks where it didn't appear to compile properly, and only showed a consistent advantage against Mono in the other cases.

 

Many tools/modules are only available in Unity as C#, e.g. ECS, Cinemachine, HDRP, URP, practically almost all packets installed via the packet manager. Either Unity would have to implement everything again in C++, or doing without access to these modules in C++ scripting.

 

I think it was also mentioned several times by Unity staff that IL2CPP was never intended for performance, only for platforms that don't allow JIT apps.
I recently heard somewhere in a video that even with NativeAOT there is no increase in runtime performance, but in some cases it can even be worse than the JIT version. However, the startup time is accelerated enormously and the build size is extremely reduced. Use case here are cloud applications that are only started when needed.

 

I know I shouldn’t respond to this… ah what the heck.

I have actually been quite impressed with Unity’s ability to coordinate a bunch of different teams into consistently designing component based packages with well written documentation. It’s no accident, you can find their style guide online, as well as a documentation on how to document packages and they’re pretty in-depth.

By its component based design most packages already work together, and if they don’t it’s not hard to read the documentation and add the functionality you’re looking for. The urp and hdrp are unfortunate… everything else I never had a problem feature parity wise.

Then there’s performance but I’m not fighting you on that, that’s been discussed enough.

 

There was a time when Unity supported javascript (unityscript) and python (Boo). Supporting multiple languages was apparently a lot of work on the devs and wasn't much benefit. By dropping those, the devs were able to spend more time on more valuable features and drop some tech debt.

I seriously doubt Unity would go back to supporting multiple languages given that history.

 

I have started to passive-aggressively write "//Dang you Unity!" at ever spot in my code that I am having to do unity specific C# things because it is so outdated now... there are a lot of those in my code.

 

Moving more code from C++ to C# can actually improve overall system performance by reducing the number of hops from managed to native code. Better inlining, less marshaling.

 

This might be theoretically true. But won't pan out that way.

 

Have you read any of the The New Input System documentation? It's exactly as poorly written as the schemes of usage were designed. The Animator Controller's documentation is legendarily poorly written. The docs on just about everything to do with UGUI are horrendous. Nothing to do with the TextMeshPro documentation got much ported or updated from the original website of the founders. The wordings for the low level stuff are so obtuse as to defy maze like thinking about poor writing, and most of the features aren't documented, at all.

And that's before we get to the fragmentation that's going on with three UI systems, three animation systems, two input systems, no certain lighting systems, three rendering systems, two post processing systems, package versioning stepping and aliasing, there not being change logs in the package manager, etc etc.

There doesn't seem to be someone in charge of how Unity should come to be, a product manager or CTO with his hands on the wheel, and there's nothing like that for individual packages in terms of how they're communicated or feature developed. It's sort of "ohhh... we should do this now... maybe..." and then we, the users, wait.

By now, given the 5+ years to get close to having ECS and DOTS as a whole being seemingly another 5 years off, it might be time to consider open sourcing everything and figuring out how to let users see if they can wrangle performance out of Unity, by themselves, or dropping source access pricing to something attainable by individuals and small teams.

 

This does actually pan out and is covered in many of the perf blog posts we (the .NET team) release each year as the source of multiple performance gains.

C#/.NET code is not somehow inherently slower or worse performing. There are cases where the .NET JIT/AOT produce better code than the C++ compilers and there are inverse cases as well. They end up balancing out for the most part with each having their own strengths and weaknesses in various workloads.

There are many factors that contribute to this including not only reduced number of GC transitions, but also in being able to light up newer instruction sets that better take advantage of the underlying hardware.

A typically C/C++ app, by default, targets the same 2004 baseline that is the original x64 CPU. It has SSE/SSE2 and the baseline instruction set. Some operating systems require a couple additional instructions like CMPXCHG16B or other systems oriented instructions like LAHF/SAHF.

This, by default, excludes the ability to use SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, FMA, BMI1, BMI2, and all of AVX512, among others. C/C++ can of course still use those, but they require an actual check to occur at runtime in the main execution path of the code or another form of dynamic dispatch. This ultimately impacts binary size, the ease of which you can deploy your binaries (they are no longer simply portable). It can also impact perf due to an inability to inline across various compilation boundaries and the cost of otherwise hoisting those checks to take advantage of one off instructions, particularly like `shlx`, `shrx`, `mulx`, and others which are more flexible and faster versions of their baseline counterparts.

 

Unity has always been pretty clear on which packages they recommend. And besides, between fragmentation and backwards compatibility I feel they make the right choice replacing features one by one.

I have been very happy with the documentation but that’s subjective.

I DO agree that the Input System and anything it touches is a complete and utter mess, it needs a complete uproots rewrite but their github only shows they’re working on ANOTHER method of implementation

 

I'm not arguing with the theory, nor the work that's going into modern C#, nor the power and precision and performance of low level C#, which is (quite frankly) amazing. And I love it. But we're talking about how Unity would use these features, or obscure them, in their goals. And within the context of their current systems and approaches. Which are parallel to the benefits of modern C# rather than exploitive of them. I whole heartedly agree with the many threads and request for Unity to modernise around modern C# and to listen to and learn from Microsoft's enormous endeavours in modernising, opening up threading and providing low level performance gains, and core affinity controls. All wonderful stuff. If you're on any part of this processes, please keep up the great work! And, THANK YOU!!!

 

Speaking of CPU architectures, @JoshPeterson will we get an option to have IL2CPP generate separate GameAssembly.dll for various sets of instructions that are supported by the CPU like Burst already can with this?

This would be useful to have once we get all the intrinsics from CoreCLR available to use in non-Burst code, without such option we'd only be restricted to baseline ones. (CoreCLR will also bring AVX512 support so this list would need to be extended)

 

FYI kunalspathak from the dotnet JIT team told me they'd be interested in adding the IL2CPP tool to the dotnet/runtime SPMI infrastructure that's responsible for tracking the JIT performance and codegen generated by it. If your team would be interested in doing so, you can make an issue here and tag him there. (The code wouldn't need to be made public for it.)

 

I think we should talk more about Il2cpp not compiling properly. I have unfortunately also seen wrong results when compiling more advanced C# code using lambdas and generics, Linq and Tasks, resulting in memory corruptions or invalid cast exceptions etc.
I hope with the .Net Core Update a new deeper look into Il2cpp will happen to fix those bugs.

 

Come to think of it… not a completely unrelated question to ask: the year is 2033 and Unity updates the .NET runtime every new release and uses the CoreCLR and Native AOT to build. In this hypothetical where Unity does not need to maintain any part of it, how will it free up the development team to do more exciting stuff with Unity, and what areas would then be focussed on?

 

Yes! I found that even a simple static constructor in a struct doesn't run in IL2CPP...

 

Regular coreCLR updates must also be maintained, one must not forget that unity also has its own unity specific code in the runtime, and also all the communication between the C++ engine and the hosted .NET assemblies

 

Would this include Unity contributing missing features from IL2CPP like generic sharing into NativeAOT?

 

I did, it's CASE IN-43207.

 

I think the last update was this one at the end of April.

Can you tell us something new? In my case it is more curiosity because I find interesting all this adventure in which you have embarked with the many challenges it involves.

Thanks @JoshPeterson!

 

This silence could mean positive or negative things...
@JoshPeterson please give us some insight!

 

Unity just recently had a bunch of mass layoffs - it's possible things slowed down a bit. Depending which teams got gutted.

 

The best we can hope for is some polish to existing systems.

Newness probably on hold throughout the organisation - as it's hard to justify and quantify.

Sadly, the layoffs you speak of were the third, creating a pattern that aligns with statements by their leader that more are to come.

Combined, this means that everyone needing their job at Unity has probably gone into various forms of bureaucratic arse covering, rationalisations, record keeping and justifications for their survival, whilst othering those they'd prefer to see gone. Those others are often the best risk takers, and more capable, and inclined, able and likely to be consciously packing their bags and looking for greener pastures.

A pattern as old as time in corporations going down this phase of merger and acquisition staff cleanups is all subsequent creativity and growth can, thereafter, only ever come from ever more acquisitions.

 

Wow, das pretty sadge-

Josh has already responded that no one of his team was affected by the layoffs, dude's probably either busy or on vacation, the question is not a week old. It's odd because he usually has quick response time but to couple this to the layoffs is a bit dramatic I think.

That being said, I too do not understand how leader of "voted worst company twice in a row" got given the keys to the Unity kingdom, but that's neither here nor there.

 

Is there any plan to migrate more C++ code to C# land and burst it?

 

For sure! How do you think so many packages keep popping up? But I don't expect them to migrate engine critical code, like multithreading, physics, (although a good chunk of that also was migrated => Havok, also the rendering pipeline).

 

Great job @JoshPeterson (and the rest of the team)! I'm sure you are suffering with the technical debt but learning a ton along the way.

Edit: On the part of Microsoft in these last months I am reading very interesting articles about the challenges they are facing to migrate complex services from .NET Framework to .NET 6:

https://devblogs.microsoft.com/dotnet/tag/migration/

In case it inspires you to see that you are not the only ones embarked in this type of adventures!

 

@JoshPeterson this is very nice to hear.

I totally respect the "silence" and the small amount of detail you give us in each update. This is the type of project that is impossible to predict, be it the deadline or the amount of resources that can be completed in a timely manner. So it's better to be safe and create a polished product than to rush and disappoint the user base.

But I'm very curious. This looks like a very difficult project to pull off, and I'll be happy to hear the details! So, please, when everything is ready, write a post here (and on the dotnet blog too) telling all the details about this migration. It will certainly be very fun and enlightening to read about the technical challenges.

 

@JoshPeterson, do you think we will have these changes in which alpha version? 2023.3 or 2024.1?

 

There are a few upcoming events such as the Unity Unite 2023 (November 15-16) and also next years GDC (18 March 2024) - I would expect a roadmap to be revealed at such events (of course as long as it is ready).
Personally I would not expect .Net Core features to be available before Unity 2025

 

Please be as tech nerdy as possible when you do write it up! I for one am fascinated by how much legacy stuff you will have to update to pull this off and would love to know as much as you can reveal about the nuts and bolts of it all.

 

Rome wasn’t built in a day, I see, but on the other hand does it mean that we can't expect new .NET work be published in one or two years?
Coding would be very different between Mono and CoreCLR, lots of code writen now may not be necessary when new .NET features are introduced. It would be awkward for me if I put in a lot of effort in "c#9 style" coding, and then Unity suddenly adopts .NET 8 or even 9. Not urging, just wonder.

 

What redundancy are you facing exactly? A lot of people here may be able to help you with how to best approach it, but it’s on a case per case basis.