Official DOTS Development Status And Next Milestones - December 2021

I would rather just stay away from ECS altogether than deal with the mess that is hybrid. You had a good vision. You were going to rewrite the core of the engine. Now you are creating an even bigger mess than what you started with.

I seriously don't understand you guys anymore.

 

I think it's all assumptions until we see 0.5 and then 1.0.

 

I think hybrid is a good approach as long as Unity provides documentation on how to do it. I prefer that we get a hybrid DOTS production ready release than to have it stay experimental until all of the core engine packages have been rewritten (given that there are not publicly available experimental packages on things like DOTS terrain / environment, DOTS navigation, etc it could be many years before a full rewrite of the core of the engine). A hybrid DOTS 1.0 is by my estimation a steppingstone to a pure DOTS future.

What's the status of DOTS animation? It is best to animate with playables / animator or should we plan on using the DOTS animation (even if it is experimental / preview with the DOTS 1.0 release)?

 

ECS is mostly about performance but only a fraction of the components that make a game are sensitive to that performance gains. For all others, it doesn't really matter if they are ecs or gameobject based. All we need is a simple to use, bug resilient way to handle interaction between ecs systems and gameobjects.

 

Two things:
1) All they said is that it won't be ready by 0.5 (which they already branched off of (in other words 0.5 is likely going through QA right now) and that it will be ready by 1.0. That could mean we see it in a 0.6 or 0.7 or something. They never said they needed another full year on it. That was simply extrapolation on your part.
2) This technique is fast and powerful and makes a ton of sense. However, there's a lot of nuance with it not to make it work, but rather to make it intuitive and easy to use at every API level. I would rather they take their time to get it right so that we don't run into nasty pitfalls as is the case with change versions and in-place archetype changes.

 

I'm with Enzi on this one. I hope Unity is not going to spend 1-2 more years adding features most of us don't even need.

I think the entities and entity physics + netcode runs quite well. Not sure about the netcode, but at least they fixed the vr-issue a year ago.
This leaves audio for desktop, which is working, but not as well documented as the other core packages.

What I'm trying to say is that the whole DOTS stack is quite close to being usable in development by everyone(except project tiny I guess).

Without Latios we wouldn't have an audio solution that you can just install and it works.
I guess my end goal is to point out how cool it would be, if there wouldn't be preview flag on the entities and
people would start making more high performance plugins/frameworks that would enable people to make higher quality games that even run on slower machines. And also better games.

Now that its on preview state for this long, people are wondering, if they should drop out or never go in in the first place.
I know people from Unity says they are committed to this, but it doesn't mean its impossible that some executive decides to just drop out the development on the whole feature, because they need to cut some losses on the development and focus on the core business. Seen that happen during the Corona pandemic way too many times.

 

Tldr DOTS production ready in early 2023 or later.

 

This is an interesting question that got no reply. Anyone?

 

I guess we'll see when 1.0 releases.

The idea / hope is to have some of the ease of use and safety of C# with no performance compromise.

The worry is that it will feel like code that belongs in another language, forcibly jammed into C# and requiring much more awkward boilerplate than it would in a language more suited to it like C++ / Rust / Zig / Jai / Odin / Whatever.

But C# is seen as this big advantage that Unity has, so I really can't see how they could go "we're dropping C#", even if it was the correct decision, without their stock plummeting the next day.

For the immediate future, I hope they start writing their own tools and features using DOTS, so big parts of Unity will get a performance boost, without me having to interface directly with it. The announcements in this thread point in that direction, so I'm somewhat optimistic.

On the other hand, this is a company that hasn't really produced a great feature or tool in the last 5 years (at least from the ones I have used), so maybe the underlying tech was never the problem.

 

Yeah, ultimately the big advantage is that, in comparison to things like Rust and similar is that Unity provides an entire toolset that is interoperable with standard C#, which is a language that is excessively more accessible than something like Rust and the various C 'em Ups. Rust especially has a fair bit of a learning curve due to how it handles accessing variables and C++ has a whole host of syntactical issues that can trip up a lot of developers compared to languages like C#.

On top of that, adding another language for Unity to support would be kind of a nightmare when it took us this long to finally get down to just one.

 

If you are interested, I can share some first impressions from porting some netcode to Rust. This is after two years with DOTS. In no particular order:

• Everything is native compiled to begin with.
  
• Fast compilation. I assumed this would be slow, but compared to Unity, it's super fast.
• Everything is stable. Code you write often works the first time. It's a bit weird to describe. Haven't encountered any compiler / language / tooling bugs.
  
• Arrays are stack allocated. This was a nice surprise. For example, for netcode if we convert an integer to bytes, the to_le_bytes() function returns a byte[]. I assumed I can't use it because heap allocations. But it's stack allocated.
  
• Zero cost abstractions. As far as I understand, pretty much all of the convenience features are as fast as doing it manually. Compared to Linq, which we basically can't use for gamedev.
• Fast. I implemented a few networking tools and benchmarked delta compression. It runs about the same speed as the Burst compiled one in Unity. Pretty impressive for Burst though (!).
• Syntax is very convenient. Feels more like Python than C/C++. Easier to write high performance code than in C#. Code is very readable and elegant. I always missed that in C#.
• Compiler forces you to write stable code.
  
• Borrow checker seems great for game development. Both in terms of freeing memory & being free of data races. Concurrency in C# is hard.
  
• IDE support is great. vscode is free & open source (can't use in Unity because it's still missing heap allocation viewer). Jetbrains CLion is great, way faster than Rider for me.

Note that I am still learning, there's definitely early excitement bias here.

This is from another thread:

And I have to say I get a similar feeling.
Native & no GC seems what you want for game dev.
C/C++ are too hard to use, so Unity thankfully picked C#.
But with C#, we have to avoid GC at all costs and are limited to HPC# subset.

Rust however is both native, and easier to use.

Reminds me of this:

---

I don't work for Unity, but the topic sure must've come up there.
I guess what they do still makes most sense for them.
Huge C# ecosystem for Unity. Their engineers know C#/C++.
Switching to a different language would be super risky for them.

 

This is still a fascinating point, and something early in the lifecycle of Burst, IL2CPP and DOTS Unity produced a few good comparison benchmarks showing off how the performance of Unity was improving.

The best comparison I have seen was made by a community member who took standard benchmark C++ algorithms and converted them to run in Unity...

Spoiler: Big Graph C++, BUrst, IL2CPP, Mono compared

As you can see even with Burst and IL2CPP Unity often is a lot slower than C++ it would be interesting to see if DOTS could boost it's performance?

Benchmarking Burst/IL2CPP against GCC/Clang machine code (Fibonacci, Mandelbrot, NBody and others) - Unity Forum

 

I think the question / request here was not whether there were plans to use some other language, but rather to "sell to us" why HPC# is a good solution, even though C# does not seem like a natural fit for DOD.

 

What if Unity compared DOTS to other ECS systems like this benchmark?


This benchmark only compares the speed of two systems updating data.

 

On the topic of possibly using other languages than HPC#.... I'm under the impression that maybe we should just pick our battles and stick with what is already working currently. A switch to Rust/C would surely delay everything by another bunch of years, and would also restrict accessibility (from the perspective of the Unity community as a whole, who is mostly familiar with just C#)

 

I don't think anyone is asking for a language switch. Just the sales pitch / vision for why HPC# is a good idea for those that have trouble seeing it as such.

(and at this point I will stop clarifying, because personally, I do not care for a reply right now. IMO HPC# is not a great idea and the only thing that can change my mind is a DOTS 1.0 release that has things I like in it)

 

I don't know if people have noticed, but there is a limit on the amount of systems.
Creating too many will result in "not enough system state slots" like exception. Don't remember exactly if it was like 4096 system state slots. Could have been 2k.

So would be interesting to see how this scales up with systems added to the benchmark.

 

Well that's one additional dimension to the systems issue others to consider could be:

• Graph complexity - system internode connections - S = n(n-1)/2 (potentially bottlenecking from memory bandwidth).
• Bottlenecked systems - e.g. 2 or more systems feeding into one. Testing resource allocation e.g. Threads/Cores/Memory
• System size vs complexity - Where is the optimal balance between data and processing.

 

Is also possible with C# (.net standard 2.1). However, only with some restrictions
untested Code (net standard 2.1):

Code (CSharp):

1. class Test
2. {  
3.     public static void Main()
4.     {
5.         int number = 25;
6.         Span<byte> bytes = stackalloc byte[Marshal.SizeOf(number)];
7.         ToBytes(ref number, bytes);
8.     }
9.    
10.     public static void ToBytes<T>(ref T value, Span<byte> bytes) where T: unmanaged
11.     {
12.        unsafe
13.        {
14.            fixed (T* b = &value)
15.            {
16.                for(int i = 0; i < sizeof(T); i++)
17.                {
18.                    bytes[i] = *(byte*)(b + 1);
19.                }
20.            }
21.        }
22.     }
23. }

I like C/C++/C# Syntax more than Python syntax.

 

Unity's focus is on completing DOTS features and making it relatively easy to use which is the correct path. It is not reasonable to ask for a language change after 3-4 years of usage.

HPC# might not be the greatest idea ever but has the advantage of it is easier to convert unity's user base to and supports all the features needed for writing jobified code for burst.
@AcidArrow if you prefer to use C++ for development, that is understandable. Many middleware are easier to integrate and some people are just more used to it since they've used it before but that is not most of the unity community and honestly for allowing the community to write jobified bursted code, a more ristrict langauge and a domain specific one is better than just C since unity can imposetheir rules on the DSL and limit it. This said however, in a parallel world they could use something like ISPC and i wouldn't be unhappy but at the end of the day. What we need is tooling and features and the language difference would not matter much between C and this subset of C#.

 

I am not asking for a language switch, I don't understand how so many people keep misunderstanding me.

I would actually rather they didn't go after DOTS at all and just kept improving the engine and editor. Because I agree, we need tooling and features, but what we got is a company determined to "fix C#". This is actually a big source of my frustration with Unity.

I've already said my piece in a previous post quite well and I don't feel I have anything else to add here.

Let's talk again when DOTS 1.0 releases, which I await with the tiniest bit of cautious optimism.

 

Well in that case you need to be convinced of DOTS overall regardless of HPC#. It has nothing to do with wanting to fix C# and rather everything to do with making it possible to make games with unity which were not possible with the previous approach and also with doing something future proof.

Many AAA games use data oriented design in their engine and need a modular and modifiable engine. Unity wants to penetrate that market and DOTS is a part of that alongside HDRP and other things.This is a talk which @mike_acton describes this in more depth Building a Data-Oriented Future - Mike Acton - YouTube


I don't know what does he think about the current direction of the company regarding DOTS.

I'm not going to argue with you if DOD is good for small teams and other games which don't strictly need it for perf but that is why. If all of what you wanted to make was possible with unity or you preferred to switch engines for things which are impossible with unity, it is your choice of course but the company naturally wants to improve and many others like it.

 

You’re acting as if their only path of improvement was DOTS, so it was a choice between doing nothing or working on DOTS, so obviously they chose the latter.

To me, it seems that if they really wanted to improve, choosing to stop improving for 5 years to do a lateral move that may or may not benefit everyone in another 5 years is a weird decision to make.

You may have noticed that “fix C#” was in quotes since it’s a term that was used in some Unity talk, it might even have been that Mike Acton talk you link, I do not remember and I will not look it up, but it’s not my words.

 

They really haven't stopped improving really though.
HDRP, URP, new terrain tools, new NetCode (admitably this one is not much of an improvement yet), lots of 2d stuff are all happening.
I don't say this is the only way but rather This is the only way to expand unity to some types of games and smoother experiences.

Some parts of unity historically did not improve much overall before DOTS as well (navigation and AI comes to mind) and the only part which stopped improving due to DOTS was animation which is not in a bad shape atm.

The same situation applies to other engines too. UE5 has lots of new tech totally unrelated to prev ones, UE4 abandoned their decades old unreal script and ...

The quality of features and delivery speeds are different in different places but not wanting any radical changes would mean many features would not see the light of day and it is not a logical thing to ask really. Somebody said they want to fix C# (maybe it was lucas in his HPC# blog post, maybe not) ok but still that is not the holistic reason of DOTS. Yes they have to change the language for DOTS/fix it so they can be free of GC and allocate better.

IMHO the talk linked above is very convincing regardless of you/me like the direction or not.

 

I disagree with all your points, even about the talk being convincing. Cheers.

 

Now I'm concerned about the state of v0.5. This is a huge old-announced feature and it's again delayed.
Looks like v0.5 will have nothing new. Nothing major at least. Maybe some UX like proper editor integration.

EDIT. As far as I remember it was said enable/disable component will land in the next update. And this info was after the announcement of the 1y of silence policy.

 

I somewhat agree they did a side step some years ago with DOTS. I'm not saying DOTS is a bad idea but for a while, it stopped development on other things like getting C# up to date which increases base line performance of the gameobjects and editor.

.net5 is on track for this year (hopefully) but it would have been nice to have both things worked on in parallel since day 1

 

Disagree

DOTS have completely separate team behind it and "9 women can not give birth to a child for 1 month" you know
Actually y I think DOTS have good boost for .Net 5-6 because it is first internal tool inside Unity to use it

 

Are Unity DOTS packages (Entities, Hybrid Renderer, Physics, NetCode) going to leverage ISystem in 0.5 or 1.0?

 

Good joke. Bit old, but the more I think about it, the more I think it really doesn't match software industry though.
Women cannot work on the same baby, but developers can work on the same feature.
So its more like 9 monkeys cannot produce a feature in a month.

Just use tdd with 100% coverage.
If the feature is well designed, then the work can be split in to units.
Unit testing will make sure code is working as it was designed.
Integration tests make sure developers know, which feature broke what feature and when.
That way monkeys work doesn't mess up the other monkeys work.

Adding skilled and well educated workers really do produce more.

 

U don't have to agree with me, just go read the 4yo forum posts. I don't know why but for 1/2 years after DOTS announcement there was no progress at all in the .net front. Up until early this year we were still running on a 4+yo mono fork. This year we got new mono version, support for standard 2.1 and steady progress towards .net5.
I don't really know nor I care if "core team" members were tasked with DOTS stuff or not, fact is Unity looked to the other side for a couple years. Only they know why :shrug:

 

We've managed to derail this thread to a point it's nigh impossible any member of the DOTS team will bother with it anymore

 

Bold of you to assume communication with Unity from a thread lasts more than a week and a half even when they stay on topic.

 

Will support such as
EntityManager.AddSharedComponentBurstCompatible Or
EntityManager.CreateSharedComponentDataIndex & EntityManager.AddSharedComponentDataIndexBurstCompatible?

 

They have different teams for different areas, but it often doesn't make much sense to move teams, especially if their areas of specialization are completely different. ECS / DOTS and .NET runtime are completely different areas. The core C++ engine must be adapted for the .NET runtime. Who knows how many developers who work on DOTS are really familiar with C++? I would say most of them are specialized in C# and even if they do, they have to learn how the .NET runtime works and how to integrate it. That would mean that for a while they will have practically nothing to contribute in any area. In addition, there is also a maximum number of developers who can work on a project. DOTS consists of several areas with different teams (as I heard from a previous post).
It would make more sense to lend individuals who are fit enough to support other teams, but certainly not to stop and entire areas. There is also the German saying "too many cooks spoil the broth"
A somewhat exaggerated comparison would also be if you had 200 developers develop a Flappybirth clone, that wouldn't make much sense.

 

As far as I remember, UT will make ISharedComponent component obsolete in the future.
And you can't make burst compatible something that by definition isn't as it can hold managed references.

 

What about ease of use?

For me the biggest hurdle to using DOTS is not it's paradigm, it's the layers of complexity, jargon and longwinded-naming-convention the API puts between me and using DOTS easily and simply.

Then there is the lack of middle ground between using DOTS and using MB making transitioning between MB code and DOTS way more complex/time consuming that it needs to be.

What if there was a super easy to use way of working with DOTS that allowed MB code to be quickly and easily converted.

And the thing is if you want to optimise MB code one of the easiest things to do is move from Solo components to Managed components and that is DOTS (or batched based processing).

Use Case Scenario:

1. Developer finds that the bullet collision code is showing up on the profiler in action heavy scenes.
2. Changes the Bullet : Monobehaviour class to Bullets : DOTSbehaviour class.
3. Behind the scenes Unity converts the Update and OnCollision calls and data to DOTS systems.
4. Developer profiles and now the Bullet collision code is a lot lighter and the next MB that needs optimisation pops up.

 

It's the corporate culture.

Why did I get the feeling they just came here with fire extinguisher to calm people down and went back to work

 

I mean, as long as they do that semi-regularly, that's not too bad.

 

At least it shows that ppl are interested in DOTS.

 

It has been repeatedly pointed out to you that this is not something realistic to expect. There is no proper way to transform managed code to unmanaged code (given all the constraints at hand). If you truly think this is something feasible, then you have to, at the very least, provide a theoretical example where a complex OOP, managed piece of code is translated to a DOD, unmanaged code.

I think it's safe to bet that if Unity had a way of doing what you are suggesting, they would have done that instead of spending years and countless man hours on a new technology/API.

 

[QUOTE="Lhawika, post: 7756065, member: 848927There is no proper way to transform managed code to unmanaged code (given all the constraints at hand). If you truly think this is something feasible, then you have to, at the very least, provide a theoretical example where a complex OOP, managed piece of code is translated to a DOD, unmanaged code.
[/QUOTE]

Just to confirm your point the data going into a Systems Update() function must be direct within struct data at the point when the loop cycles through that data to prevent data changes and allow for multi-threading locks.

So any DOTSbehaviour class that works in a batch based fashion must 'lock and load' the data it needs for the duration it runs.

Therefore any code that is written within an Update() or OnCollision() DOTSbehaviour class must avoid or remove any indirect data access e.g. flatten any data structures/classes and update those when it is complete.

Or you could end up with 3 cycles to a MB/DOTS class:

1. Extract and lock data
2. Process data
3. Update and unlock data

Or another way to think of this is that the Data is pre-structed in memory for the DOTSbehaviour system to ensure it can be updated in DOTS fashion without needing the Extract and Update loops.

And this memory structuring of Data is the key to any ECS system.

Side Note: The managed/unmanaged code argument is that really as strong as you think it is when most DOTS systems will need to use IL2CPP and Burst to really perform well?

 

I think you're dramatically underestimating the amount of work those processes actually require.

 

@Arowx go find at least one automation code, which converts C# OOP to DOD/DOP.
Once you got that, comeback with evidences. There is no point speculating in this thread.
Oh and btw. please use one of your existing threads, instead of hijacking this one.

 

The point I was making was the DOTS is now hybrid and needs to work with MB so therefore if there was an easier way to use the power of DOTS with MB then I would be overjoyed.

That easier way may not be 100% DOTS but if it allows the 20% of code that needs optimisation to be moved to DOTS/Burst for optimisation then it makes Unity so much stronger and better for game development on modern hardware.

I can always convert Solo MB to managed/batched MB using structs, Burst and Jobs and get near DOTS performance but if there is an easier way to convert MB to DOTS then that makes my job easier.

 

There is an easier way. In fact it is so easy that Unity doesn't bother to do it because it would take one person a weekend to implement and it would be one more thing they would have to support which would be confusing. It doesn't come with all the bells and whistles of ECS, but it will give you Burst and better data layouts. PM me or start a thread titled "Using generics to automate Burst-friendly MonoBehaviours" or something like that and ping me. But that conversation does not belong in this thread.

To bring this thread back on topic while also carrying the sentiment of recent posts, I'm quite curious if Unity is going to allow class IComponentData to serialize scene GameObject references or something similar. I can't see how else they plan to fully embrace Hybrid workflows now that they are removing support for custom hybrid components.

 

What we do is have an object pooling manager. We store an int ID in a IComponentData and use a system to process the entity and get the attached game object from the objectPoolingManager which has a managed Dictionary keyed to the ID and storing the associated game objects. We don't use entity IDs we have our own way of issuing IDs.

 

I'm not talking about runtime pooling. That's trivial. I'm talking about serialization of scene GameObject references from entities. The only solution I can think of is a GUID system that works with disabled domain reload. I haven't come across such a thing yet.

 

I am not sure what you are trying do. Right now, everything is tied to the Conversion Workflow.