Official DOTS Development Status And Next Milestones - December 2021

Its slow to implement. Many things in games are one offs. If your writing a manager to manage data for something that happens once your spending a ton of time to do something very simple.

Its just like an organization. You want overhead where you need it and want to cut the overhead where you don't. Its the same reason I prefer C# over C++, I don't need the control 90% of the time and C++ requires more lines of code to do the same basic thing.

Dots is great, when you need it.

 

I really hope we will be able to chose between full ECS or full GameObject model and mixing is not going to become a standard.

 

Doesn't seem to be the direction things are going.

 

It used to be that with low entity count performance was terrible because of overhead. Is this an area that has improved since?

https://forum.unity.com/threads/ecs-code-overhead-for-mobile-games.690265/

 

If you want pure ECS, Bevy may be the best choice at present.
https://github.com/bevyengine/bevy

 

I'm more conserned about How to make a TPS game in ECS0.50 and 1.0, since there is no built-in ecs animation support. Is the best practice for now, is to use an animator driven character gameobject syncing with a pure dots physics rigidbody entity ?

 

While I wouldn't argue for separating the two completely (makes sense to keep UItoolkit for UI, etc), I would say this:

Hybrid workflows add a ton of extra cognitive load, extra steps, & messiness to the project, compared to pure GO or pure DOTS. In order of "easiest to use" to "most difficult to use", I would rank them like this:

1. Pure GameObject
   
2. Pure DOTS
3. Hybrid

In my opinion, Hybrid is the least accessible tech of all 3, and demands the steepest learning curve. If you need to learn Hybrid, you not only have to learn pure GO and pure DOTS completely, but you also have to learn all the different kinds of strategies you might have to use in order to make Hybrid work. You also need to be fully aware of what APIs work in what tech stack, and what is conversion-compatible and what isn't. For example, someone who starts learning Unity will see that there's 2 different kinds of "Rigidbody" components in the engine, and will wonder what's up with that. And then there's all the .WithoutBurst() and the .Run() constraints that won't be obvious to a ton of people. It's a big learning obstacle, and I can imagine this creating a lot of confusion compared to a workflow focused on one approach

With Hybrid, "Performance by Default" turns into "Performance for advanced users only". Because users now need to go through the trouble of figuring out special project-specific strategies in order to take advantage of DOTS. With pure DOTS, all of it just happens naturally without you having to think about it. Does it take time to learn how to use DOTS? Yes. But you'll have to learn most of it either way if you use Hybrid, and then more

It's true that Hybrid allows you to reuse existing tech, but in practice that reusability will probably be very limited. Any Asset Store asset that used monobehaviour physics instead of DOTS physics will not be reusable (terrain systems, vehicles, AI detection, character controllers, etc.). Moreover, any kind of "inventory system" or gameplay tools of that sort will probably be too monobehaviour-specific to be nicely integratable with some DOTS-implemented functionality. Existing tutorials focused on GameObjects are also likely to be much more misguiding than helpful.

All in all, I understand how hybrid is useful in the short term for teams with ongoing projects, but it's hard for me to imagine this "two simultaneous mix & match tech stacks" situation staying for the long term and not being confusing to newcomers. A clean & focused workflow is important for accessibility and ease of use, but this is not what Hybrid gives us (at least not in its current state). I've written an example of how Hybrid can make things more complicated here:
https://forum.unity.com/threads/dot...nes-december-2021.1209727/page-2#post-7727868

 

(For easy reference to PhilSA's discussion)

The pure GO approach is baseline with Unity and all users will be required to learn this. It's pretty high level though, so I do not consider this a time sink. Most of Unity's vast resources and messaging funnel people into this relentlessly so you can expect it to be the price of admission or baseline - an absorbed cost.

The real "cost" is still only learning DOTS, and if it's hybrid by default then you're still only learning DOTS and only as a subset of Unity to solve any performance bottlenecks that *might* arise. For many customers, they will not get that far or ever need DOTS.

You are (I'm assuming) coming from the position of an experienced developer, and experienced developers will all know the GO workflow.

But I'm fairly passionate about hybrid being made much easier and basically as easy to assemble and use as components in the GO workflow. If they do that and leave pure-ish low level DOTS for the experts then it's not a big problem.

It just means all they really need to do is really make hybrid as accessible as regular components are. Maybe that means going heavy on syntax sugar, I don't know. But I don't really see a future any more where Unity can make DOTS exist in a vacuum (or why).

 

This is a self referential assumption. One can imagine a future where this isn't true, because the expected way to work on new projects is to do everything in DOTS. Projects like Bevy are building that workflow, so we'll see how well it ends up working practically.

The "performance by default" idea that Unity claims to be pushing is incompatible with gradual adoption of DOTS only where it's needed. The claimed benefit of DOTS is that you just write ergonomic code in the framework, and it's automatically able to run better on a wider range of devices than is true in classic Unity.

Hybrid workflows also limit some of the claimed performance benefits around entity creation and destruction. "Look how easily we instantiate thousands of bullets" is a classic boast, but when somebody iterating on the design says "What would it look like if we added lights (or particles, audio sources, etc) to these bullets?" you've got a headache, because lights are hybrid components that can't be instantiated and destroyed in that carefree manner. "Just pool them", you might say. Well yes, but we could have done that with GOs anyway, so what's the actual benefit we're getting here?

I understand having a hybrid mechanism for backward compatibility, and I think that's sensible. I don't understand not committing to providing pure DOTS versions of all of the classic components, so that hybrid can be a rare fallback rather than the default. I would be understanding of there being a slow rollout and transition to the pure DOTS versions.

 

One thing that I find quite poor about Unity's ECS at the moment is the handling of global or singleton data. As I understand it, the recommended pattern is still to store these things in singleton entities. Other ECS engines like Bevy have a nice solution using "Resources", which are basically singleton data that can be ergonomically included in a query for dependency management. Is Unity going to adopt an approach like this?

 

Am I the only one who somehow lost around here? In what engine and what setup would you add individual lights to every single bullet in a swarm of bullets? Which engine can handle hundreds or thousands of lights appear in a frame and gets destroyed 5 frames later?

 

maybe lights would be more uncommon, but audio and particles would be pretty common.

Same for animation when spawning waves of enemies, or if your projectiles have animated components to them, etc

 

I don't know about thousands, but hundreds, here's Unity 3, 12 years ago. (go to around ~1:40)

Hardware has gotten a bit better since then.

 

Don't worry everyone, remember when Microsoft introduced the metro interface in Windows and they moved some of the control panel stuff to the new UI, but some of it remained in the old one? I expect the Unity hybrid DOTS / GO workflow to be about as smooth.

 

I'm using lights as a simple example of a surprisingly basic component without an apparent DOTS version planned. My point is about the general hybrid workflow, rather than focused on this particular example. As @PhilSA says, there are better examples. I'll edit that comment to clarify.

However, modern renderers can actually do pretty well with large numbers of small lights. Deferred and forward+ rendering has lighting cost mostly scale by the number of lights affecting each pixel, not total lights in scene. Each bullet having a small area of glow in a bullet hell is within plausibility.

Instantiation and destruction of lights shouldn't necessarily cost anything. In a deferred pipeline, they're basically just instanced circles drawn to the gbuffer. It shouldn't be any harder than spawning things with meshes.

 

Agree with this and with @PhilSA .
It's a bit like hybrid cars, where you wind up with double the complexity / maintenance.
Pure solutions usually win in the long run.

 

Actually, modern hybrid cars using eCVTs have far fewer moving parts and are less complex than their pure gas engine equivalents. Of course, a pure electric is better yet.

So, to continue the metaphor, a pure DOTS game should offer the best performance and greater simplicity than a hybrid game, but a hybrid game that uses game objects for things that do not need the performance of DOTS and uses DOTS in performance critical areas may still be better than a heavily optimized pure game object game that makes a lot of compromises.

I would prefer that a hybrid DOTS that lets most users of unity take advantage of DOTS in some places sooner, than to push DOTS into the distant future in the hopes of the release of a pure solution.

 

Good to see someone knows things about cars. I'd like my GO-DOTS development as I like my car: Plugin-Hybrid.

 

This isn't my opinion though, just Unity's facts. They've said GO pattern isn't going away, that they will purposefully keep some things in GO land and so on.

Until you can demonstrate and prove that new users are going to stick around when punched in the face by an undocumented and changing beast for the next 10 years then I doubt very much Unity will push DOTS first. In fact that would harm their business model, when they can introduce DOTS at any stage of the work to fix any performance problems like they are proving right now.

I'm not saying you can't do a full 100% DOTS paradise: I'm saying it's poor business for Unity to set up all or nothing like that, when they've established a well over a decade of doing something totally different. That would be confusing to new customers much more than what you're passionate about.

 

That's all true, and I was imagining that it would be five or ten years before it would actually become The Default Way. However, I was hoping that it would become a valid and usable default in one or two years, for those who make that choice. With the apparent lack of plan to port over some fairly basic features like lights and cameras, this is blocked.

This is confusing to me, because they're actually so close to being where I want them to be. The DOTS infrastructure is on course to be in place for this, it's only those unported core components that are the blocker. It looks to me like they're basically there, but just not bothering to do the final 5%. Which, to be fair, is par for the course for Unity and all of their unfinished features.

I think it's indisputably true that the "performance by default" idea that DOTS was sold as only works if we can confidently set out using DOTS as our main framework for a whole project. If we don't use DOTS by default, it doesn't give us anything by default.

I recognise that DOTS delivers real value to teams that just drop it in in the few places where it's needed. But if that's how we're expected to use it, we also need to acknowledge that DOTS is no longer what we were told it was going to be. Personally I find that disappointing, and am becoming increasingly hopeful and interested in alternatives like Bevy instead.

 

That's one way to look at it. Another way is that whenever you are dipping into DOTS, then by default the code structure is going to be performance oriented.

I think the marketing term can stay, it doesn't need to die from a thousand semantic cuts like democratizing development did.

 

I will say it's not they dun want to port all core components to dots but it really takes insane amount of time to get there. The current hybrid dots already takes so much time to get there. I dunno how much time will need if want to get to full dots by default. Currently what I see the practical way is to push hybrid dots to truly production level as hard as possible and implement hybrid feature to make hybrid integration into dots project as easy as possible.

 

That's great! I got this impression because I believe that I've seen the exact opposite claimed by Unity staff before - that things like lights weren't going to be done. I'm very happy to hear that that's not the case! Thanks for updating us all.

I'm sorry if that was a misapprehension on my part, as I haven't rechecked my sources right now, although I'm pretty confident in my memory.

 

With stuff like vfxgraph dots runtime still categorized in the "under consideration" part of the roadmap, I don't think this was a misapprehension at all. Optmizing the hdrp lightloop is nice though a bit reminiscent of transforms, my understanding is that is optimized under the hood, yet as long as it remains native to gameobject only, it's still a big old performance anchor for any project looking to utilize that type of behaviour in great quantity

 

No worries, but I am now a little confused. For clarity, should we expect ECS lights of some kind, which don't require a managed component? And seeing as lights were a somewhat arbitrary example that I used, should we also expect the other various core engine components like cameras, particle systems, and audio emitters to have unmanaged ECS versions at some point?

The question that I'm really trying to get to is whether the long term plan is for us to be stuck with hybrid/managed components for any of the core engine features?

 

IMO Unity should adopt a release cycle like the one .Net has now with the LTS versions.
Support 1 Unity version per year for 3 years.
Add new ways of doing old things in a new unity version, without worrying about backwards compatibility, and mark the old api as deprecated so people can adopt the new way.
After 3 years delete all of the legacy code.

Unity is haunted by backwards compatibility.

With this approach you don't have to do work on backwards compatibility, can adopt new technologies easily and give your users plenty of time to adopt the new api if they even consider upgrading to a newer unity version, and remove lots of the bloat.

 

I wonder; with DOTS embracing hybrid, is there a possibility of DOTS Physics becoming the new default physics engine for monobehaviour?

Not only does it have vastly better query APIs and better performance than the old PhysX integration, but it would also unlock a whole new world of Hybrid DOTS implementations in monobehaviour projects. Rigidbody sim, Raycasts, and Overlap checks are very often a significant portion of a game's frame time, and having the DOTS Physics backend in monobehaviour projects would theoretically unlock the possibility of doing all that stuff in Burst jobs. I know there are batch Raycast APIs for monobehaviour physics currently, but they really can't compare to the usability of DOTS physics queries directly in jobs.

It would also provide a common physics API to use for third party assets. I've read in another thread that some devs are working on making the PhysX integration more job-friendly, but I don't really like that scenario because we'd still have the problem of having too many different physics engines, so too much unnecessary division in third parties. Not to mention two physics engines cost more to maintain than one.

I think this would be one of the most promising things to have in order to allow most people to get a lot of benefits out of Hybrid

 

What I would like to see are DOTS examples in a tutorial in the learn section.
i.e. This is the example project (Survival shooter) -> These are the problems to solve -> how to solve this in a DOTS way.
How the example project is built from scratch to finish solving these problems one by one.
It could even be the existing MonoBehaviour project and converting it to DOTS.

These projects can be broken up in very simple things too and can be easily compared with the MonoBehaviour way of doing things.

Personally I find having bite size pieces of tutorial a lot easier to understand how things work than "here have a big example project". Usually the example projects are overwhelming and you wouldn't know where to start looking.

Subjects such as:
- DOTS fundamentals, define some

IComponentData

. Define a system changing that data. Use of burst. Use of the EntityCommandBuffer.
- How MonoBehaviour components and DOTS components can work together in a hybrid setup (read / write)
- Events & Input System. (Hooking up WASD to move a capsule. how to define an event with data and have it handled by some DOTS system)
- UI Toolkit (i.e. Use of a slider to change the volume of the game. Some form of interaction to change some DOTS data. An inventory, a skill bar, Player HP, Enemy HP. Many examples can be made to read / write data)
- Instantiation of a prefab (Spawning system, instantiate every X second)
- Animation of a character, change / manage states. (a 3rd person character hooking up WASD and shift for running.)
- AI Navigation (NavMesh)
- Physics (make a character jump, throw a ball against some stacked cubes)
- Material / Shader manipulation (player throws wood into a fire and then it burns, becoming charcoal)

More advanced subjects:
- Netcode, running a server instance, syncing data from server to client. Instantiate and move a capsule with WASD. Have multiple instances of a game and move their character. Creating / joining a lobby etc. Simulate disconnect / reconnect or lag.
- Animation instancing (render many units for RTS or Open World)
- Large Open World area streaming (like the Mega City example)
- Addressables
- UnityWebRequest

Probably a lot more that can be thought of.

I have around 7 years of experience with Unity. I mostly know my way around with MonoBehaviours. But if I were to handle the same problems with DOTS, I wouldn't know how. It's a different way of thinking and programming with data oriented design.

 

I second this. A step-by-step tutorial like this would be greatly helpful in better understanding of DOTS/ECS. I would like to see it when 1.0 is finally released.

 

This is extremely dangerous, in my opinion, and I hope Unity knows it as well.

Games nowadays have waaay longer lifecycles; the game-as-a-service approach is everywhere.
10-years old games are still updated and monetize decently.

Putting a death sentence like that would mean losing one of the biggest advantages in using the most widespread engines: the warranty they won't disappear in a short while.

 

What do you mean?
Especially for games as a service this would be optimal. For every new way of doing an old thing you have 3 fricking whole years to adopt the new way. Especially for games as a service, using old code that has been superseded is cancerous.

Also: .Net is used in 'normal' applications for way longer than 10 years in some cases, and the .Net team adopted this release cycle and people have been immensely excited about it.

 

.NET keeps backward compatibility... It's not telling you "You have this gigantic codebase that uses Entity Framework to access data? Ok, from now on you have to re-write everything from scratch using this new data access tech."

You're proposing something very similar for DOTS, here: forcing entire games to be rewritten from scratch every 3 years. Come on, games often take more than that to even approach early access...

 

No one is expecting you to rewrite your whole game every 3 years. Only the parts that have significantly changed. And you actively gain benefits by doing it. If a game is supported for more than 3 years the devs will be tasked to adapt the new api anyway so there's no reason to keep the old one around after the transition period.

Example: Why are there TO THIS DAY, the old UI elements available, aka all the UI components that do not use TMPro? Why TF do we need to keep this around?

Fine. Another proposal: Mark every old api with
[System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
and remove them from the documentation OR add a toggle to the documentation to 'show deprecated api's'.

 

I agree with you to some extent.
Some minor elements should be probably deprecated sooner (old Text...), but I suppose they have good reason not to remove them (IE tons of old plugins made unusable despite their core functionalities are still intact), and/or maybe the fact they're still there is not compromising new developments anyhow... I don't know.

But the topic here is DOTS, and going full DOTS in 3 years, stopping any support for all the rest, looks like a damn hell of friendly fire on your customers, to be honest.

 

It was experimental all the time and still is. Just make a fricking breaking change, that's really no problem. It's not like you developed with the expectation that everything you write now will still work tomorrow.

 

I don’t know what you think would be solved if they removed the old text component. It’s not like they’re touching its codebase and it still has some minor uses.

 

That was an example. There's also a lot of code itself that has no uses since years ago because it's either been deprecated or superseded. Also the legacy text has literally no uses. It's been fully superseded by TMPro. You need the legacy text for absolutely nothing.

 

I mean this was definitely false until a few months ago, not sure about now. But anyway, not sure what you think would be accomplished by them taking it out.

Taking out the text component won’t magically make DOTS more complete.

 

I am currently working on this with a hack and slash game. Either you have to hybrid to un animator and navmeshagents or go low level and write your own animation system using playables graph.

 

You are surprisingly underestimating the hidden cost of the engine changes.

It's affordable and acceptable for _our_ code to change frequently. It's not acceptable for the engine code to change all the time. Because we control our code, but have no say in how the engine behaves. Allowing the engine to make breaking changes frequently is a huge risk no sane company will accept. Change = cost, sometimes, too expensive.

 

You're missing the point.

You're missing the point too.

 

What is that in this suggestion of you I'm missing? I think I understand this paragraph perfectly fine. And this is just horrible to me. The current upgrading process from one LTS to another LTS is too costly already. With your suggestion of "delete all of the legacy code" just "after 3 years" I don't know how much we need to pay in order to fix some bugs in the older LTS that are only fixed in the newer LTS. Usually those bugs involve plugins integration, platform compatibility and requirements, which force us to upgrade Unity version. Do you have any solution for this scenario?

 

I am as dissatisfied as you about backward compatibility. But my subjective feelings are worthless compared to the cost I have to pay in case there is no backward compatibility.

 

And you're missing the point in berniegp's post that led to this whole tangent in the first place.

The issue with DOTS has nothing to do with legacy APIs and breaking changes, but rather accidentally breaking working, stable, money-making features in Game Object land to add new API the original features weren't designed for.

While this might explain why Unity has struggled so much with subscenes and making a good hybrid solution, it has nothing to do with ECS and is more about making other things in the engine Burst-compatible (PhysX comes to mind).

Anyways, the reason we can't get all these core features in ECS yet is because recreating the entire engine in this new paradigm requires a lot of work and Unity would rather take that on incrementally to ensure they get the designs right. They were originally going to move a lot faster but decided to slow down and focus on quality, and I respect that decision (even if the communication was poor).

Speaking of poor communication, the hybrid vs not-hybrid thing is being fueled by poor communication in the form of mixed messaging. On one hand, Unity has stated they are embracing the hybrid workflow (which makes sense). On the other hand, the only other news about hybrid is that they are removing (well limiting) a hybrid feature. Hybrid is already hard if you care about GC and intuitive serialization. So making it harder is making people uncomfortable with this decision.

As for lights, I actually know what is up with that. Basically, the HDRP team realized that all the real data they cared about lived on HDAdditionalLightData and that funneling lights through the engine just to extract all the custom properties later was dumb. So instead they directed all the data into an unmanaged representation and did what the engine used to do using Burst instead. This was mostly for performance, but it also introduced a new code path where unmanaged light sources were feasible, and they called out ECS with those changes in GitHub.

 

Just to clarify; does this refer to user-created class-based IComponentDatas? Or does it mean monobehaviours won't be components on entities after conversion anymore?

I've just noticed that trying to instantiate a gameobject prefab with EntityManager.Instantiate(prefab) gives us a message saying that this function is obsolete "since there are no proxy components anymore".

I have some concerns about how cleanly & easily we can implement things like ECS <-> UI interactions without these. Imagine if we want to update a value in UI only when that value has changed on an Entity (example: update displayed Score in UI only when score value has change on the score entity). That entity will need some kind of way to hold a reference to the UI Text that it needs to update when it changes. We can do this by making that Entity have a managed component which holds a managed reference to the UI Text component. But if we can't have that, I'm really unsure how I could handle this (unless I do some terribly high-maintenance hard-coded strategy where I "find" the associated UI Text every time, based on pre-made enums and Dictionaries and whatnot). There are similar complications when we want a UI button press event to update a value on whatever target Entity is assigned to it at runtime

 

Structs can have references to classes so it should work.
Anyway, entities/systems updating UI from their side seems counter-productive as u would need to have a main tread job blocking almost everything else.
I think UI poling is the most un-intrusive way to do UI with ECS (yes, it's not top performance but ECS is not the panacea, u can't have everything, trade offs must be done).