Official DOTS Development Status And Next Milestones - December 2021

I second this question, I use that to create 1-1 relationships between entities and game objects where either the entity or the game object can query whom they are linked to.

 

Does this mean we are going to have some visualizations for Data Flow Graphs as well?

 

I second this; I really loved that demo but it's been more than a year since they've shown it IIRC. I'm planning to use ragdolls in the game I'm working on currently so I'd love to hear an ETA on that.

I'm assuming Entities 0.50 will release alongside other DOTS packages, so I hope these questions are still relevant to this thread:
- Any updates for a medium/high-level audio package coming soon?
- How about visual scripting, any updates on the horizon about that? I feel like visual scripting could potentially relieve us of a lot of boilerplate code writing.
- Any improvements to writing collision/trigger code? Right now it's about 100 lines of code for what used to be a single line in MBs. I get that DOTS will always be different from MB's when it comes to how to write code, but it's currently definitely way too cumbersome. I've been hoping for some kind of equivalent to Entities.ForEach or something.

Anyways, glad to see more communication from the DOTS team, I'll be eagerly awaiting the 0.50 release.

 

We've used DOTS since 2 years ago and did our pre-production using it and until 0.17 came out things were advancing reasonably well and while not perfect we were ok with doing Cameraa and UI and VFX outside of DOTS and we only needed navigation and animation and navigation is pretty solvable on our own as well.

Unity's animation team seem to be working on the DOTS version for a good amount of time and you guys effectively stopped updates to none-DOTS things to focus on it.
Not talking about it at all here made me a bit worried so please share your animation related updates sooner rather than later since this is going to be a big factor for us to choose between a pure DOTS game or not.

Also I'm a bit confused, from one hand you talk about MegaCity which only can scale to these crazy numbers because it is not hybrid and also about single threaded multiplayer code of the DOTS shooter which doesn't scale on streaming and logic as much and also say hybrid is your focus.

I understand that hybrid should be a focus since many features are not DOTS ready/native yet but I don't understand how are we supposed to rip the benefits of DOTS with hybrid and I'm specifically talking about perf and scaling benefits because at the end of the day

Either DOTS is used for game logic related parts and the game can scale and as you said relatively feature complete rendering and animation systems are needed

or

DOTS is used for parts which are mostly environment and cosmetic and will be just used for streaming those in and out which doesn't help with scaling the main game logic and we will be limited to the same number of NPCs and objects logic-wise

Am I getting something wrong?

 

Questions:
1. When you mentioned an opt-in way of writing game code that’s compatible with the existing Unity architecture, will the scope of DOTS include other language such as Rust, or it will still strictly include C#:

The reason is that our team has develop a workflow using Rust being an opt-in way of writing game code via a series of codegen tool while also being fundamental compatible with Unity. We are currently migrating our both C# and HPC# code to Rust, and developing more features in Rust over in C#. We still use C#/HPC#, but only for interoperability purposes(glue code).

2. Maybe this might be out of the scope of DOTS, but will Unity provide more API(such as higher level binding to Graphics than existing ones) to native plugin by like providing more function pointers via header, or even interface library that could used for linking?

 

Good to hear. I am curious to hear what approach Unity would recommend for entities interacting with gameobjects given the hybrid nature of existing DOTS and the plans for DOTS 1.0.

Given that the scope of DOTS 1.0 appears to be the entities, physics, netcode, and rendering packages what approaches should we leverage to do things like navigation, animation, audio, etc? Low-level experimental navigation APIs? NavmeshAgent? Experimental DOTS Animation package? Playables? Roll your own?

What is the most performant and best way to connect entities with game objects?

 

I'm curious as to DOTS and ECS in regards to Visual Scripting (and Animation) now that we're looking at a 0.5 and 1.0 release.

Also, what about streaming floating origin worlds? -- Megacity doesn't cover this use-case, and yet Unreal 5 is built upon it. Any word on environment streaming (especially in the native environment authoring tooling, such as the editor itself?) -- I am looking to use Houdini to build meshes with 64-bit world translations that Unity struggles with -- in particular with particle systems and physics shifting. I would genuinely want this all to be handled natively during authoring such worlds, and all of the hard technology (such as virtual texturing and particle/physics shifting for floating origins) should be better-handled by the engine itself, without input from the user.

I have to respectfully disagree with this. The tools to create programmable tools are still quite subpar AT BEST. Not every artist who needs to make artist/designer tools is a programmer who knows web development. The value is, people like these can stop making buggy, half-baked tools and simply "build" their UI components together without knowing deep-level object factory generation algorithms, specialized API classes, etc.
With an ECS-based UI Elements system setup, one would simply define a component (say, a radio button) that behaves in a particular way. Then "stack" this component with various other layers of behavior in a similar way to gameobjects -- but using something akin to my "Dataless Tags" setup to sort out functionality in an easy-to-understand, visual, way.

The current UI Elements solution (particularly UI Builder) is still a viable one, but it generates lots of garbage files on one's hard drive (that could be better-handled with a better authoring UX to begin with), plus it fails at helping non-programmers / non-webdevs create their own tools, visually and programmatically, without A LOT of effort spent on NOT designing.

 

I guess the point i was trying to make is tiny was "pure" and removed the complexity of trying to figure out how to mix gameobject and entities. For me the biggest headache I run into is trying use non DOTS with DOTS. Now I know this might be my own lack understanding but to counter that I dont know of a lot of good example that demonstrate how to mix the two. Maybe the can be taken into consideration when new examples are created.

 

@uDamian Currently at Entities 0.17. The entity u put into Subscene will not no longer have entity name. Is that still the case for Entities at dots 0.5?

 

Thank you!

One of my biggest fears is that in a year's time without community feedback, DOTS would accidentally close off a lot of its flexibility and extensibility in favor of streamlining workflows for specific teams and use cases.

Your replies have put those fears to rest. I'm really looking forward to the next release now!

 

@uDamian Class based system I mean SystemBase in dots Entities package. Struct based system I mean ISystem in dots Entities package. For UI Toolkit runtime UI API I mean everything at UI Toolkit is just still the same and work the same. What need to change is just make UI Toolkit runtime UI API to struct based instead of current class based to make it compatible with ISystem in dots Entities package. Or maybe I say it in another way that make UI Toolkit runtime UI able to accept by burst and not throwing compile error. I just want it able to compile and work properly at runtime when I put UI Toolkit runtime UI API to update UI inside ISystem on main thread is enough for now. Not need support multithreading if it's too hard to get there.

 

@s_schoener Ya. I know u are talking about hybrid components. But still if I understand correctly internally hybrid components just wrapper of game object. I would like to tell u what hard problems that I found that need to solve for hybrid components after using it for quite long time.

1) Hybrid components should work exactly the same just like game object after converted into subscene. Currently particle system will missing some effects after converted to hybrid components. Skinned mesh renderer not support hybrid components that need to fix. And also for UGUI and UI Toolkit runtime UI that needs to able to convert to hybrid components and work just like game object. Currently I put ConvertToClientServerEntity / ConvertToEntity with convert and inject game object conversion mode at UI game object as I can't converted to hybrid components exactly the same properly yet. Another issue is I can't change something to debug at UI after converted to hybrid components as everything is locked after converted entity and u only can inspect the data as read only.
2) Hybrid components currently have gc issue when updating translation and rotation. Currenly hybrid components related system that doing the update will allocate gc every frame which is really bad and need to solve.
3) Hybrid components needs to have game object pooling feature by default to solve performance issue. This feature will provide some kind of tooling and APIs. The best use case I believe is particle system that needs to spawn and destroy frequently.

 

I did some rough benchmarking awhile ago with (I think?) that software float library and found that arithmetic can be an order of magnitude or two slower than hardware floats. Things like trig and sqrt even moreso, so not sure software floats has much of a future for situations where performance is at least somewhat important. FP performance for basic operations is quite quick in comparison (add/sub are just integer arithmetic, mul/div just add a bitshift), but yea does suffer from other issues as you said.

 

Thank you for all your input thus far - super appreciated.
I have some concerns around ISBs - particularly good patterns of how to use them amongst the ecosystem of hybrid & SystemBase.
The concern is general but I'll try to give a specific (psuedo) example:

Code (CSharp):

1. class MyManagedSystem: SystemBase
2. {
3.    public JobHandle someJob;
4.    ...
5. }
6. struct MyNiceISB : ISystemBase
7. {
8.    // Some job that wants to depend on job from MyManagedSystem
9. }

So, unless I'm missing something, this isn't a pattern we can use? Because any reference to the managed system kills burst of our ISB. I could see how we could move the jobhandle to the ISB and set it from the managed system but then, what if 10 ISBs depend on that job? Is this a situation I should be trying to avoid in the first place?

Part of what I've felt a bit frustrating with ECS is that although the feature set is limited, there are still many ways I can write things that produce dead-ends - leading to countless refactorings. I hoped this would get better with time but it seems to me that even with 1.0 there will be plenty (actually even more) dead-ends. If the hope is a less frustrating exp for devs new to ECS, I'm worried the experience will actually be even worse.

This is tricky for me to articulate well so apologies.
Another example might be trying to write APIs or convenience methods.

Code (CSharp):

1. public struct Car
2. {
3.     public float SetVelocity(ref EntityManager ..., float value) => { Set<Velocity>(value); Set<Parked>(false); }
4.     public float SetVelocity(ref SystemState..., float value) => { Set<Velocity>(value); Set<Parked>(false); }
5. }

Not a good example, please be generous in thinking of a better one. The point being, this is a trap. Car can not be used or referenced in a future ISB. Ok, so, I know that now... but are we assuming that should have been obviously a bad idea to me from the start?

So static extension methods for everything? And duplicate all the methods for ref SystemState & EntityManager?
Side note: Surface area for the duplicate ECS state. & EntityManager. API must be huge - are you using any tricks? Are we all expected to have double the surface area for all of our APIs going forward?

I am assuming that with 0.5 or 1.0 we'll be able to use NativeContainers in ISBs. ECBs too? What about lambdas? I missed it if somebody already asked.

In my experience so far, ISBs need to be used as much as possible for the speed, so "you can just continue to use SystemBase" would be a pretty unsatisfying response for me - especially if the whole concept here is to push the hybrid flow.

Fundamentally, given nothing unmanaged can ever touch managed, the glue is the most important part. Examples that illustrate e.g. how a bursted job in an bursted ISB waits on a bursted job from an SB would be really useful.

I recognise I may have some esoteric concerns as there doesn't seem to be much chat about these kinds of things. Thank you for your time - I hope the above made some sense.

 

so first q of 2022,we will meet the 0.50
or just " get more information" in first Q

 

So firstly I want to say Wow it's coming :O Christmas came early this year.
I also want to say thanks to unity's engineers for working on these tools that my life depends on!

I've been working on an ECS game since 2018! I'm still learning new things. I want to ask some questions, as my code base is 80k lines atm and my game is pure ECS, where I even removed Hybrid Renderer due to the better control with custom render systems. So:

Will there be much rework this time? Particularly for ScheduleParallel Systems.

Will there be AudioSource, Camera, Light Source components in ECS (atm I use a shared component data to link to gameobjects and just shadow the data from ECS to GO)

To clarify, I just need a way to Camera.Render and AudioSource.Play with my data! I don't use realtime lights atm so I probaly don't need this but I know many do.

If you are depreciating Entities Debugger, are you going to add the Filtering to the Entities Editor (and holy %#@$ I didn't know about this editor until now, it pleases my love of organization)

On my components, I am using alot of BlitableArrays, Unsafe memory things, will there be a way to display this in the editor?

Why does it have a GameObject link in the inspector for the entities editor? Are you hinting that I use them? :O

Will it work with WebGL? This is a huge issue as the original ECS did work with it, and then it stopped, and I really want that sweet webgl build so I can get more users.

Will I be able to remove even more of the editor, and just ship a pure ECS game? Or will there still be the bloat of all the older packages / Editor stuff that I don't use now.

I am really looking forward to authoring my entities. And never seeing GO's again!

 

Currently, there is a limit to how many entities can be named in the editor. Will this limit disappear or at least be increased to a really huge number?

With the better debugging tooling it might not be necessary anymore but that have got in my way several times (and the error is rather cryptic too)

 

Wow, can't wait to start blogging about DOTS again! I'm looking forward to continue learning all the new stuff.

 

Awesome!
I really appreciate your blog and it was very helpful to get a better detailed overview on how to use systems and features, such as the conversion pipeline, that currently lack some proper documentation IMO.

 

What's the ETA on being able to disable/enable individual components? There was a thread https://forum.unity.com/threads/new-enabled-disabled-state-filtering.902531/ from a while back that could use an update so that people can search up the answer easily.

 

Trig functions etc are also problematic on fixed point. You basically need to use lookup tables and will get low accuracy.

If you are doing a 2D game and are very careful fixed points would probably be fine. If you start doing anything with multiplication you probably need to go to a 64-bit fixed point especially if you have slow moving units/effects. Remember 0.001 * 0.0001 is an instant underflow if you use any kind of reasonable scaling factor. You can literally get huge cumulative errors.

In a modern 3D game fixed points are just not practical at all. If you are doing movement via integration, you will basically instantly overflow or underflow. You basically end up with a jittery bug mess with weird singularities and overflow errors. Just running a benchmark is not that same as the hell of dealing with fixed points for a modern 3D game. I wish people would stop suggesting fixed points because it is just not workable for a complex 3D game.

So basically, you either switch to C++ and try to get deterministic floats that way, and you have a lot of options with C++, or you need to use software floats. Whatever the performance of software floats is it should be enough for a classic lock-step sim on a modern CPU. What we worry about is it may not be enough when you start running a lock-step sim with physics that is more advanced than classics RTS games + rewinding. Rewinding allows you to have a more responsive game because you can have client-side prediction but decreases your sim performance by at least factor of 60. Luckily it will not affect your graphics performance at all if you keep it in a separate world or use classic Unity.

Our game will have a max unit count of 144 per player so we will still be less then classic RTS games with unit counts of 1000+ per player. Network bandwidth has increase to the point where we maybe could use a nondeterministic sim but, in that case, we would have huge bandwidth requirements for a video game.

 

In general when linking entities and game objects is it preferable to use managed component datas over EntityManager.AddComponentObject?

If I have a MonoBehaviour

public sealed class MyComponent : MonoBehaviour {...}

, is it better to do something like

Code (CSharp):

1.     public sealed class LinkedMyComponent : IComponentData, System.IEquatable<LinkedMyComponent>
2.     {
3.         public MyComponent Value;
4.  
5.         public bool Equals(LinkedMyComponent other)
6.         {
7.             return other != null && Value == other.Value;
8.         }
9.  
10.         public override bool Equals(object obj)
11.         {
12.             if (obj is LinkedMyComponent other)
13.             {
14.                 return Equals(other);
15.             }
16.  
17.             return false;
18.         }
19.  
20.         public override int GetHashCode()
21.         {
22.             return Value == null ? 0 : Value.GetHashCode();
23.         }
24.     }

over

entityManager.AddComponentObject(entity, myComponent);

?

 

The problem we've run into with IComponentData classes is that they could break our build depending on the order and content of its fields. For instance, having a single referecence to a TMPro UGUI in a IComponentData class caused our build to crash. It was so weird to debug that we never touched these classes again.

The best way we found was to store these refs in a Monobehaviour and AddComponentObject. Then read/write to these things before or after simulation in order to leave the simulation as pure ECS as possible.

I also wonder what Unity suggests on that matter. IComponentData classes or Component Objects? And why?

 

Hey, thank you for replying to some of my questions! My first question wasn't asking for a different work around, I just (and others) want a way to use the same functionality inside ECS without any GO references at all! This has been brought up accross several forum posts. The audio package wasn't an ECS one but a new tool for sound engineers to use. What some ECS enthusiasts expected was to be able to use Camera and AudioSource inside jobs for improved speeds.

I use mostly ScheduleParallel (SystemBases) for the highest speed. The only exception is with access of Mesh and Material classes. Which is another shame. So I would also request we be able to use Graphics.Render inside jobs! Possibly store our own pointers to the GPU and trigger it this way!

So I believe WebGL builds break because of entities package, I recall doing extensive testing and when we added the package it broke the default URP example. I am just wondering if 0.5 has fixed this. But I am also wondering if our Parallel jobs systems will work with WebGL as well.

 

Thank you for the update.
I am a bit sad to see that Entities is going the Hybrid rout as it feels like it will be very messy. Though I understand that it makes it much more accessible and 'completable' than supporting full DOTS out of the box.

You keep referring to Entities 1.0, this to me implies there are (potentially) much larger plans after 1.0.
Are you able to give any insight or hints in to the direction or future things we could see from Entities after 1.0? I'm not asking for specifics, more just broad vision/direction longer term. Though I understand if you are not able to share at this time.

 

Will there be a way to disable this functionality for those of us not using the conversion workflow in a particular project? That is great news on filtering of components! I understand many of my questions have no answers but, any possibility of File Access (IO) in jobs in the next 5 years? I rather ECS be more complete before deeper integrations as we can use the tool better then using a half finished one with integrations, that being said I guess the UnityEngine code itself limits some of the core stuff (Cameras/AudioSource) from being used in jobs, and any engine part rewrites will take years. 'won't be read-only in the Editor' Awesome!! my other question I forgot to ask. Amazing. I also want to add in, having Cameras and AudioSources in ECS would let me get the advantage of them, for both my portals and thousands of sounds everywhere. I assumed this was the end goal of having less limitations from architecture! It's exciting to push the machines we have to the limits.
Edit: After much testing, with all the editor versions available, I figured out that LTS now works with entities! I last tested several months ago, so it's good to see that it works on a version of unity now.

 

This sounds like great news, thank you. Particularly around nativecontainers & lambdas recognised as key ingredients and being well on their way. I am pretty surprised EntityManager is now a struct - curious how that migrates for people!

Re dead-ends, I actually think ECS being opinionated is great - my preference is generally for that to be pushed further wherever possible.
A lot of things I've run into are performance related - too many systems, too many jobs, having some managed references, having complex dependency chains, limitations with generics, where to store shared data between systems, what systems should know about each other etc etc. These lead me to refactoring in quite a major way each time. Some is just upskilling in DOD but some is working around the glue I mentioned - e.g. finding the combination of setup that's faster.
I know it's been mentioned before that the aim is to make it fast enough so you don't need to worry about e.g. number of systems. If this does end up coming to pass then that will certainly make a big difference - making more variety of patterns performant. I guess I'll just have to wait and see what it's like when it all arrives.
Thanks again for all the hard work - excited to try out the next release.

 

@uDamian I see. The main reason I request this feature is I really want a entity & system visualization tool that able to have bird eye for all my entities so that I can see all the components of the entities and data relationship that which component references to which entity easily. Furthermore, there's toggle to add system into the visualization that which system is access which data. The tool will have some kind of smart filtering to avoid display too systems at once and make it not readable. I'm not really sure if there are even better way to visualize this other than graph-like.

 

@LaurentGibert I still think Entities 1.0 needs to release as soon as possible as preview package for early adopter to battle test so any bugs and usability issues can be solved and improved early. From what I understand, the transition from Entities 0.5 to Entities 1.0 will have massive breaking API changes. It will become super hard for me to upgrade to Entities 1.0 if u still plan to release at 2022 LTS as my dots project will implement a lot of things at that time.

 

I second this, specially when it seems 0.5 wont be compatible with unity 2022 at all.
So any DOTS heavy project wanting and whiling to migrate to 1.0 would need to also upgrade to 2022 forcing a double transition/update.

Being able to work on early 2022 versions would spread the effort along several months. Even if that total effort ends up being bigger for us because of changes in the dots packages during that time, I think it would be still worth it.

 

Thanks you for this early (referring to the good news) or late (referring to the .50 release) christmas gift.

Good holidays to the whole unity team and thanks you for your hard work over the year.

I'm eager to play with that new release !

 

0.5 WILL be compatible with 2022 relatively shortly after the release for 2020 tls. they have stated this. so you will be able to spread your transition over a few months.

 

Where is this coming from? What I read is 0.5 would support 2020 LTS and later 2021 LTS. 1.0 would be released with 2022 LTS. So far we only know that LTS versions will be supported. 0.5 only being released to hold us over till 1.0. That release schedule seems reasonable to me.

 

I'll be more than glad if 0.5 works with 2022 but they have dodged (or missed) my question so far.

 

Does the original post not answer your question with what they're targeting?

> The first is a set of experimental packages updated to version 0.50, compatible with Unity 2020 LTS at first and targeting compatibility with the upcoming Unity 2021 LTS later in the coming year

 

No, it doesn't.

Edit: I mean, yes it implies no support for 2022 but some people think it will so a clarification from Unity staff would help settle the matter.

2022 introduces breaking changes with regards to 0.17 hybrid rendering, ergo any package supporting 2021 and 2022 would need to account for both "paths" which might be possible but I think unlikely.

 

Can you people at Unity give more examples on how to use the audio graph package more deeply?

It would be really nice to have some examples on how to mix two or more audio clips, so they sound like they are coming from some location and blended together. Some super simple example. The play clip example wasn't enough to get me started using it and I don't want to use whole week end blind testing an api.

 

I'd also like to know about DOTS Audio, which was mentioned at least in the Unite 2019 DSPGraph talk as being in development. Is DSPGraph and DOTS Audio still being worked on in tandem with the core DOTS packages? Will those be updated or new preview versions released with 0.5 and 1.0?

 

It seems I'm the only one bringing up this argument here but when such a simple feature takes such an enormous amount of time and still is left with 1 year of dev work because of edge cases, I've to ask: Are you sure you're on the right track? It could very much end up being a feature that will never be good enough or really buggy with weird performance characteristics. To me it seems like patchwork. Integrating something that's clearly not meant to be.

As the answers were also, that this feature will not play nice with SIMD. Honestly, what's the point? I very much feel the point has been lost completely what this feature should be. I could almost assure you that 99% here only need an enable/disable state for no-data components/tags! Not both, as in enable/disable components with a bunch of data!

You try to solve too much at once here and the users and early adopters are paying for it because such an integral part of Entities is delayed over and over again which is beyond frustrating at this point.

Make some well informed decision about it. It may seem entitled but I can hardly accept that this feature is being delayed yet another year! Not when the point has been missed that much. Split the problem apart and have something ready for 0.5. As Joachim mentioned some form of working state has already been programmed, so make efforts that you release some part of it, namely, supporting the enable/disable state for tags (no data IComps) only.

TL;DR: Release a smaller subset of the enable/disable feature that only works with tags or IComps with no data, sooner than the full feature later.