Official DOTS development status and next milestones – June 2022

How different is DOTS 0.51 from what DOTS 1.0 will be? I have stubbornly thought I won't get into it until 1.0. I dabbled years ago with 0.17 I think the version was, and thought it was amazing/the future, but it has changed quite a lot with versions. Wouldn't want to work with one setup and it all changes/breaks in the final version.

 

Frankly, not much different. The core development (I was about to say gameplay, ha) loop will be functionally identical to the current 0.51 or even 0.17 for maximum burst performance.

IJobEntityBatch with manual assignment of ComponentTypeHandles and the ultra-tedious copy paste of get component handle and manual .Update() to refresh handles before passing to the job struct.

0.51 with all it's code generation hasnt changed the fact that the lambdas and code generated IJobEntity access individual entities and their components when most, if not all, of the massive performance gain is done through access to component chunks and the unsafe utilities (mem ops and by reference access).

So on that front, you'll be safe from any breaking changes. In fact, the original chunk based access, IJobChunk is still supported despite being completely succeeded by IJobEntityBatch so if you had code written from the 0.9 days it'll still work.

 

It's still DOTS, and I'd say it's definitely worth learning right now. The foundational concepts will stay relevant, and API changes are cosmetics.

Anyway, check out the DOTS roadmap for a list of features they're focusing on in DOTS 1.0.

The "Adaptive Game Architecture" card looks interesting. It seems like they're experimenting with making some easy-to-use MonoBehaviour-like APIs. Should be nice for prototyping (and probably even performant enough for production in most cases).



At GDC 2022 they snuck in this interesting "work in progress API" that looks like a new lambdaless Entities.ForEach syntax. Looks pretty convenient if you ask me. Obviously, no idea if they're actually going to ship this, but this does seem like the last good moment for large changes like these.

 

if this is released as an update it will make work easier and more intuitive

 

Is there a VOD anywhere of this stream? I can't find it on youtube.

 

You can find the VOD on Twitch: https://twitch.tv/videos/1517796421
Worth a watch, tons of cool custom tools in this one!

 

just watched half of it to catch up some ( didn't use since 2019 when it was already awesome stuff..), very cool team and impressive work to say the least ! quick question ( but shall test in any case haha ) : things are working on consoles as well for the current version ? ( switch especially ), thx Laurent !

 

Is DOTS 1.0 useable for Unity AR Foundation? or are there plans to integrate both with each other?

 

When will it be available in the latest Unity 2022?

 

Please don't ask when, since no one will be able to give definitive answer.

Version 1.0 doesn't mean it will be fully featured. Most likely it will bearbone DOTS, from which direction new packages will be added over the time.

Do not rely on features, which are not released yet.

 

I remember DOTS introduction by Joachim Ante in general. He represents much of Unity, and it would be better to see him continue to introduce progress with DOTS.

DOTS < ECS + JOB SYSTEM:
I also remember that the first introduction made much more sense, but was more difficult to implement.
Now you have switched to a hybrid DOTS (monobehaviour & gameobject friendly), in which most of the coding is done by the engine instead of the coder (as far as I know, I have very low experience, and this is my naive feedback).
I think its a step back, but maybe take 1 step back to get 3 step forward.

More small DEMOS
I have tried to work with the flocking fish demo, it was quite difficult to understand at the time.
I have also downloaded the rendering/sound (megacity) entities demo, and studied it.
I have also downloaded and studied the Nordeus demo.
Then I have been very busy since covid and war; and as a hobbyist I have had to lose interest.
Making much more demos, maybe on very small scales would be good. It doesnt necessarily have to be huge and have a use for DOTS, just show the basics.
I suggest making a 2D sprite based simple game with DOTS (maybe the asteroid/space shootemup demo but with DOTS)

Priorities
I would prioritize: RUNTIME > UI & CANVAS/TEXT > VISUAL SCRIPTING (PATHFINDING, AI, MATH, GAMELOOP) > AUDIO & RENDERING > NETWORKING
Rince and repeat because NETWORKING will have implication in runtime/scripting/rendering/audio again.
This is not to take seriously, its how I see it genuinely. Make DEMOs of each: what is runtime setup for a 2D game, what are archetype setups, etc...

NETCODE
I see that you have outed security from your netcode endeavors. I think its a big mistake, security should be the top priority. Preventing the read & alteration of memory by the client. Double verifying the state of memory on the client and on the server should be the priority instead of having more entities. I see DOTS has being able to double check data as its goal instead of having more data. Example: For a FPS game, the game should be running entirely on a 3D area inaccessible by the client; the client run a shadow copy of the 3D area as a 2D version made with pixel-based sprites; the client gets very limited access to the 3D area memory and only interacts with the 2D version (tv screen style). Then you can freely double check client interactions on the 2D version, which don't appear on the real 3D version. EMBRYON IDEA, but running 2 versions and double checking and making random checks?

NON-GAMING PROJECTS
For sure DOTS will be great for lots of projects, but this should be left for developers to think about, not Unity. Anyway you have the rights to code made by developers.

The roadmap is very well done congrats!

 

You have many missing gaps in the post.

It is not just ECS and jobs. It is stack of technologies. That includes jobs, burst, jobs as prime, but then also comes with a ath lib, physics, hybrid rendering. Further down the line networking, animation, audio, and more.

DOTS doesn't support 2D per say. It uses 3D space. Quaternions and float3. Saying that, you can easily create 2d game if you want to. But not with sprite system as with game objects. You may need create custom support for animating atlases. Basically is just shader / material operations.

A bit odd priority order.
While UI is not directly DOTS compatible, you can use any UI system you want. Nothing stops to do so at given time. There were even some examples from fellow devs, making dots based ui. But definatelly is not the top priority.

Visual scripting is just ice on the cake. Shouldn't be touched, before we have solid foundation of DOTS. Is just waste of DOTS dev team resources.

AI and pathfinding is usually such project specific. Should not be part of top priorities.

Audio and animations however yes. However, most game will be fine, with current Unty native audio. None DOTS. Wonder why you put these at the end of priorities.

Networking is a bit iffy, what priority it should be.

There is no way to prevent reading of memory on client side. It is waste of effort trying fighting with it. Just build server authorative system, if you want to prevent cheating.

I rather see Unty team focus on DOTS solid core, than spreading thin all over the place.

 

So, there is a new blog post about Unity Networking and related stuff.
https://blog.unity.com/games/multiplayer-networking-tools-launch-announcement

And there I found interesting note:



Following that DOTS 1.0 will be released with Unity 2022.2.
And I am assuming the announcement/release will happen at Unite 2022.

@LaurentGibert, am I right?

 

I actually suspect, that DOTS 1.0 will release for Unity 2023.
DOTS team did some stuff for that release already, if I recall correctlyl, but I am just speculating atm.

Also, supposedly there are some significant changes upcoming with DOTS 1.0.

 

Some of the trainings-Samples at github already use ECS 1.0 Unity 2022.2 is used there.

 

Very interesting to read
https://github.com/Unity-Technologi...training-2022-09-na-group2/Ported/combat-bees

 

Honestly looking at most of these API changes and they - to me - range from "what's the point" to "a terrible idea".
I hope we get a good justification for most of these points (in no particular order):

• Conversion to Baking: What functionally changes aside from the naming? You now need another extra class (something that is often cited as being more confusing for beginners as its more mental overhead to keep track of) but is the actual "baking" boilerplate now easier to write? What about BlobAssets?
  
• SystemAPI and foreach: quite the generic name but okay, what is the advantage over .ForEach? It is now much more difficult to switch between main thread and scheduling (IJobEntity is awesome btw) and requires more than a one line change; I am assuming it requires SourceGenerators, can we turn off the feature if we do not use it the codebase (scripting define)?
  
• Aspects: I really tried to give this the benefit of the doubt but introducing the problems of OOP and data encapsulation into the entities world is a terrible idea.
  You can see the problem with this immediately in the sample tutorial: all you are doing is rotating an entity but you are using the TransformAspect that encapsulates all three components, you now have no idea except looking/stepping through the code to find out which data gets read and written to. One of the great advantages of the IJobEntity and .ForEach is that you can just look at the declaration and instantly see the inputs and the outputs of the data transformation (which are what you should worry about foremost).
  If this is the way we are teaching beginners then we will end up with the monstrosities that we all know like "Player" classes or "GameManager" just in Aspect form. That will lead to bad cache utilization and not learning to lean into the more modular nature that an ECS can provide and also lead to the same Spaghetti OOP code that we tried to get away from. In the worst case it will lead to a lot of "entities is slow" complaints from newcomers that were expecting the promised "performance by default" after watching a ten minute YouTube tutorial.

If the package will provide defines to disable the new features then I will reluctantly mourn the removal of .ForEach and start converting everything to IJobEntity but these new APIs really feel like trying to make something easier that was never the real issue: understanding the new concepts, DOD, Burst restrictions and learning new patterns to work with it and getting away of thinking in objects was the most difficult thing learning DOTS. The actual writing of the code and using the API never was.

 

Aspects, as far as I can see, don't encapsulate the data, they just group them together for easier reading/writing. As usual, the data itself is individual components. Instead of specifying all components individually in the query, you can use an aspect.

ISystem is only an interface, so you don't have a base class with inherited members, like Entities.Foreach. But SystemBase with Entities.Foreach still exist.

 

That's my main point: you are grouping together data that does not need to be grouped together. It's encapsulation in the sense that it tries to hide the fact of seperate components from you by using some code gen and neat accessors.

That's what the SystemState was for, you use this.state.Entities.ForEach instead.

 

I don't know the implementation of 1.0, it's possible that the ForEach version no longer exists in SystemState and the SystemApi gets along without source generation. SystemAPI is used in the tutorial for main threaded code, which is probably faster here, since you don't have to do a detour via job .Run().

You group together what belongs together, but it doesn't prevent you from using the components individually. TransformAspect most likely has access to Translate, Rotation and Scale components, and some helper methods.

 

It seems like ISystem no longer have state.Entities.ForEach anymore which I think it's really a regression that I no longer can use state.Entities.ForEach to write simple parallel code off the main thread. I really hope official can bring back similar feature

 

My understanding about Aspect is not just grouping but making sensible aspects of a game like mob, tank, boss...

In Monobehaviour land we often create big god Mob behaviour with everything inside and when we want to reference some GO as Mob we just look for this Mob component

In ECS this is no longer works because this Mob is maybe 10 components and every entity that has at least those 10 components is Mob. So if you want to write system that will process Mobs than you create Mob Aspect and use it with guarantee that it is Mob for sure in every place you use it and can not break if refactor components Mob consist of.

Something like this

 

That's what I hope we get an anwser for: why was .ForEach removed, I am sure there are good and not so good arguments for it.

Of course you can still access the individual components outside of the aspect, my argument is that this should be the preferred way people learn to do that and that Aspects imho are a bad abstraction that hides too many important details for you to gain an understanding of what's actually going on and for learning a more modular approach to your components/data structures.

 

In most cases the developer himself creates the aspect, so he knows what's going on. And aspects should only be pure access structs, which themselves do not contain any data, only references to the components and optional getters/setters for the component values. With this you can also avoid a lot of boilerplate code.

 

My guess is that they removed lambda jobs from SystemState because the lambdas can't capture the SystemState object.

Personally, I'm more worried about the number of times one needs to type a ComponentLookup member (formerly CDFE) to use it in an IJobEntity. Same goes for ComponentTypeHandles in IJobEntityBatch. You have to type it at least 5 times for a given type.

As for IAspect, it does show signs of OOP. The one thing it has going for it is that it is opt-in. By itself, I don't see anything wrong with it, but I know people are going to abuse it and make the components it abstracts internal, which would remove batch reasoning options and consequently cut off a lot of optimization potential. Besides that, I think I need to see how it operates in parallel jobs, especially the TransformAccess when other entities in the hierarchy have mutated local transforms in a different thread.

 

Not really sure what u mean but ISystem at 0.51 works really nice for me that it has state.Entities.ForEach.

From what I understand, official remove multiple ways of doing the same thing and only left 1 single way of doing thing. But I don't think it's good idea to only left long typing way to write parallel code off the main thread.

 

From what I understood, Aspects is just for grouping, if you want to you can have 1 Aspect per system and call it a day.

The key difference here is that it is re-usable across different systems, so if you have the same components that need to be processed by 2+ different systems you modify it in one place (the Aspect) instead of needing to copy the Entities.ForEach signature everywhere.

So basically in the same way that you can re-use logic between systems through jobs, you can now re-use grouping of components through aspects.

 

My memory might be fussy but the SystemState is just a pointer and the .ForEach gets converted to a job by the SourceGen, what issues with capturing does that cause?

That's a good point! Though I will add that ComponentLookup is a much better name then CDFE.

Agree completely, I would like to be proven wrong on this but I also see that people will abuse it.

I'm interested, is that something you would run into often? My argument is that you don't have enough repetitions to be worth the tradeoffs.

 

There is as much case that I run into that, as I run into re-using jobs: sometimes I can make it generic enough, sometimes I just need a new one because one specific detail.

Besides the re-usability, there is still the grouping mechanism itself. You don't need to remember that you DamageProcessingSystem requires a buffer of damage and a health component, you can just have a DamageableAspect to better document why those components appears alongside each other.

I liked ECS since day one, but the one thing that I always hated was how I always needed to remember to which components composed a feature and how hard was to a new team member to understand the dependencies between the components because you couldn't just put a "RequireComponent" (and neither would make sense, it is all data, why would a data depend on another? systems should be the ones with dependencies on specific data or other systems).

With Aspects, I have all of that.

 

The source still has to be compiled. And right now, you can't capture and use the passed in SystemState inside the lambda body because it is passed into OnUpdate() by ref. You could probably capture a pointer to it, but I doubt Unity wants to force users to write unsafe code. Granted, this issue is fairly easy to work around and I still prefer lambdas.

I like the name too. I think it provides a better CLU of how it works.

There's actually quite a few potential pitfalls with the new design that I have gathered from that repo. But (1) nothing in there is final, and (2) things could have been changed in the backend to solve the pitfalls, which I won't know until I have the actual source code.

 

You're right, forgot about ref variables/params not being capturable.

We'll see, if they are not addressed then I would hope atleast for good documentation calling them out (as is usually is the case tbf).

 

That is quite a bit upcoming changes. Interesting.

 

Thanks for the explanation. I have a couple of questions want to ask and feedback.

1) You mention that Entities.ForEach destroys iteration time in a huge way. Do u mean Entities.ForEach takes a lot of time for rosyln codegen to gen final code?

2) Since ISystem goes to Idiomatic foreach approach only for main thread code and for writing parallel code off the main thread is using IJobEntity, I suggest to try to figure out less boilerplate code solution that can achieve as easy as Entities.ForEach instead of only left IJobEntity option. I guess can provide some convenient parallel API to able to achieve writing parallel code too in Idiomatic foreach approach and then roslyn code gen the final code. I still want to write parallel code as easy as main thread code.

3) Now conversion system change to Baker and it looks really nice that lots of things has been simplified. I would like to ask how Baker works for dots netcode that has client and server world that I can easily tell Baker which entity and component needs to goes client or server world?

Edit:
4) Do u mean at main thread code, idiomatic foreach approach at ISystem won't generate any of the job related code like Entities.ForEach that has lots of job related code overhead even just run on main thread?

 

This is actually my biggest concern. While I definitely welcome baking systems being more like runtime systems and the riddance of having to declare dependencies explicitly, I have no idea how one would implement the following the way I currently have it in conversion.
In conversion without codegen, MonoBehaviours implement interfaces to get callbacks to participate in a conversion process. These are invoked from conversion systems in a dedicated conversion world. Conversion systems can also perform conversion logic directly. This meant that I could have the following sequence:
1) MonoBehaviours get a callback. They request blob assets to be generated for a specified input. They receive a handle with each request.
2) A conversion system processes all blob asset requests for a specific type of blob asset. It can generate these blob assets in parallel and perform both input and output deduplication. All the conversion systems that do this run back to back.
3) MonoBehaviours get another callback, where they can resolve the handles to get valid blob assets. Then they can decide what to do with those blob assets. The same blob asset type may be requested by different MonoBehaviour types, and stored in different IComponentData or IBufferElementData types.

Got any personal examples of this?

 

For us EFE never been "convenient" for more or less serious jobs. Yes they're cool (with Job.WithCode) for 5-10 lines of code and we use them for that (and on that matter changing that to idiomatic foreach wouldn't change it's usability, and IMO it becomes even more convenient), but when job more or less bigger, should be easily readable, should be structured, without ugly brackets nesting (formatting EFE always - hell), with better code reuse, without local functions or split to methods without HUGE amount of arguments etc. Yes you can make wrappers for method arguments for static methods in for each, but at this point it becomes the same boilerplate code as writing your IJobEntityBatch but in case of later - it's much more convenient, structured, readable, extendable, debuggable, doesn't require spending time to codegen etc. When EFE become a thing we started to use that, but really fast we found out that it's just mess, and we rewrote all more or less big jobs to our own jobs and\or IJobEntityBatch, and with our own "extensions" for IJobXXX we reduced boilerplate a lot and got more convenient code which just easy to modify, find bugs, add new features.

 

Thanks for the answers, really appreciate the input

I agree, GenerateAuthoringComponent was never really usable outside of prototype/game jam code. Putting the bakers into a separate assembly sounds like a great idea though, didn‘t think of that!
Do I understand correctly that the manual ConverterVersion and dependency declaration is not necessary anymore with the new Baker API (two huge pain points with conversion imho)?

The compile time difference is a good argument, any improvements are highly welcome. I‘ll happily remove EFE if the difference is that significant
Out of curiosity: what was so slow about EFE? I have been using SourceGenerators myself and would love to hear any pitfalls you ran into.

I‘ll reserve final judgment of course but I think I disagree still, I think the convenience is what makes it so slippery for beginners.
I recently had to interview and then onboard people into entities/DOTS so my opinion is skewed into that direction, I don't doubt that it will have usefulness for some situations but I don't think the positives outweigh the negatives.

Fair enough, I'll give the benefit

Super hyped about 1.0 btw, if 0.17 and 0.50 are any to go by I am sure 1.0 is going to be fantastic as well.

 

With all the changes, will there be an experimental/pre-1.0 release? so the smart people on this forum can give some feedback before a 'no more breaking changes' full 1.0 release is done.
0.17 -> 0.50 -> 0.51 were (for the most part) easy upgrade paths without to much changes to the core setup. But this sounds like a bigger change in the overall syntax.

 

https://forum.unity.com/threads/subscenes-dlc.1339274/#post-8458826

 

What's the baker API to declare dependency for like scriptable object?

 

Can I conclude that not only baking data into subscene now is insanely fast and no more sometimes failed to bake properly and need to restart editor and clear entities cache to fix it?
https://forum.unity.com/threads/0-5...tity-header-file-couldnt-be-resolved.1339757/

 

I see. At very least I just want baker not to suddenly stop working properly just like 0.51 release even I just change fews line of code that is not authoring code it will get this issue.

Btw I still have question want to ask. How Baker works for dots netcode that has client and server world that I can easily tell Baker which entity and component needs to goes client or server world? I hope to get something like ghost component that u can put C# attribute to setup.

 

I can give you a fairly precise example with code examples. Let's say I have a blob type which defines animation clips for transform hierarchies called SkeletonClipSetBlob. Now, my project has many different types of skinned characters with different needs. For player controlled characters, I may want to use a motion-matching algorithm. For enemies and bosses, I may want to use a state machine controller. And for some things, I may want to just have a single simple animation playing.

All of these systems can use the same blob type, but would like to define their own components or buffers to store these blobs. In addition, the animation data in these blob assets gets compressed using a minimization algorithm which isn't cheap and something that I would want to run in parallel. There are utility methods for reading the compressed data on the fly at runtime which are well-optimized.

In the case of a simple animation that only uses a single clip, this documentation of my solution shows what the MonoBehaviour looks like:
https://github.com/Dreaming381/Lati...a-blob-asset-using-an-authoring-monobehaviour
RequestBlobAssets is triggered by a conversion system which runs very early in the conversion process but after the prefab declaration or "discovery" phase. Then the blob asset generation happens, and then Convert gets invoked.

I use this mechanism for other things too. For example, I convert audio clips to blob assets (by the way, AudioClip.ReadData during subscene conversion for builds crashes in many but not all environments). In most cases, there is a single AudioSource ICD that contains a single blob asset reference as well as customization parameters. But in some cases, I may also want a DynamicBuffer to store blobs that can be swapped into the AudioSource component randomly at runtime. I do I similar thing with mesh skinning data.

What I am worried about is that any time someone would want to use these blob builders, they would have to write a Baker, a baking-only ICD, and a baking system to resolve the blob assets. That's awful compared to what we have with interfaces on MonoBehaviours and conversion systems.

Does that answer your question?

 

This sounds really good. I never trusted the old Conversion Workflow to be deterministic. It sounds like the new baking workflow serializes everything into a format that is guaranteed to be deterministic. Or am I reading into things beyond what was intended?