Official Experimental Entities 1.0 is available

I would like to give my constructive feedback here after removing DOTSNET from the Asset Store.
It's been 3 years for me, and I think it's time to move on from this tech for the time being.

I know Unity leadership has their own vision and they don't care about my feedback.
But since I have been using Unity for over 10 years, I want to post it anyway in hopes of improvements down the road.

• Open Source C# packages are great, and a huge selling point for DOTS. It's great to be able to fix bugs 10 years down the road. Would be even better if more of the GameObject modules could be moved to open C# packages.
• Choosing ECS was a bold move. Very impressive that Unity leadership decided to take this huge gamble.
• DOTS performance is insane. Learning about data oriented design and seeing millions of entities in the world was certainly an eye opener for many of us.
  
• HPC# / native causes a lot of friction. Often times this might be an indicator of using the wrong tool for the job. For example, a systems programming language like Rust would've allowed us to use all of the language with a stable compiler.
• There seems to be significant focus on performance over usability. Unity got so popular because it was the easiest to use Game Engine. DOTS/ECS is just not easy to use. It would've been amazing to have the easiest to use ECS in the world. Instead it's probably one of the fastest ECS, but it's not easy.
  
• Hybrid causes too much friction. It's a nice idea to offer a slow, easy transition. But in practice it seems a pure DOTS editor would've made life much easier here. Both for the community, and for Unity.
  
• Floating point precision seems to contradict the goal of building large worlds with millions of entities. Now we have the performance to build EvE / WoW, but we can't actually build a world larger than a few kilometers without origin shifting. Unreal Engine seems to support double precision now, so it's a bit unfortunate that we are still stuck in the small world of floating points.
  
• Development could've been in the open, but it's still mostly behind closed doors. I am sure there are plenty of reasons against fully embracing Github and accepting community bug fixes. But still, it would've been nice.
• It was mentioned that DOTS would become easier to use than GameObjects. However, there seems to be no concrete example of how this is going to be possible. Of course it's not finished, but it's not obvious how this will ever be as easy, or easier to use than GameObjects.
  
• In the end, DOTS/ECS feels like fighting the language instead of being productive. Personally I switched from C++ to Unity because I didn't want to fight the language anymore.

Please understand that I don't mean to step on anyone's toes here.
I have huge respect for the DOTS team, building this must've caused unimaginable difficulties for the past 5 years.
Posting this with the best of intentions, in hopes of still seeing an easy to use DOTS in the future

 

That's all very reasonable.
I don't agree much on the 'Hybrid' case or needing a full DOTS editor. MonoBehaviour could be renamed to AuthoringComponent and you could call it full DOTS solution but it still would be the same. Yet it's a problem I see weekly on Discord. It's weird as the problem seems to vanish with a bit more experience. (no offense, I'm sure you know what you're doing)

I commend them for forcing/making it possible to properly split authoring and runtime data. But I understand that Hybrid is a broad term so you could mean something very different here. Where Unity dropped the ball, HARD, is the lack of animation. The GO route is really annoying and a very huge blocker for adopters.
I can't say much on Rust, I've never coded in it nor could I possibly say if it would make it easier. TBH, I doubt it as long as there are managed objects around, and those won't go away, even with Rust.

All that still makes me wonder why you pulled DOTSNET though. Was the baking system the final nail in the coffin or what happened really? I read your post in the DOTSNET thread but am still puzzled. Entities 1.0 changed very little outside of same-same-but-different bakers and enabled/disabled comps. Was the problem on your side or on the user side who couldn't figure out to make it work? I mean the skill requirements in DOTS are huge. It's not in the same ballpark as drop in some assets and voila. Mainly because of the fragmented workflow that again points to the missing animation package. But even though it's annoying, it's not that hard to make a proper GO <-> Entity link.

 

This highly depends on what APIs you were using. If you were using GameObjectConversionSystem, you got wrecked. If you were relying on IJobEntityBatch with API-facing EntityQueries, you got wrecked. I could go into all the ways this release was problematic. But also, there are bugs and design flaws everywhere.

The editor and engine has most of the bugs. BRG regressions. APIs crash in worker processes during subscene import. TypeCache will cause the baking initialization to try and instantiate an authoring component instead of a baker in beta 13 if you use a custom generic abstract baker base class on top of Baker<TAuthoring>. Then if something goes wrong in the Editor World, you are either forever spammed with errors or you have to detect and shutdown and restart the editor world, which will prevent subscenes from loading back in until you manually change scenes.

If you were making high-level gameplay, you probably didn't encounter these issues. But if you are making custom solutions like animation or networking, pretty much everything is set ablaze right now. It's easy to hack things together sufficiently for a project. It is much, much harder to build a robust general-purpose solution.

I'm quite scared of 1.0 final. Enableable components and Aspects are a mess right now, among quite a few other issues. If Unity "locks in" this API as-is, I don't think they will be able to fix the problems with the API, especially the low-level performance problems with enableable components and query filtering in general. At that point, if I wanted to share any of my personal tech, I would probably have to share modified versions of the packages too. 1.0 just needs more time to settle, because with the approach they are taking, they are setting themselves up for another PR disaster. For Unity, "1.0" just means "officially supported", which actually means very little to most of us. It will not be "production practical" because there are still major workflow issues and missing technology that most users would expect.

I owned DOTSNET, and you can keep the refund. But I would certainly appreciate having the option of running wild with a fork or derivative.

 

Yeah, that's a real problem right now. b13 is a dumpster fire, that's why I'm still on b9 which works fine for the most part. I'm also using generic abstract bakers so I'm even more glad I haven't upgraded right now. Same goes for a nasty bug in AssetDatabase where a lot of problems and crashes stems from. Scripts that are lost in caching or not recompiled. Pretty concerning stuff if you want to be productive. Best advice I can give is to downgrade to b9. It also has some of these problems but not as prevalent. Seems to me a regression made an even bigger regression. At least that's my best explanation why something like that can even happen. I get that's all very frustrating, especially because 0.51 and 2020.3/2021.3 was really stable.

I've said it before but it just isn't a great idea to keep breaking changes internal for so long. At this point, and merely 2 months away from 1.0 not being experimental any more, my hopes of having a well rounded 1.0 version are not that high. Pessimism aside though, only a small amount of fixes can untangle this mess very quickly, especially because the problems are so wide reaching.

Isn't that mostly replaced by a baking system? The most important feature that is missing in baking systems are proper blob creation, which is baffling how that went unnoticed. I've read your posts about its problems. Did you make any progress on it?

Can you elaborate on that?

 

I rolled back to b10 and that is a lot more stable, though it still has a lot of caching and update issues with the editor world which makes it really hard to debug baking systems. And at some point I need to move to a version with the BRG regression fixed.

There's a rough one-to-one with IConvertGameObjectToEntity and Baker. There is not a one-to-one for GameObjectConversionSystem. I pretty much had to rewrite Kinemation's authoring workflow from scratch. I'm still ironing out bugs with it. Assuming I understand the design correctly, I actually like the new workflows from an API point-of-view. It is much easier to reason about conditional dependencies which helps a lot when doing really complex things. My only real gripe with the new bakers besides the lack of documentation is the lack of granular control over bakers. I hacked in that feature and it works reliably, so I have no idea why Unity struggles with it so much.

The All and None of Entity Queries consider the enabled bits of components, which means you lose pretty much all ability to do reasoning on a chunk level if a query has them. They should have been a totally separate filter independent of the "types" part of the query. Enabled bits slow down chunk iteration quite a bit, so you want to use them only when they are helpful, and not by default. I could go into a lot more details about how they make a mess out of system update control and the number of situations where they force sync points. Plus it doesn't help that the tools for them are horribly inconsistent. You can't even set enabled components safely in parallel outside of chunk iteration, even if you knew the component values themselves are safe to set.

Expect more posts from me with real examples once I get my 0.6 out in the wild. But I think you got my point that I sympathize with vis2k's decision, as extreme as it is, because I fully know how frustrating that is. My brain is built for DOTS, and I'm near the point of giving up and focusing on art for the short term (I probably won't fully reach that point, but we'll see). Anyways, people need to be patient, because this rush rush is ruining what was an amazing tool.

 

Any timeframe for this no matter how wide the frame

 

I have watched vis2k via Discord over the years with his fantastic products. I am proud to say no need for a refund from me. I also had high hopes on the success of DOTSNET to the point I paused some aspects of my project because DOTS/ECS was always just around the corner. Now I am sitting here wondering if I should re-evaluate Unreal or go forward with new updated Mirror for high CCU MMO? Either way, I agree completely with vis2k and greatly appreciate his efforts.

 

I feel like I have to chip in here with a story because I'm in a similar situation like vis2k but with almost a complete opposite conclusion on many of his talking points. My path is from C++ (6 years) to C# in Unity and absolutely ragequitting the games industry in 2019 after 5 years of object oriented, technical debt inducing, "normal unity workflow". Working with big games in Unity hasn't been easy and we all know it. Ever since the Entitas talk on Unite 2016 I've been interested in this new ECS workflow and through working with the SpatialOS GDK I was introduced to "Unity ECS". I started looking for alternatives to SpatialOS after their pricing model and shenanigans pissed me off so I found DOTSNET in late 2020 when Entities was somewhat a good enough package to use.

Here's where things start to differ. I found that DOTSNET had a different vision of how I wanted a networking package to look like in DOTS. I wanted more Source Engine/Overwatch features with source generated code and more client/server world separation. The code in Netcode for Entities looked weird to me so I decided not to trust a library built on code that looks weird. I spent the year after that fully deep diving into GDC videos, bit shifting, snapshot delta compression, clientside prediction, lag compensations, spatial snapshots, code generation and made my own networking package for a battle royale game using the 0.17 and later 0.51 version of DOTS. It was slow to work with but I managed. It was already million times better than the MonoBehaviour workflow.

https://github.com/polartron/open-netcode

A year later I'm working on a multiplayer horse game in another game company and getting by slowly with the 0.51 API and a boilerplate-ish annoying way of doing literally anything in ECS until 1.0 came out of nowhere and set things straight for me. This is the main part of what makes me and vis2k's conclusion different:

I could finally see what the whole deal was. The end goal. High level and high performance user code made possible by Roslyn source generators and a fixed number of API concepts to learn. 1.0 made me humbly shut my own networking project down because of how easy it was to make a multiplayer game using Netcode for Entities. I was like Rock Lee in Naruto taking off his weights. My coding workflow has literally boiled down to just a few core steps on every feature I make.

1. Make components
   
2. Make an IJobEntity, define WithAll/None/Any attributes
   
3. "define the data transformation" (read from input, transform data, write to output)
   
4. Schedule from main thread

No more custom architecture, no more references, no more SOLID

I believe I'm able to hire students now because I can just give them the DOTS_Guide and have them be able to contribute "close enough to production code" because of the burst compiler and the fact that the concept has boiled down to an analogy of you installing transformation machines on a conveyor belt.

I'm sad to see DOTSNET go because I feel like we both suffered the same faith in the end. Getting made completely redundant by the 1.0 release and seeing all this work we have to do in order to get our packages to the standard that Netcode for Entities has set.

So I don't really know. It's strange. We both made a really advanced networking library but ended up with completely different opinions on the future of making games in Unity. Me being completely on board and vis2k quitting. I have a suspicion that competing against Netcode for Entities just made it easier to have a reason to let go of DOTSNET, not DOTS being difficult to work with. My package was just an open source project on github so I didn't have any customers connected to it.

 

vis2k's Has some good points. I think Rust would have allowed data structures to be standardized using standard Rust code and less C# baggage. However, as a general scripting language Rust would be more complex. I get the feeling that some within Unity wants to keep HPC# as a scripting language and try to separate details from the programmers. Whatever the case it is now too late and Rust has only recently become increasingly popular. What HPC# really needs is better documentation/standardization and probably a low level book. That will take a C# programmer from zero to hero.

ECS 1.0 may have temporally broken projects, but it is over all a much better API surface, and I am much happier with it. ECS 1.0 is full of bugs, and I am not sure how they are going to get it polished in time for 2022.2. We will see what ECS 1.0 preview looks like. For now, I will not read too much into what I view as simply an API preview we can build code against. It has helped me to get up to speed. I am the kind of person that can literally go for years without having a project that will compile, while I focus on architecture and algorithms.

 

Thanks for the info Scott!

I hope they keep CompanionLink or something like it in 1.0 because it's a common concept in game engines for people to want to create a "prefab"/"blueprint"/etc that has a particle effect, audio effect, or light attached to it. Not supporting that would just be silly. It would be a shame if every time a developer wanted something so simple as making a torch mesh with a point light and fire particle effect on it - the dev would have to write systems to create and dispose the attached game objects at the appropriate times as well as systems that ensure the transforms are synchronized. Every developer would have slightly different solutions and it would be a barrier to newcomers - something that is ZERO lines of code in regular unity would suddenly require two or more systems and some data structures.

 

Nice to see two very diverging thoughts on this topic from two people who have used it extensively.

Thank you both @PolarTron and @vis2k

 

We use the 0.17 Entities package and disable the automatic bootstrapping as we currently do not use subscenes, as they did not support light probes when we adopted dots. Are subscenes supported in the familiar build workflow that many deverlopers are used to? A section about "How to build with subscenes" in the docs would be nice.

Also, when playing around with 1.0, it seems like we can throw anything in subscenes now? . What happens to colliders and other non dots components? In 0.17, converting such components threw errors as they are were "pure ecs". Now we just don't understand what's going on. A section about "what happens to your components during conversion?", covering various use cases would be nice too.

 

I still have yet to find much info on this, is there anywhere I can find some clarification? Can someone chime in with any usage details, such as what is allowed/restrictions?

 

If a native container contains another native container, then the outer contain can't be used in a job. Only the innermost containers can be used in jobs. Besides that, you can pretty much nest things however you want, as long as you handle disposal correctly.

 

Just sat down to begin digesting entities 1.0 and saw the news on DOTSNET. I've been using it for a couple years now and am quite impressed not just with the technical aspects of the package, but also with @vis2k's willingness to be out-front on ECS, transparency with us users, and support for the broader community. Nice work @vis2k, and thanks for helping get us newbies off-the-ground!