To ECS or not to ECS

I am professional programmer but a Unity hobbyist. I have an idea for a game I'd like to make.

It would look much like the sprite baed space shooters of old that used to be in the arcade halls. It will feature a lot of objects floating around in space that can be interacted with and/or blown up. There would be a bit of building like say Theme Hospital and a bit of resource gathering like Starscape. Actually Starscape is a good reference of what look I am aiming for.

So, now comes the questions; for this new, non-deadlined, hobby project, do I dive into ECS? A technology that shows a lot of promise but that I have very little experience with. Or, do I stick to the MonoBehaviour techniques that I know?

On one hand, new tech and the promise of easier scaling and the insane performance boost that ECS delivers or promises to deliver are very attractive. On the other, the tooling isn't quite there yet and the amount of work to do just to get a few sprites working is quite extensive. ECS seems to be the future, but is that future ready to be used yet?

 

That video really helped me. Showed me that the way your thinking about it which is the way most people, myself included, think about DOTs is wrong when it comes to whether to learn it or not and when to learn it or not.

You dont have to commit to it at all, you can incrementally add bits of it to your current projects and reap benefits all while learning it incrementally.

Follow that video and I am 100% certain by the end you will have a good idea of how to start immediatly, as well as have ideas for bits in your own projects that could use it.

In short: yes its ready to be used, and its also 100x easier than you would expect due to the new workflow detailed in that video.

Good luck

 

I would bang the prototype in classic way first. Ensuring prove of concept. It may be faster, if you are comfortable with GameObjects.

Then, look later, how to transfer relevant bits to jobs.
You grab that concept, then you can convert to DODS.

What you can do, if you are programmer already, start from Data Oriented Design, rather strictly focusing on Object Oriented Programming. Will be easier to convert later.

 

You dont need to commit to ECS 100 procent. In our game we use it were it makes sense. For example our AI is being rewritten to it.

The core domain is still classic component driven and probably will not we converted to ECS, atleast not during the lifetime of current project

 

If you plan to have lots of objects flying around in your scene, then using ECS instead of using a game object for each projectile makes sense. For example, if you plan to have thousands of lasers flying around in your scene, you want to use ECS or something similar to handle those projectiles in a scalable way.

You don't have to use ECS specifically, though. You can write your own systems from scratch instead of using ECS, but ECS is generally a better option at this point. For example, I released Disputed Space in 2017 using Unity 5.5, and I build my own data oriented system from scratch that used DrawMeshInstanced. ECS was not an option when I built that game, so I developed my own system. If I was starting a new project today, I would probably just use ECS. If you have never built something like this from scratch, then you should definitely use ECS instead of trying to design your own system.

As for pure scalability, using a separate game object for each projectile will never achieve the level of performance you can get with a data oriented system (like mine or like ECS). Data oriented systems are often 6-10 times as scalable as a typical game object pooling system. By using ECS instead of sticking to the MonoBehaviour techniques that you already know, you can get that 6-10 times scalability improvement.

 

Thanks for those answers. My background is math and data science. Work revolves around Python and R. I play games and I know a about game theory in the math sense. ECS and data oriented is really enticing.

 

Awesome. You should definitely try ECS. There will be some initial effort learning about data oriented systems, but it will definitely be worth it. Enjoy.

 

Won't using ECS for projectiles force you to use ECS for a lot of other things? If you have a player Game Object with a rigidbody and a collider will it still be able to collide with an ECS bullet?

 

It depends. For example, some projectiles are easier than others to implement. For example, a laser only collides with other things. Nothing can collide with a laser, so a laser does not need its own rigidbody or collider. But a large slow moving torpedo might need to be able to accept collisions in addition to colliding with something, like if a laser shoots a torpedo to blow it up.

Another example would be moving asteroids. If there are thousands of asteroids in a scene, then ECS would seem like the answer. However, a detailed asteroid simulation would need a rigidbody and collider on each asteroid. To make an ECS based asteroid field, you would need to implement some kind of manual collision check within the asteroid ECS to fake some of the physics. But if the asteroid field is merely a visual thing in the distance, then it would not need any collision code and could easily be implemented in ECS.

I think there has also been some progress on ECS based physic this year, so always check with threads about the status when working on this:
https://forum.unity.com/threads/unity-physics-discussion.646486/

 

It doesn't force to anything.
It depends how you want to use it.
Remember to use right tool for right job.

For rendering high volume of objects, for intensive AI, for many algorithms and more, DOTS with ECS is perfect.

But for Characters kinematic ragdol, is not yet suitable.

These are juz some examples.

 

Depending on how many "a lot" is, I'd be doing this data oriented anyway. And now Unity have a built-in system to standardise that, so it sounds like a clear win to me.

Of course, if "lots" is really "a few dozen" then I'd go with whatever is faster and/or more fun to work with.

 

I always think it is important to prototype quickly, actually I follow an agile scrum type model.

Normally, the code you throw together is unorganised and difficult to navigate, but it is a proof of concept phase. Somewhere down the line you need to consider adopting good practices to modularise your code, use higher level concepts to improve performance and maintainability.

E.g. I'm looking at State management + concurrency now (unrelated to unity) and it's amazing.

 

Just know that you need to throw the prototype out. Too many prototypes stick

 

That was a truly excellent video. Before that I thought that whole convert to stuff was some sort of crutch. But it's actually quite effective it seems.

 

Especially under tight deadlines and funding issues!

 

Yes it too changed my perspective on the conversion workflow, it seems like it will be the defacto way to author ECS whilst retaining the speed and ease of prototyping that unity is famous for

 

Personally I don't think you need to try the whole DOTS thing if you are not doing something that needs a lot of performance optimization. I mean, you have to learn new stuff, the coding for it is still fairly complicated IMO, the whole job system is still not as easy as it could be, it takes more thought to plan your data structures and stuff etc. I would think actually MOST games don't really need it. Especially not any kind of 2D game unless you're trying to do a heck of a lot of pixel manipulation or something.

 

Did you watch the video above? The coding is certainly not that complicated anymore, it basically is the same amount of code too now, DOTs has changed a lot recently and its probably not like you remember based on what your saying. It certainly is not just for "big things", which is something they address in the video.

Performance is good for performance sake, and as software developers we should be trying to pick the most performant option vs time consumption to implement it.

Now that DOTS takes no longer to author than monobehaviours and is convertible from one to the other, you should be picking the more performant option where it makes sense. The video highlights that not everything is good for DOTs, but almost any project has at least 1 or 2 places it can and therefore should be used.

 

^ I don't necessarily agree with that. Sure it is more performant but async coding is harder to grasp / debug, so your developers need to know what's the deal.

I'd still opt for the old way for smallish games IMO.

 

I think small and simple games are good in terms of opportunity, to learn DOTS principles.

 

There is nothing stopping you using a hybrid approach. Do all your physics with MonoBehaviours, then do all your game logic with ECS. Or vice versa. You can have a MonoBehaviour and an ECS both affect the same GameObject.

 

Just wished the Hybrid Renderer worked already. Keep hearing "Any Day Now" for an update for weeks. Was told I can't do entities without Hybrid Renderer as it contains the rendering components and such. (on the beta).

@Kiwasi : I read that using physics with entities can't collide with standard colliders from a game object. Is this not true? Because I'd prefer to use a hybrid approach - simply use projectiles as entities and everything regular game objects.

 

Someone else who has played with it more then me will have to respond.

My assumption is that the two physics systems will not interact. But I could be wrong.

But there is still no reason why you can't make every physics object object in ECS. Then just add the regular MonoBehaviour components to the ones that need to do something other than just collisions.

 

The old way had async programming in form of Coroutines

 

Appreciate the insight.
Hard to test it when hybrid renderer isn't working with 2019.3 beta lol.

Making an RTS and figured all the projectiles would be better suited using ECS rather than standard game objects.
It's going to be a unique subgenre of it, MMO RTS max 250 players. (Granted players will only be able to build at most like 25 units), so ECS will be pretty important. I know the nav system will be getting updated sometime in 2020 version so not really stressing 6,250 units max lol. But projectiles will be the killer I need to focus on.

 

You can make the two interact but manually. As in rigidbodies wont automatically affect the ECS systems, but you can regularly do physics checks from ECS land against standard colliders etc and react to the results

I mean I wouldnt call async coding difficult to grasp, but I guess I am not thinking about this from a new persons perspective. However OP did mention they are an experienced programmer so I doubt the concept of async will confuse them.

At the same time, not doing something that is arguably more performant because you find it hard is a good use case to actually go and learn the hard stuff, rather than cobbling your applications capabilities. And a small project would be easier to learn ECS with than a big one.

I was super cautious about ECS and hadnt really messed with it properly until this month, but now I am a convert and realise that its waaaaay easier than the docs make it look like. Honestly its not that much more "difficult" than normal unity scripting. In some ways its easier as its closer to normal programming compared to unity scripting which has always had its quirks compared to standard c# etc. If you have ever written a multithreaded application, a lot of the concepts of ECS are already things you have done such as operating on data in tandem

 

You can also handle the rendering using DrawMeshInstancedIndirect. Use ECS to drive the projectile logic, which writes to NativeArrays of position and rotation. Then on update, copy those to a compute buffer and then on into the shader. In my experiments this was much quicker than the hybrid renderer.

 

Yeah, I mean I remember when async was pain because of callback hell (I'm looking at this from a Javascript POV I haven't honestly touched unity and c# in months so I can't comment)

Now they have async + await which makes it easier to read. With flutter it took ages racking my brain trying to understand state management but now my app seems organised.

Async progging still confuses me sometimes, but it is slowly sinking in.

Personally, I understand why people would want to use it, but I still stand by my POV that sometimes the old synchronus way is easier to digest and quicker to make prototypes.

 

In this day and age, I think you need to be up to speed with knowing async progg'ing so it is a good thing either way.

 

Hey I am all for everyone coding how it makes sense for them

That said, I was totally of the same mindset like no less than 2 weeks ago, I am pretty sure as ECS learning material starts to come available and you have a practise with it and get a hang of it, youll probably swing around too Its a totally different way of going about development which makes it difficult to understand how it could ever be faster (at least thats how it was with me), but now that I understand it I am making prototypes way faster than I ever have right now and nice thing is because ECS is all about tiny generic systems, a lot of the prototype code is reusable which usually isnt the case (or shouldnt be the case in rapid prototyping)

Then again maybe I am just lucky and its "clicked" with me and perhaps it will be more difficult for others, sometimes I find it hard to think like other people

 

Plus, remember that people do that "hard stuff" because it some things easier.

 

Yeah I agree, rapid prototyping is like a S***storm, but it is more a proof of concept, it takes time learning these higher level paradigms, but when you do it's great.

 

For those that are interested I'm working with Flutter and provider here is a good video on State management and one on Async programming

All I'm saying is these concepts are high level and difficult to grasp initially, so sometimes it pays to just get things done in a non-organised, non abstract kinda way.

 

Thanks for the tips, @GameDevCouple_I & @MrArcher

I think I'll give the hybrid renderer a bit more time before going to trying other routes, assuming it'll be released at or right before 2019.3 official release.

 

N1warhead, Classic GameObject physics uses PhysX and DOTS uses custom solutions. So in normal fashion they are not intractable. Unless you make some form of interface.

But to be honest, I don't know why you need that for RTS. DOTS physics offer already all you need, for RTS case. In fact, you probably don't need physics at all. Unless you make physics driven projectiles and moving units.

 

@Antypodish : yeah that's what I figured honestly.

Yeah, that's perhaps true I suppose. I'm sure there's all sorts of ways I can make it work without any physics interactions at all actually. Such as doing simple distance checks, within distance just destroy the entity(missile). But then - how do you damage the entity if you don't do any form of physics check.

I've only ever used Jobs and Burst (love it by the way). But Entities haven't had the chance. So what they can and can't do yet is sorta out of my realm.

 

@N1warhead
For such problem, I suggest watch Sepllout video from Unite Austin 2017.
It represents good concept on the topic.
Also discussed already many times in our DOTS forum.

 

You think a beginner to unity is going to be able to set up a DOTS project and code it correctly? Absolutely no way. That's why I said it's too complicated still. Until it's all integrated automatically into the user interface of the editing process with high level tools, it's not accessible to beginners or non-programmer artists. The whole component and gameobject-based system needs to be replaced with it internally, this whole optimization/performance thing should not require ANY end user to have to make decisions about data storage at all. All this stuff should be behind the scenes and hands-off and automated. Not the kind of thing humans should be doing.

 

@imaginaryhuman who said DOTS is targeted for beginners?
Is targeted for performance.
And please mind, DOTS visual scripting tool is in development. This is as far as beginner tool can go in my opinion.

 

Trying to work it out now actually. The async part is not really worrying me. Am I actually correct in assuming there is no spriterenderer out of the box yet? Cos it's all fun and chaos as long as I use meshes, but sprites immediately show great swatches of emptiness, even if the entities are nicely there in the debugger.

 

Just select it as a texture type instead of sprite and apply it to a quad using the hybrid renderer. Not going to be visually different at all to user, only you will know Itll still display as flat and you can then use it as you would a sprite

 

One side note regarding hybrid approaches is to be careful with memory allocations. I am a firm believer in zero memory allocation per frame. Per frame memory allocations can eventually lead to weird microstutters in otherwise fast running games. Anyway, I experimented with using jobs based RaycastCommand to batch groups of raycast checks in hybrid code. While this did offer measurable performance improvements on systems with lots of cores, this also created some unavoidable per frame memory allocations.

I could not find any way to use RaycastCommand in a hybrid system in a zero per frame memory alloc way. It is possible that Unity has not built a zero memory alloc solution for hybrid systems, since they know many serious devs will choose a pure ECS solution instead of a hybrid solution.

Anyway, I just want to put that information out there to remind everybody to double check per frame memory allocation using the profiler when using a hybrid approach, because it is very possible you may be causing memory allocations per frame and that will eventually cause other issues.

 

The answer to your question is really premised on knowing Unity fairly well. The best way to approach this is having some idea of the performance characteristics of the features you are using and which ones will likely be performance bottlenecks. Without that knowledge you really have no context for where the value add will be using DOTS. Better performance isn't really an issue if performance isn't an issue to start with.

Jobs + burst are the two biggest bang for buck features. Everything else just adds more incremental value. Jobs allow moving work off the main thread, and burst is just a huge ass performance leap. After that ECS is a significant boost but burst really overshadows everything else performance wise.

My suggestion is for every major feature you are using put together some test cases at the scale needed. Get familiar with the performance profile using standard practices without DOTS. Once you have that you will then have a fairly good idea where it is DOTS would even help you.

Now you have something concrete and your DOTS exploration is a lot more focused.

 

Does it need to be "beginner" friendly? Obviously I'm biased, but I'd rather a system that requires some knowledge and skill and enables excellent results over one that does everything for you and gets ok results.

That's not even getting into the whole thing of "beginner" being a super broad term...

 

I am a complete unity noob and the results of ECS, Burst & Jobs sounds amazing.
My weeks experience of unity is progressing well but with ECS I have come to the decision use standard objects and then just start moving things to hybrid ECS when needed as a 2nd phase.
I have been blown away how easy the standard component model is and keep reading how the abstraction of ECS makes things cleaner, easier to read and understand.
Seriously doesn't for me

 

I reckon this is just because of its current chunkiness by being new. Once its done and implemented and the net is drowning in tutorials, I doubt getting started with DOTS is going to be any more difficult then getting started with MonoBehaviours.

 

This. It's important that you pay attention to the Unite Copenhagen videos or whichever videos are the latest when you go to learn Unity DOTS because it's changing for the better with each new release and the available learning resources don't show this at all because they're simply too far out of date.

 

Yeah, I am not sure how anyone can claim its harder for beginners than monobehaviour after video I posted above (which you actually posted me originally @Ryiah ) its literally same number of lines of code and you set up gameobjects to do everything. What more do people want, to be able to speak to the pc and it write your game for you on the new system?

Its literally as simple as monobehaviours now and I have a feeling people in this thread who are saying its difficult have not watched that video end to end and given the things talked about in it a try, yet are still trying to make claims about the system. And no, having used ECS a couple of months ago does not count, everything is different right now compared to then.

 

Once I really dove into the Jobs system, everything was much simpler to understand, all it takes is getting out of the trained 'object' programming mindset, it's pretty hard when that's all you're used too, but once I gave it a real firm chance - it was so easy to understand and use. But it took a mindset of opening up to it, otherwise you're always going to hate the workflow because it's so alien to what we are accustomed too.

 

Its still object oriented programming, but it emphasis data driven composition over classic class driven inheritance. You shouldnt do the latter in any software nor business nor games.