Why vanilla ECS is not enough

I found this on reddit and I think it's interesting as I experience some of the problems that was written. The writer defined a new "ECS" which addresses the problems. While I don't think Unity will move this way, I find the ideas compelling.

To be fair, I've read about Unity's plans to add a feature where components can be enabled/disabled. This will address some of the pain points mentioned in the article.

 

Would've been awesome to learn about DataFlowGraph, since there's second to none documentation on the package's page https://docs.unity3d.com/Packages/com.unity.dataflowgraph@0.15/manual/index.html
Though I understand that it's a preview package. I guess we don't have Animation package docs for the same reason. Just wanted to point out that this package, even if considered low-level, may get handy for some of us and will need docs in the future.

 

I'm curious which of these problems you ran into, because Entities solves most of the ones I read:

• Hierarchies - TransformSystem (albeit it could benefit from static parent-child relationship optimizations)
• Component Sharing - SharedComponents and BlobAssetReferences
• Multiple Component Instances - DynamicBuffers
• Runtime Tags - SharedComponents with a unique identifier
• State machines - Depending on what kind of state machine you want, ECS may or may not be the best fit here
• Declarative Programming - Authoring and GameObjectConversionSystem (Unity has never really tailored to the declarative mindset)
• System Execution Order - ComponentSystemGroups

 

Hi, author of the article here.

> Not every problem should in fact be solved with ECS.

That is of course true, but that shouldn't stop us from thinking about improvements, especially when those improvements don't adversely impact all of the good stuff, like having a fast data storage.

FWIW, all of the suggested improvements have been implemented in a framework that storage wise is very similar to Unity DOTS (also archetype based), without giving up on things like being able to iterate over contiguous arrays, vectorize code etc.

One of the things I find appealing about ECS is that it combines high-level abstractions with efficient storage, and I'll want to use that efficient storage wherever it makes sense. The article addresses what I think are arbitrary limitations in ECS as it is commonly defined. By removing those limitations I increase the number of situations where I can still use this high-level API in combination with the fast storage.

Few notes:
> Multiple Component Instances - DynamicBuffers:
Sure, but at that point you're basically managing your own pools which is less efficient (higher fragmentation vs. having a regular component array), and you're no longer able to use regular ECS operations.

> Runtime Tags - SharedComponents with a unique identifier
What if I want to iterate entities that are organized across different chunks / archetypes? This may be my lack of understanding of Unity ECS, but I thought shared components are bound to a chunk.

> System Execution Order - ComponentSystemGroups
This looks good. My goal is to ultimately have a mechanism whereby you can import systems from an asset store, import them into your project and it should "just" work. I think ComponentSystemGroups are a good step in that direction.

 

@SanderMertens joining the discussion. I agree, DynamicBuffers do have higher chunk fragmentation and majority of the times hurt performance, also the problem comes with loading them up to a cache line when you load the entire buffer into the cache but only use few elements which indeed is wasteful.

As for SharedComponent Unity introduced ChunkComponent that acts more or less like you would like them to act. https://docs.unity3d.com/Packages/com.unity.entities@0.11/manual/ecs_chunk_component.html

The article is great and I am looking forward to flecs v2 and all what comes next. Keep up great work!

 

I'm not sure what you mean by "managing your own pools". But I think it is good that there is an explicit difference between component types where only one can exist and where several can exist. It allows for the code to make meaningful assumptions.

It is more like shared components bind entities to chunks. By assigning a bunch of entities with the same shared component, the entities all end up in the same chunks. Then they can all be fetched together. Obviously this isn't that useful if the entities are changing shared components frequently. But I don't find that use case to be all that common compared to entities classified at spawn time.

 

Multiple Component Instances - DynamicBuffers
How though? Say component A and component B needs a separate Timer. You store the Timers in the buffers then assign an index in the components? That's clunky. My current solution to this is to use a separate entity with Timer then assign this entity to the components.

Another problem is that you can only add one type of Dynamic Buffer. If you have different components that need some kind of container for the same type, you're left with creating separate entities for the buffers instead then use these entities in the components as reference to the buffers. I may have fixed the data modelling but the nagging question now is "Is this really the right way?"

State machines
Anything that needs a dynamic ordering of execution is not very intuitive in ECS (command pattern, AI behavior). It takes some hoops to implement either by tag components or flag checking. Not to mention the fixed system execution. Say you need to run B then A preferably on the same frame but systems were ordered to run SystemA then SystemB. So B will be executed this frame but A will be executed in the next. This may seem like a non problem but considering that a lot can happen between SystemB and SystemA, it's a recipe for hard to find bugs. What's your solution to this? I've thought about using OOP for these kinds of problems but my issue is that accessing/modifying data by EntityManager is slow or not suitable to do for every frame.

 

Why not put the 'timer' in each component ? Yes you'll need to have one system to update the timer of each type of component having a timer but that would be I think the most runtime efficient way of dealing with it. additionnal htoughts regarding timmers, I think it would be better to have a timmer logic that store the next tick time instead of storing aremaining time that you update every frame. It would allow to make use of the DidChange filtering. Also you could probably define a ITimer interface that your ComponentData would implement and taht should allow you to stream line the timer update systems a bit through generics.

 

That doesn't scale well though. One, you'll be adding overhead to every component you want to potentially add a timer to (which also requires foresight about which components need timers, ever, but let's put that that aside for now).

Secondly, an expiry timer is only one kind of generic "trait" I could apply to a component. What about a trait where I want to add a component after N seconds? What about a generic lerp system, or a system to store the last N values of a component?

 

IMO, either the component need the timer behavior or it does not and if you have case were the same type of data sometimes need the timer and sometimes don't, I would argue t's not the same type of data and therefore would justify 2 components.

For the second point, I would probably go with reactive systems or a system that implement a native stream/queue with a delay mechanism.

Each problem as it's solution, not everything should be components.

 

That is fair- though the solutions you propose all perform worse on any metric but one: that we don't have to change ECS =p That is a perfectly reasonable reason and conclusion, but the premise of my article is to allow for those small tweaks so that we can still do such things in a clean DoD way.

 

For the timer case I don't see how it performs worse on every metric. Regarding the memory footprint having the timer in the component itself avoids having to store any reference to a third party structure (buffer or entity). This also avoid having to do the indirection towards the other data structure so less computation.
The fact that they are properly alligned in memory allows for quick loop through with burst.
The only less performant aspect of it I can see is teh convinance of build/use, you'll need to define more component and more systems instead of having a single system manage all your timing, but even with that in mind I don't feel that manipulating indirection every time i need to operate on the timer is convinient either...

Granted I did not test both solutions but intuitivly it make sense to think that less work mean more performance and less indirection mean les memory usage, so...

I feel like your view of ECS involves a lot of 'pointer' and indirections. I don't know how it will impact your data layout.

I don't consider myself an expert in hte matter at all but ECS and DoD are two different things IMO and maybe to have great DoD implies you have to sacrifice some of the conviniance of more abstract approach.

 

We use this approach for ai behaviors on entities, and it’s worked very well. I wrote a custom job type to automatically execute all ai behaviors which are stored in a buffer like this.

It’s been extremely fast. And the job abstraction also made it intuitive. We also use plenty of state machines. What do you consider clunky about the approach?

Solving this problem has been a major focus of ours since starting to use DOTS. This was kind of a litmus test for us - we had to make sure ECS was capable of performing both behavior patterns A,B,C and C,B,A on the same frame, with strong performance. It absolutely can be done, without resorting to OOP. Seeing the results made DOTS believers out of us.

 

How did you solve this? I'm all ears. I tried converting our GOAP into ECS and the resulting library is kinda hard to use as I relied to adding/removing tags to order action execution.

 

I am heartbroken to say that this goes into proprietary tech that I cant share here. I’m truly sorry to have to say that, @davenirline. Maybe one day we will be able to. GOAP should work with our solution, and we have a pattern in mind that’s on our schedule for later in the year. Maybe we’ll run into some of the same issues you did.

 

I'd probably use a timepoint approach and also store the earliest timepoint in a component so that I only have to iterate over the buffer whenever the next timer expired.

You can have different types of DynamicBuffers attached to a single Entity. Unity already does this. Most entities with hierarchies converted from prefabs have both a LinkedEntityGroup and a Child buffer. Unless you are talking about packing different types into the same buffer? In that case, you may consider unions or some clever use of BlobAssetReferences. But really that is more of an OOP concept anyways.

I explicitly order my systems hierarchically to ensure things operate with as little latency as possible. https://github.com/Dreaming381/lsss-wip/blob/master/Assets/_Code/SuperSystems/AiSuperSystems.cs

 

Your solution performs worse, because even if we forget about the memory overhead for a second, you will have to evaluate every component to see whether the timer should be progressed, regardless of whether the timer is active or not.

I haven't actually explained how my solution works, and I don't want to go into too much detail here, but you can probably guess some things from this code: https://github.com/SanderMertens/flecs/blob/v2_in_progress/examples/c/42_traits/src/main.c

TL;DR: the timer component is stored as a regular component array, just like any other component, that you can iterate like a contiguous array. There is no indirection. You iterate the components, update the timer value, and when the timer expires you delete the component the timer is associated with. The one thing that makes this different from a regular component is that I can add it multiple times to an entity.

So to summarize:
- You can iterate N timers per entity as a regular component array, which is just as fast as the fastest kind of iteration
- You only iterate over timers that are actually active
- You don't incur additional memory overhead besides what is necessary for storing the Timer components
- Because of the above, there are no indirections. If it weren't for the check on whether the timer is expired, this code could easily be vectorized.

 

Just want to say few notes

OOP and DOD is not opposite thecs the can leave with each other. Even more OOP can be just way to use DOD.

Do any one know that Unity was always be ECS? every MonoBehaviour is actually C and S, GameObject is E
and under the hood every thing work exactly (mostly) like new ECS tech from Unity (Entities).

I want to say that actually nothing changed new tech does not change anything in a way we make code.
They propose new way and it is right way if you want best performance but no one stop you use Entities Package in OOP way. And even then you can have better performance, forgot about pools and GC and just be somewhat happier.

We just do that in our project. added few extension methods for entity like GetComponent, SetComponent, HaveComponent etc to mimic GameObject world and easy translate OOP project to Entities and nothing changed in way we write code. Yes we absolutely dont use DOTS potential I just want to say that

DOTS is not DOD.
Plain C# id not OOP.

DOD is how you use tool in your hands OOP is the same.
DOTS is just new tool from Unity. GOOD TOOL

If you say want to store many timers on one Entity.

Old GO tool naive design:
1.Add many Timer Components to GO.
2.Add reference to exact timer to Components that need timer.
3.Every timer component is heavy thing with big
(more than 64 bytes) overhead just for empty MonoBehaiour and it is in random memory and GC and

New Entities Package from Unity naive design:
0.Create ComponentDataArray<T> that will help store regular TimerComponentData struct on Entity many times
1.Add many Timer Components to Entities.
2.Add reference(handle) to exact timer to Components that need timer.
3.Every timer is cheap structure may be just 8 bytes all timers of one Entity can lie near entity no GC and this is
not best performant way it is just OOP way on Entities with all benefits of new Unity tech

 

I understand know that you want to add some sort of experiation time on any component.
In that case I agree adding the timings data to the compoennt itself would not be a good pattern.
In current ECS implementation you could probably have a system maintining a dictionnary of <Entity,ComponentExpiracy> where ComponentExpiracy contain the time of expiracy (not the remaining lifetime but the time at which the component should be removed so to avoid updating the data every frame) and the component type (or type id) when the expiracy time is reached, issue a remove component ecb command and remove the entry from the dictionnary. This would need additionnal work to make it run in parrallel in a burst job though.

Edit : Having a dynamic buffer of ComponentExpiracy should allow the samebehavior.
Iterating over them should be fast has you can have a bursted system iterate in parrallel over each entity's dynamic buffer. And the iterating over inactive timer is a non issue as you would remove both the component and timer one the timer is expired.

 

But in all seriousness, I kinda miss OOP features in ECS... Just bringing one concrete example ruins the entire concept of ECS for me: state machines.

Of course you can make state machines in ECS and have one system berbe for processing a particular state, but what if I want to have multiple states processed during a single frame? What if I want to process state machines in a particular order (for example, have the Player's state machine be processed first)? I can't do that in pure ECS, at all.

Now, of course, it's possible to use class IComponentData and at first it felt compelling to me. Wouldn't it be great to export some of your gameplay processing to class components and in turn get something like a MonoBehavior for Entities? Because there are arguably instances where it's definitely usable.

But then there are two concrete problems with such an approach. The first one is that it's quite a bit of a pain to use, you always must generate equality/hash functions for your components. And also the overall performance of constantly getting and setting struct component datas is terrible, some kind of an api to get a pointer to a desired data would be great in my opinion (instead of relying on get/set methods, which gets that pointer behind the scenes, copies data to a struct, returns it, then we modify it and tell the entityManager to set it back by copying to that address again).

 

How would you design that with GameObjects?

 

Each state machine is processed in order. After a StateMachine finishes processing a state, it can check for a bool or such that would force it to execute the current state again.

There are definitely usages for such a weird thing. Maybe you want to transition to another state and have it executed this instant? But no, such a thing is seemingly impossible in pure Unity ECS.

 

Why is that impossible? Couldn't you iterate over the same set of entities multiple times in a system?

 

For a state machine, simple IComponentData should be enougth, I may even consider a reactive system with a IStateComponentData depending on the case.
For successive change, iterating multiple time in the same system has the issue of not knowing how many time you need to iterate.
The state machine should not have a reason to change state several time in a single frame IMO. If it needs to it means you are missing a transition. You can probably solve that by extracting pieces of behavior in separate methods and compose the expected behavior based on the transition (bigin and end state).
If you need several kind of states (one for combat, one for movement,...) you can make several component and system.

 

Yes, that is correct, I also believe that having a state machine run several times in a row is not ideal. There may be cases when it's absolutely needed though, but I think that would be more of a developer's problem. There's also one more problem with it with transitioning to multiple states over a frame. In OOP land I can transition to another state and, in its OnTransition method, I can decide if the transition is valid and if not I can transition to another one. This may be harder to do in Unity's ECS because now it's a current state that decides if another one is valid to transition to and I don't believe that's a good deisgn. Maybe I'm too inexperienced.

However, my point about the execution order of a whole state machine still stands I think. If I absolutely need to have one state machine and its states processed before another one and both share the same states, then I'm out of luck, only because its whole systems that process states individually.

 

You can order your state machine (systems) with system groups order before/after and job dependancy.
Nothing prevents you in you state machine (system) to invalidate the state transition. You can have one change state request component that says to which state you wish to transit to and your system logic either update the state component or not.

 

Then I'll need to run my systems several times in a row and keep track over which components should be processed 2nd/3rd/4th/etc... time and which shouldn't. Constantly adding and removing tag components for that purpose is not performant at all.

No, if I need to update one state before another one and both are the same type, then I have no guarantee about the execution order. I can mitigate it by making two systems and a tag component, for example, but it feels like a major hack.

And regarding the transition request component, yeah, that'll work, but what if I want to have a chain where I transition to multiple states in the same frame? For example, I wish to transition from state 0 to state 1, state 1 immediately finds that it's not eligible so it transition to state 2 and so on. Making this chain and accomodating for every possible scenario in a single system for state 0 or state 1 is unreasonable to me.

 

That's your state machine logic, if you can access a state only coming from another specific state then you check that when the transition is requested, if it's not allowed, don't do the transition.

Same thing here, your state machine should not allow to transition to a state it's not eligible, either don't do the transition or do a default one.

 

I'm able to achieve this in my current project. I'm using a system and a tag component for each states. To make things more reusable and modular I made

StateSystemBase<T>

abstract class and

Transition<T>

component type, where T is a

IComponentType

representing a state. Basically they handle

OnUpdate

for their state component types and they handle state initialization and deinitialization.

The order of which state component type to process first can be easily defined with system attributes. I'm not sure about why the order of state updates of the same state type matters.

And for multiple state changes within a frame, I've made

EndSimulationStateTransitionBufferSystem

(stole that name from ECB) that buffers every transition changes in the frame to (if it's not empty) play it back at the end of the frame. This is where the sync point is made and all structural changes happen so no other jobs will get disturbed. Every

StateSystemBase<T>

will be notified of this change, and using

EntityQuery

to check if a system need to rerun its initialization is extremely cheap.

If within the notified

StateSystemBase<T>

initialization process a new transition change is buffered to the

EndSimulationStateTransitionBufferSystem

, then the same process will run again until the buffer is empty.

 

Don’t be sure something like this can’t be performant...I’ve seen it be very performant. Looks like yondercode has too.

I would test it to make sure. Testing our obvious assumptions about perf has made us much better DOTS users over here, because we were often wrong about what we thought would be fastest. This is a good example.

Also - if an approach is the only way to do something, then it is also the fastest way to do something.

If after weeks of searching, no one can think of a faster way - as long as the known solution performs well, sometimes it’s ok to move on for the time being. You can see that kind of thinking all over the place in Unity’s own DOTS source, and in every game engine I’m familiar with.

 

I am not sure i get this. You sure you are doing State machines right? Why are you transitioning to a state preemptively without validating the transition first? This is a basic principle of FSM. And i mean not for OOP or ECS or anything. this is weird if not plain incorrect logic flow.

 

It was just a hypothetical question regarding cases when it is absolutely needed, like generic state machines where the validity of a transition is not known before the transition itself. I know perfectly well that it's suboptimal, I just want to stretch the abilities of jobified systems, because I'm not quite sure if it would be up to the task for such corner cases, and no project is safe from those.

 

Tweaking ECS to handle state machines more gracefully won't mean that we can build all state machines in ECS. It just means we can build more state machines in ECS.

 

I agree with OP's linked post where it talks about parents and entity links - having to repeatedly index into arrays feels awkward, I miss python's getattr mechanic but wouldn't know how C# could let us write

entity.parent.component

. It does belie the fundamental purpose of an ECS to have that use case in particular, but hierarchies, especially search trees, graphs, are ultra important structures that are rather painful to implement right now.


However - the first thing I miss most in DOTS is a reactive system that triggers when a component is added or removed, at a defined point in time, regardless of when and where that component was added. (not immediate - more like "at start of frame, run for all entities that had their Translation components removed, ideally exposing the last value at the point of removal). In fact a "remove" flag on a component would help here and you could run a system that processes these. Sadly Unity opted for an interface instead of inheritance for components, so all the inbuilt components can't do this without an extra "TagRemoveTranslation" component.

I kind of mimic this with queries, but it never feels like it's reliable because the OnUpdate method is so opaque (or rather, what's inside it - the Entities.ForEach), and changefilter seems to apply to chunks, not entities.

ECSes like Entitas do this way better, but obviously don't go into native generation as deeply.


The second thing is - we absolutely need syntactic sugar that obviates EntityManager and SystemBase's version of EntityManager's accessors, i.e. as each Entity should know which manager world it belongs into, the following should be possible:

myEntity.hasTranslation

or

myEntity.has<Translation>

being an automatic property that is true of myEntity has a Translation component; and

myEntity.translation

or

myEntity.get<Translation>

returning that component's value, and providing a setter for it if and only if we're in a context that allows writing at compile time (not build time or run time).

Furthermore,

myentity.AddTranslation(float3 ...)

are prime targets for code generation after the above. The context should know that was created as a structural changes context, and delegate to the appropriate entity command buffer (chosen by the user, but with a reasonable default).

 

How does Entitas do this?

I have found times when I wished entities stored information about their archetype Or chunk. But internally, that would mean a pointer, since the archetype could change. So that could hit perf pretty hard, depending on what you’re doing.

Something like an “ArchetypeForEntity” struct could let us choose when we care about that data, without adding it to all Entities. I support keeping Entity super light, with only the data needed to use it as a key.

How would this work in Jobs? It would choose an ECB for you? That doesn’t sound good. IMO, syntactic sugar is only useful when it doesn’t obfuscate important information. Personally, I think Entities.ForEach comes uncomfortably close to the line sometimes.

 

This has several problems.
If you have a big game with around 200 components, the entity will have for each component an property and Add Method? With extensions you can only add methods, so you need at least 6 methods: add, remove, set, get and set/isActive. With 200 components you have 1200 methods. And with a type pro archetype you lose flexibility. It's no longer ECS.
Next problem, each entity needs to know his entity manager, and so far i know the manager is an struct, so with 200,000 entity you have 200,000 entity manager in ram. And with an reference to the World, you lose Burst compatibility.

The entity is not an container and shouldn't be used like that.

 

This is possible now, using SystemStateComponents, as you described. Especially if these system state components are empty tags, the cost shouldn't be high.

But it should also be possible now without using an extra component. Add a "ToRemove" bool to the component you want to track. Set it to true when you want to remove it. Have a system check for this flag, which runs before the ECBSystem that actually removes it. That would also give you access to the component's last value.

In the next Entities release, the Enable/Disable component feature should offer a cleaner, more performant way to do what's described above. Rather than adding a "ToRemove" flag explicitely, just disable the component to signify the same thing. Then filter for entities with disabled versions of that component (before a later ECBSystem actually removes them).

 

In general, I think convenient syntax shouldn't be the highest priority goal. The best goal would be the highest level of convenience possible that doesn't require sacrificing significant performance.

Ideally, you'd have a solution which scores very high marks in both performance and convenience. HPC# is a great example of a team trying to do this.

But if one consideration has to win, it should usually be perf (as long as the convenience isn't so low that it's unusable).

 

Generic reactive system, you just have to extend the system and add your component as the generic parameter (it need to be registered in hte assembly for burst to work).
https://forum.unity.com/threads/reactive-system-generic-way.919997/#post-6042521

 

This will not allow the system to run only once per add/remove. you will end up iterating every frame on all disabled entities, except if you ad some sort of tracking (bool or tag component) to say that you already processed the removal of the component.

Also I don't know how the new enable/disable will work, will we be able to diferentiate from the query that a component is there but disabled or that the component is not htere at all.

 

I was imaging a scenario in which you only disable the component when you want to remove it:

1. In System1, you'd disable the component, and also queue it for removal with an ECBSystem.
2. In System 2, you'd filter for any disabled components of that type. This is the "reactive" system.
3. System 3 is the ECBSystem, which actually removes the component.

- - -

As you smartly pointed out - since it's not out yet, it's hard to know. I'm guessing you could also do this even if you don't reserve disabling for only that purpose:

I'm assuming we'll be able to combine a change filter with an enable state filter in an EntityQuery. That would let you achieve it.

...But you know what they say about assumin'...

 

There are already system that allow run once for every added removed component. You can do this using SystemStateComponentData and Unity plan to make this semi auto so you can just create Query for this and under the hood Unity will generate those SystemStateComponentData that will track adding and removeing of components.

There is post about this from Joachim somewhere on DOTS forum

 

I think you are talking about.
https://forum.unity.com/threads/new-enabled-disabled-state-filtering.902531/#post-5933756

In the mean time, the post I did and referenced above should ease the process for now.