Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

EntityCommandBuffer - Will we ever see improvements?

Discussion in 'Entity Component System' started by Enzi, May 25, 2021.

  1. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    954
    I never had much trouble before with the ECB but currently I have to create a lot of entities, components and buffers from code that have relations to other entities.

    Some problems with ECB have okay workarounds but the simple fact that you can't create an entity and store the reference or know the entity id, forces me to write code in a way that's jumping through 3 hoops often ending in, okay, it's not possible, I've to think about a completely different way.

    So my question, will we ever see improvements and a fix for the odd behaviour that is creating entities with the ECB on main-thread? Something smarter than "patching"?

    I get why it's impossible to know the entity that will be created in threads but on the main-thread it should be perfectly predictable. (Okay, even in threads it should be possible if you put in the engineering work and the performance hit)

    And while we can use the EntityManager, the fact that create, add/remove comp and add buffers causes invalidation, it's not a good solution for the problem. I've tried it and given up on a pretty simple script which involved 2-3 linked entities with buffers. The amount of re-acquiring buffers leads to code that's just looking desperate.
    The ECB results in very readable and nice code but if you need to know the entity id NOW, you're screwed and there's no work around I know of which is a pretty glaring flaw in ECS that no-one seems to talk about.
     
    davenirline likes this.
  2. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,753
    What's stopping you? You can store the reference on another entity simply by doing a SetComponent or AppendBuffer with the same ECB.

    However if you mean store the reference in a native container I'd probably want to know the reason why.
     
  3. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    954
    edit what i'm doing: I have a set of entities on a grid which get processed to form a path with an ordered LinkedList, this path then gets translated into an optimized and merged path entity. The LinkedList is inside a dictionary with its own id and doesn't fit into ecs that well. This is all fine but updating paths start to be a problem because i don't have any references from the entity path to the LinkedList. I can bring the LinkedList into ECS or store a reference but I'm honestly not here to talk about workarounds but to give my opinion how this simple flaw leads to coding around the problem.

    > However if you mean store the reference in a native container I'd probably want to know the reason why.
    Is there a way to patch a NativeHashMap or Dictionary?

    And the question prevails, why is it not possible or never been addressed? It's unintuitive and restrictive.
     
    Last edited: May 25, 2021
  4. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    There is no amount of engineering that would get you entity references before they are created that wouldn't have some major issues with it. Commands can fail. Any scheme that tried to say pre assign id's would be bad because that would bake into the design that failure results in bad data. Sure you could concoct a scheme to then patch that up, but that's just making an already broken design worse.

    What you seem to need is the batching api's. You can make entity creation extremely fast using those.
     
    mikaelK likes this.
  5. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    954
    I've found a simple enough solution for my problem.

    See, as I'm grouping together entities, I can't have this group also exist as entities in ECS but in the system, as I need two lookup dictionary while this is happening. One for node entities to groupId and one for later which has groupId with the new grouped Entity and several meta-data values like status.

    When this is done, I know how much entities to allocate, which I'm creating now on MT and with EntityManager.
    After the creation, the next phase begins where I add the data where I use ECB again.

    I still don't understand why this is a major issue. If you think about it, doing this in parallel jobs, okay, but I'm not asking for that.

    > Sure you could concoct a scheme to then patch that up, but that's just making an already broken design worse.

    I'm confused, this is exactly as ECS is working now?
    Why should incrementing an integer cause this many unknowns? ECB is mostly designed to be used in parallel jobs, but if it's not used in parallel, how should it fail in creating valid entities? Okay, commands can fail but they don't fail with no reason. There's 100% something wrong in code if it does happen. Commands can fail in a 100 different ways with wrong code so why should it affect design?
     
  6. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,753
    That is not how entity Index works.
    Entity is made of a Index:Version

    You delete Entity 37:1 and create a new Entity it might be 37:2 or it might be 145:1
     
  7. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    You can always instantiate range of entities from prefabs, as spares, or pooling system.
    You do that once on main thread.
    Run a job, which grabs these spares. Use them for your calculations.
    Or use them, to create valid references between entities.

    In some cases I use system, which specifically assigns relations, usin ecbs and frame delay mechanics.
    That requires relation entity to be valid and to exists. You create entity with ecb, and assign existing entity relations to it. I. E. Having source(ECS created) and target (valid) entities.
    Next system in next frame, runs the job, to assign them accordingly.
     
  8. Enzi

    Enzi

    Joined:
    Jan 28, 2013
    Posts:
    954
    Right, I've only simplified the question, but you make a good point when the entity gets updated later in the process. Storing a reference to the entity seems impossible if that's happening.

    Just from a creation perspective, It either gets incremented or pulled from the reserved entity pool.
    There's still nothing unknown about it that would prevent the ECB on main-thread to return a valid entity with a correct Index:Version even if the entity technically is not existing at this point and only after ECB has run.

    It doesn't seem a good solution exists to store entity references for entities where a component is either added/removed at some point and get new version numbers. I'm not aware of any API call to get an entity index regardless of version number other than crawling through every entity and check for the index in a system.

    I still think this is a design flaw where a very real problem is mixed together. How many times do we actually care about the version of an entity? It's all about the index number.
    I think splitting this data would be a good start. As developers we care about the index but only ECS cares about the version.
     
  9. Tony_Max

    Tony_Max

    Joined:
    Feb 7, 2017
    Posts:
    349
    What i really want from ECB is command batching. For example if i will use EntityCommandBuffer.AddComponent<> twice with two IComponent data it will firstly add 1st component which will create new archetype and then add 2nd component which will create another one new archetype. But it has no reason, because ECB plays back all of his commands at a time and there can be magic which will batch same commands in ComponentTypes struct and then add it at once.
     
    Orimay, Enzi and PublicEnumE like this.
  10. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    729
    Just want to second this. We moved away from ECB as much as possible because it’s not implemented this way.

    In general, moving away from ECB (where possible) resulted in cleaner code for us, so it would ultimately a net win. But for those times when structural changes are necessary, it’s unclear why this type of batching isn’t used.

    1 archetype change will almost always be faster than n sequential archetype changes.
     
    Orimay and Enzi like this.
  11. davenirline

    davenirline

    Joined:
    Jul 7, 2010
    Posts:
    969
    Just had this problem today. ECB sucks this way. What I did instead is to create another system that uses ISystemStateComponentData to check for new entities that were created. This way, I'll have the real entity values. But it's still a hoop to jump over.
     
  12. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    448
    Version is need to validate an entity. If you store an entity in some component you need a way to be able to tell if this entity is valid. Index alone will not help. With version you can check if version of entity stored in component is equal to current version of this entity. If it is smaller it means oryginal entity was destroyed.
     
    Antypodish likes this.
  13. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    Also version is always greater than 0, if entity is valid.
    So It can be used for default (invalid) entities checks as well, where index and version are 0.
    If just having index, and it happens to be 0, your system can pull your entity with index 0. And most likely receive error results.
     
  14. mikaelK

    mikaelK

    Joined:
    Oct 2, 2013
    Posts:
    284
    Good question. I think you shouldn't at all.
    I think the entity id is not really meant to be used as an index and you should create a component that has reference to the entity or your own identification. Its quite clear on the docs that the entity id is something you should not count your logic on. Storing the entity inside component has been the best option for me along with component filtering.

    To me it sounds a little bit like you are trying to apply object oriented thinking in to the data oriented world. For example doing the search entity by index was one of the first things I did, but that's actually horrible and inefficient thing to do with entities. The way it should be done I think is adding a component that you can use to get it queried in entityBatchJob or for each. Its fast and clean.

    This ecs has been quite difficult for me, but I think I finally have figured out how to write a lot cleaner code with the ecb.
    Separate code to different groups. Handle instantiation and deletion in different systems + groups to avoid issues with ecb and entitymanager etc.
     
    Antypodish and Haneferd like this.
  15. Liquid_Nalee

    Liquid_Nalee

    Joined:
    Mar 15, 2021
    Posts:
    19
    I faced a similar issue. Long story short, I was trying to procedural-ly generate hexagonal tiles on my map by spreading them out until they reach walls or obstacles. I needed said tiles to form a graph and I tried using the patching of ECB to make new references of instantiated tiles in buffer match but honestly it wasn't glorious.

    Right now how I do it is I don't. I decided it's the job of new tiles to hook up to old tiles and not the other way around so when I spawn tiles I add a specific component with a reference to the tiles that it spawned next to and tiles with this component have a system that makes links two ways.

    Not sure if it's the best or even a good approach to graph generation in ECS but it's what I ended up with and it doesn't feel ugly.
     
    Orimay and PublicEnumE like this.