Search Unity

  1. Want to see 2020.1b in action? Sign up for our Beta 2020.1 Overview Webinar on April 20th for a live presentation from our evangelists and a Q&A session with guests from R&D.
    Dismiss Notice
  2. Interested in giving us feedback? Join our online research interviews on a broad range of topics and share your insights with us.
    Dismiss Notice
  3. We're hosting a webinar for the new Input System where you'll be able to get in touch with the devs. Sign up now and share your questions with us in preparation for the session on April 15.
    Dismiss Notice

Reactive System

Discussion in 'Data Oriented Technology Stack' started by Ziboo, Mar 20, 2018.

  1. Ziboo

    Ziboo

    Joined:
    Aug 30, 2011
    Posts:
    330
    Hello,

    I tried Entitas some time ago, and they had a ReactiveSystem, which means that it was notified when a entity was created or removed.

    Do the ECS system have something like that ?
    I only see Update function, but didn't see anything like OnEntityCreated, OnEntityDestroyed.
    One way to do it is to have a system asking for this entity and destroying it after the frame, but doesn't seem very clean..

    Thanks
     
    ChiuanWei and one_one like this.
  2. one_one

    one_one

    Joined:
    May 20, 2013
    Posts:
    522
    There are sync points, one of which is for the creation of entities. Maybe that's a place to start looking into? I haven't checked out the code, though, so I'm not sure if that's purely internal (and as such not useful to you.)

    I'm generally also interested in how to elegantly tackle reactive concepts in Unity's ECS. I imagine it would be achieved by setting certain values (in the simplest case an int representing a boolean), which represents whether an 'event' has occured in a component or not. The system that set that event value (to something that triggered certain code in other systems) would then be responsible to returning it to a 'non-event' state. However, this seems a little hacky.
    Or is this behaviour that is generally not intended with the ECS, other than at sync points, maybe?
     
  3. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    993
    i played with the ECS from spyshifty, which also had reactive systems....but I do not know if this is even possible if you run multiple threads...I am just beginning with ECS and to me it looks like it is not intended...
     
  4. one_one

    one_one

    Joined:
    May 20, 2013
    Posts:
    522
    Indeed, concurrency is the main issue here. I was expecting ways to work in more of a messaging system, anyway, and not like C# events.
     
  5. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,882
    We are definately going to add support for systems iterating over entities that were added / removed to the ComponentGroup. Thats definitely very important, but our aim is always to make it very performant and memory efficient.

    We also want to add support for iterating components based on if they have changed.

    We simply haven't gotten to it yet...
     
    Tony_Max, ChiuanWei and one_one like this.
  6. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,882
    The current pattern is to simply add / remove components. It actually works really well as a messaging system.
     
    florianhanke and recursive like this.
  7. Arakon

    Arakon

    Joined:
    Mar 8, 2014
    Posts:
    21
    I was going to ask this question as well.
    I agree - add/remove components, and even entities that function as a message/event are fairly easy workarounds.

    Thanks Joachim! - very nice work. Very excited for all the possibilities Unity is bringing!
     
  8. kbm

    kbm

    Joined:
    Aug 7, 2014
    Posts:
    66
    @Joachim_Ante is there a SUPER rough estimate on when you guys think the ECS will be production ready?
     
  9. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,490
    According to GDC video I watched it is 2018.3 (or later?)
     
  10. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    640
    I actually prefer using dedicated message entities (after playing around with Entitas a bit last year) for the following reasons:
    • Stays inside in the ECS framework (doesn't require any C# delegates or event abstractions/interfaces).
    • Event object layout is both extremely well known, and also easy to modify to add/remove/change data layout.
    • You explicity control when the message is processed (or it doesn't get processed).
      • For example, if you use a "normal" event system for entity events, you may need to buffer if mass events are expected, this happens automatically in an ECS, since event entities are processed explicitly.
    • Easy to modify or peek messages for debugging or modifier purposes by simply inserting systems that run before the system that handles the message proper.
    • Easy to recycle frequent messages or manage their memory.
     
  11. Ziboo

    Ziboo

    Joined:
    Aug 30, 2011
    Posts:
    330
    So I tried something and ended up with that...

    Do you think there is a better way to do that ?

    Code (CSharp):
    1. public class MouseClickSystem : ComponentSystem
    2.     {
    3.         private readonly Stack<Entity> mouseClicks = new Stack<Entity>();
    4.  
    5.         protected override void OnUpdate()
    6.         {
    7.             while (this.mouseClicks.Count > 0)
    8.             {
    9.                 this.EntityManager.DestroyEntity(this.mouseClicks.Pop());
    10.             }
    11.  
    12.             if (Input.GetMouseButtonDown(0))
    13.             {
    14.                 this.CreateMouseClick(0);
    15.             }
    16.             else if (Input.GetMouseButtonDown(1))
    17.             {
    18.                 this.CreateMouseClick(1);
    19.             }
    20.             else if (Input.GetMouseButtonDown(2))
    21.             {
    22.                 this.CreateMouseClick(2);
    23.             }
    24.         }
    25.  
    26.         private void CreateMouseClick(int i)
    27.         {
    28.             var entity = this.EntityManager.CreateEntity(typeof(MouseClick));
    29.  
    30.             var mouseClick = new MouseClick
    31.             {
    32.                 ScreenPosition = Input.mousePosition,
    33.                 Type = i
    34.             };
    35.  
    36.             this.EntityManager.SetComponentData(entity, mouseClick);
    37.  
    38.             this.mouseClicks.Push(entity);
    39.         }
    I tried using PostUpdateCommands.DestroyEntity() to destroy the click but that actually destroy it right away.

    I also though about doing a EntityCleanerSystem that will find any entity with a CleanNextFrame component attached and destroy it but then I'm pretty sure I will run into race condition with other systems;
     
  12. Ziboo

    Ziboo

    Joined:
    Aug 30, 2011
    Posts:
    330
    So I found out about UpdateInGroup and tried the Cleaning System solution and I think it's way nicer.

    Code (CSharp):
    1.  public class MouseClickSystem : ComponentSystem
    2.     {
    3.         protected override void OnUpdate()
    4.         {
    5.             if (Input.GetMouseButtonDown(0))
    6.             {
    7.                 this.CreateMouseClick(0);
    8.             }
    9.             else if (Input.GetMouseButtonDown(1))
    10.             {
    11.                 this.CreateMouseClick(1);
    12.             }
    13.             else if (Input.GetMouseButtonDown(2))
    14.             {
    15.                 this.CreateMouseClick(2);
    16.             }
    17.         }
    18.  
    19.         private void CreateMouseClick(int i)
    20.         {
    21.             var entity = this.EntityManager.CreateEntity(typeof(MouseClick), typeof(GarbageSystem.Garbage));
    22.  
    23.             var mouseClick = new MouseClick
    24.             {
    25.                 ScreenPosition = Input.mousePosition,
    26.                 Type = i
    27.             };
    28.  
    29.             this.EntityManager.SetComponentData(entity, mouseClick);
    30.         }
    31.     }
    Code (CSharp):
    1. using Unity.Collections;
    2.     using Unity.Entities;
    3.     using UnityEngine.Experimental.PlayerLoop;
    4.  
    5.     [UpdateInGroup(typeof(PreUpdate))]
    6.     public class GarbageSystem : ComponentSystem
    7.     {
    8.         [Inject] private Data data;
    9.  
    10.         protected override void OnUpdate()
    11.         {
    12.             for (var i = 0; i < this.data.Length; i++)
    13.             {
    14.                 this.EntityManager.DestroyEntity(this.data.Entity[i]);
    15.             }
    16.         }
    17.  
    18.         public struct Garbage : IComponentData
    19.         {
    20.         }
    21.  
    22.         private struct Data
    23.         {
    24.             [ReadOnly] public int Length;
    25.             [ReadOnly] public EntityArray Entity;
    26.             [ReadOnly] public ComponentDataArray<Garbage> Garbage;
    27.         }
    28.     }
     
    ChiuanWei and recursive like this.
  13. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    640
    This is pretty much the way mine worked before, complete with a component that simply marked something for deletion.
     
    one_one likes this.
  14. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    993
    @Ziboo only browsing with tablet...so can not try it out.

    Found this in one of the github examples...check postupdatecommands

    Code (CSharp):
    1.         protected override void OnUpdate()
    2.         {
    3.             for (int i = 0; i < m_Group.Length; i++)
    4.             {
    5.                 m_Group.Headings[i] = new Heading
    6.                 {
    7.                     Value = math.normalize(new float3(Random.Range(-1, 1), Random.Range(-1,1), Random.Range(-1, 1)))
    8.                 };
    9.                
    10.                 PostUpdateCommands.RemoveComponent<RandomInitialHeading>(m_Group.Entities[i]);
    11.             }
    12.         }
     
  15. Ziboo

    Ziboo

    Joined:
    Aug 30, 2011
    Posts:
    330
    I tried PostUpdateCommands, but it doesn't do the trick.
    PostUpdateCommands will fire just after OnUpdate but in the same System, before other systems.
    So for my case, the entity will be destroyed right away, and other systems couldn't catch it
     
  16. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    993
    Ok - good to know - thx
     
  17. DwinTeimlon

    DwinTeimlon

    Joined:
    Feb 25, 2016
    Posts:
    191
    Make sure you use the attributes UpdateBefore/UpdateAfter to make sure what should be running first/last.
     
    ChiuanWei likes this.
  18. Necromantic

    Necromantic

    Joined:
    Feb 11, 2013
    Posts:
    115
    Sorry for updating this old thread but I thought it was relevant as this Thread is exactly about this.
    Joachim's Talk at Unite Berlin was highlighting the new Reactive Systems that are coming for Unity ECS.
    As far as I can tell they will work for components being added, changed, removed. Not sure about Entities but that really shouldn't be that much of a stretch. Worst case you could use a Component to check whether an Entity is added or removed that way.
     
    leni8ec likes this.
  19. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,494
    From the talk I am wondering if ISystemStateComponentData is already the "reactive system" (ingredient?) or a new kind of system is coming and use that as a base?
     
  20. Necromantic

    Necromantic

    Joined:
    Feb 11, 2013
    Posts:
    115
    I think it's the core component of it, but there would of course have to be some internal system that reacts to that and populates the data.
    I also wonder how it'll recognize the structs since he didn't have any injects in his code. Will we just have to name them "Added" and "Removed" or will it have to contain at least one ISystemStateComponentData? I assume it'll just need to be inside a System.
     
  21. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,629
    off the top of my head I'd use an archetype as the event. Then an extension method on EntityCommandBuffer something like DestroyEntityAndNotify where you call DestroyEntity then add your archetype. You could make methods that check for updates to specific components or archetypes also fairly easily.
     
  22. aeldron

    aeldron

    Joined:
    Feb 12, 2013
    Posts:
    29
  23. Vacummus

    Vacummus

    Joined:
    Dec 18, 2013
    Posts:
    130
    Yup, but that's only for added and removed components. They have also added a "ChangeFilter" attribute that you can use in jobs only, which allows you to process only the entities for the components that have changed (like if you have 500 entities with a Position component, but only for 100 of them the Position component has changed, this filter will allow you to process only those 100 entities).
     
  24. e199

    e199

    Joined:
    Mar 24, 2015
    Posts:
    100
    With ChangeFilter if single entity was changed, then you get full chunk. And you don't know which one is changed.

    If in your example 500 entities are inside same chunk - all of them will be 'changed' when any of them is changed.

    Which is not desirable in most gameplay cases. (I'm not talking about boid sims or other ... like that)
     
    aeldron and Vacummus like this.
  25. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,067
    This is something people need to know. In fact, no entity even needs to change just the chunk needs to have had a request for write permission on that frame. ChangeFilter is more there for performance improvements than actual tracking changes. Lets you skip doing work on data that has definitely not changed.
     
  26. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    203
    I’m very curious to know people’s thoughts these days. When I first started writing Unity ECS code, I was using an approach exactly like the one described early in this thread: Tags were added to Entities as a messaging system, to be picked up by reactive Systems, which would later remove them.

    The practical effect of this was that my game’s archetypes were constantly being blown apart. Entities could have several tags added or removed every frame. And the chance of them having collections of tags in common with other entities was pretty low at any given moment. Maybe it was just the specific needs of my game. I’m sure someone might say, “Don’t use tags that way if you have so many.”. Ok, sure, but as a test of the original suggestion, that’s what happened.

    Nowadays I’m about to rewrite things to lean in the other direction: My entities change archetypes as little as possible (or never). They already start out with a component representing each ‘tag’ they could ever have, with bool flag in each. Systems iterate over everything, all the time, and early out of the flag is false.

    I’m not claiming this is right. I’m trying to find the sweet spot, or at least gather data on what solutions are optimal is different situations. 1+ years after this thread started, what are you guys’ thoughts? What approaches have you landed on?
     
    voteprolifenow1984 likes this.
  27. tarahugger

    tarahugger

    Joined:
    Jul 18, 2014
    Posts:
    121
    ChangedFilter is really confusing right now, and it probably does more harm than good because of the confusion. It also doesn't make sense in the context of all the efforts to make things as simplistic as possible. A user has their IForEach<MyComponent> and they just want to iterate the stuff that changed. So they drop an [ChangeFilter] on there and it doesn't do what they expect, its misleading.

    I've switched to using an Events system (that @tertle wrote) and its working out well so far. He's optimized it so that events can be queued concurrently from within your jobs and then created efficiently in bulk at the end of the frame.

    Events also exist for a full cycle until the EventSystem runs again - which avoids the issue with the change filter versioning where earlier systems are not aware of a change caused by systems later in the frame.

    It works like this:

    * Event Producers Group (early as possible) - lots of jobs parallel processing and outputting queued events.
    * EventSystem - actually creates the events and deletes the events from the previous frame.
    * Event Responders Group (either late or next frame) - systems that do structural changes, only when events have occurred.

    I'm very interested to see how the EntityComponentStore evolves because it will surely influence the best approach for making reactive systems/jobs and change filtering. In particular the note "Rework internal storage so that structural changes are blittable (and burst job)"

    Code (CSharp):
    1. // Notes on upcoming changes to EntityComponentStore:
    2. //
    3. // Checklist @macton Where is entityComponentStore and the EntityBatch interface going?
    4. // [ ] Replace all internal interfaces to entityComponentStore to work with EntityBatch via entityComponentStore
    5. //   [x] Convert AddComponent NativeArray<Entity>
    6. //   [x] Convert AddComponent NativeArray<ArchetypeChunk>
    7. //   [x] Convert AddSharedComponent NativeArray<ArchetypeChunk>
    8. //   [x] Convert AddChunkComponent NativeArray<ArchetypeChunk>
    9. //   [x] Move AddComponents(entity)
    10. //   [ ] Need AddComponents for NativeList<EntityBatch>
    11. //   [ ] Convert DestroyEntities
    12. //   [ ] Convert RemoveComponent NativeArray<ArchetypeChunk>
    13. //   [ ] Convert RemoveComponent Entity
    14. // [x] EntityDataManager just becomes thin shim on top of EntityComponentStore
    15. // [x] Remove EntityDataManager
    16. // [ ] Rework internal storage so that structural changes are blittable (and burst job)
    17. // [ ] Expose EntityBatch interface public via EntityManager
    18. // [ ] Other structural interfaces (e.g. NativeArray<Entity>) are then (optional) utility functions.
    19. //
    20. // 1. Ideally EntityComponentStore is the internal interface that EntityCommandBuffer can use (fast).
    21. // 2. That would be the only access point for JobComponentSystem.
    22. // 3. "Easy Mode" can have (the equivalent) of EntityManager as utility functions on EntityComponentStore.
    23. // 4. EntityDataManager goes away.
    24. //
    25. // Input data protocol to support for structural changes:
    26. //    1. NativeList<EntityBatch>
    27. //    2. NativeArray<ArchetypeChunk>
    28. //    3. Entity
     
    Last edited: Jul 12, 2019
  28. IsaiahKelly

    IsaiahKelly

    Joined:
    Nov 11, 2012
    Posts:
    335
    @PublicEnumE I've thought before that maybe they should add a special component type that is never apart of the archetype and excepted to be added and removed frequently. Call it a tag or dynamic component vs a static/normal component that is rarely/never added or removed to entities at run-time or after creation.

    However, from what I understand they're optimizing the system to make empty components have as little impact on performance as possible so you shouldn't have to worry about it that much. I also just recently discovered that they've added attributes to allow you to filter tag components on a job system without having to queue the actual component itself.

    Example:
    Code (CSharp):
    1.     [RequireComponentTag(typeof(EnemyTag))]
    2.     struct TurnJob : IJobForEach<Translation, Rotation>
    3.     {
    4.         public float3 playerPosition;
    5.  
    6.         public void Execute([ReadOnly] ref Translation pos, ref Rotation rot)
    7.         {
    8.             float3 heading = playerPosition - pos.Value;
    9.             heading.y = 0f;
    10.             rot.Value = quaternion.LookRotation(heading, math.up());
    11.         }
    12.     }
     
    Last edited: Jul 13, 2019
  29. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    Tag components are zero-sized components, and if you add a tag to all entities in a chunk at once using batch APIs, it doesn't touch the entities and just changes the chunk's archetype. I use it to mark newly instantiated entities so that I can initialize them to unique values and then I remove the tags on all of them. (I do it a little more intelligently than that to reduce sync points but you get the idea.)
     
  30. IsaiahKelly

    IsaiahKelly

    Joined:
    Nov 11, 2012
    Posts:
    335
    @DreamingImLatios Forgive my ignorance but how is changing the chunk and archetype not affecting the entities? I also thought changing the chunk was the entire issue here. Is it just better in your case because it's done in batches? Are you also using system state components or just regular ones in your implementation?
     
  31. Srokaaa

    Srokaaa

    Joined:
    Sep 18, 2018
    Posts:
    99
    @IsaiahKelly If the component is i empty (as in "has no fields and bit size of 0") the memory layout of an archetype will remain the same
     
  32. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,815
    It not moves entities in memory it’s only changes chunk archetype without memory touching, it’s one of goodies of tags.
     
    rigidbuddy likes this.
  33. IsaiahKelly

    IsaiahKelly

    Joined:
    Nov 11, 2012
    Posts:
    335
    @Srokaaa @eizenhorn That is great news. Does this mean that adding and removing many component tags no longer has any performance/memory penalties?

    Edit:
    So you just need to use the correct API calls to avoid performance issues? Can you share or point me to any further details on this?
     
  34. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,067
    No, if you are only tagging 1 entity in a chunk then that entity now has a new archetype and no longer belongs in that chunk and will still move.

    If you tag all entities in the same chunk at once with a batch API then nothing needs to move.
     
  35. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    Any API that takes a Chunk or EntityQuery is batch capable. I'm not sure if all of them have batching implemented yet. But you can expect those that don't will be optimized in the future.
     
  36. IsaiahKelly

    IsaiahKelly

    Joined:
    Nov 11, 2012
    Posts:
    335
    Okay, so changing tag components in batches on all entities of the same archetype doesn't change their memory layout, just the archetype itself. But changing just a few tag components still creates an additional archetype that must be moved to a new chunk?

    I think the difference between archetype and memory layout was confusing me here. I think I understand it now. I was at first imagining this now worked like my idea of having the tag components not belong to any archetypes at all. But I'm not sure if such a system is even possible. Perhaps just creating a kind of tag/event entity instead would be better? Since that would seem to do basically the same thing.

    So back to the original question by @PublicEnumE about a messaging systems. What's a better approach than using tags? Having to change all tags at once really limits what you can do with that approach. Is the idea of a tag/event entity the way to go here?
     
  37. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    Changing tags on all entities in a chunk changes the chunk's archetype. And changing tags on all entities in an archetype changes all the chunks with that archetype into the new archetype. Changing some of the tags causes Entities to be moved around.

    Different situations have different best solutions. If you have jobs running on arrays instead of iterating chunks or have jobs processing a different world, having Entities move around probably won't have any major performance costs as the sync point won't starve your worker threads.

    Event entities vs event tags is a well-argued topic on these forums. They have similar limitations. I say go with whatever makes more sense. But if you want to avoid sync points, use a NativeArray, NativeList, NativeQueue, NativeHashMap, BlockStream, ect.
     
  38. aeldron

    aeldron

    Joined:
    Feb 12, 2013
    Posts:
    29
    Hi @tertle . What is this Events system you wrote? Can I have a look please?
     
  39. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,067
  40. aeldron

    aeldron

    Joined:
    Feb 12, 2013
    Posts:
    29
    This guy has extended SystemStateComponentData to keep track of changes on entities, not just when a component has been added or removed. It looks interesting but I wonder how expensive it is.

    https://medium.com/@icex33/coping-with-change-in-unity3d-ecs-45422fff8dda
     
  41. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    It might be fine for Tiny, which is his target, but it doesn't work in Jobs. There's better ways to go about it, but which is best is project-specific.
     
  42. maxim-zaks

    maxim-zaks

    Joined:
    Jul 24, 2019
    Posts:
    9
    Hi, I am "this guy". And BTW, I am also co-designer of Entitas .

    My long reply was recognised as spam, so I upload it as txt file.

    Maybe just TL;DR here:
    You can use the technique I described in Jobs and it is not performance critical.
     

    Attached Files:

    aeldron and zyzyx like this.
  43. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    Have you tried it in jobs? It looks like your code depends on EntityManager which does not work in jobs.

    The real issue with reactive systems that track per Entity is that you essentially need a change buffer for each system that cares about changes, at least with your approach. The real best approach is to store the system's version number in a component whenever you write to it and always check for that. However, abstracting into a simple API is a real challenge.
     
  44. aeldron

    aeldron

    Joined:
    Feb 12, 2013
    Posts:
    29
    Hi Maxim. Thank you for the explanation. I think your helper classes are going to come handy for my project.
     
    maxim-zaks likes this.
  45. maxim-zaks

    maxim-zaks

    Joined:
    Jul 24, 2019
    Posts:
    9
    Oh I got you now. Yeah I didn't try it in a Job yet and your point regarding the extension methods on Entity using EntityManager is spot on. However I am not sure I understand your concerns regarding "change buffer for each system". The IDiffComponent:
    Code (CSharp):
    1. public interface IDiffComponent<T> : ISystemStateComponentData where T: struct, IComponentData, IEquatable<T>
    2. {
    3.     T Value { get; set; }
    4. }
    Is introduced so that we can keep "sync point" into components state. You can implement diff components for every system, but IMHO in most cases it is not necessary. You might get away only with one, even if you use Jobs.

    For IJobChunk in particular you can introduce an extension method which looks something like this:

    Code (CSharp):
    1.     private static NativeList<T1> Filter<T1, T2>(this ArchetypeChunk chunk, ArchetypeChunkComponentType<T1> valueType, ArchetypeChunkComponentType<T2> diffType, ChangeType changeType)
    2.         where T1 : struct, IComponentData, IEquatable<T1>
    3.         where T2 : struct, IDiffComponent<T1>
    4.     {
    5.         var hasCurrent = chunk.Has(valueType);
    6.         var hasOld = chunk.Has(diffType);
    7.         if ((changeType & ChangeType.Added) == ChangeType.Added
    8.             && (hasCurrent && !hasOld))
    9.         {
    10.             var array = chunk.GetNativeArray(valueType);
    11.             var result = new NativeList<T1>(array.Length, Allocator.TempJob);
    12.             result.AddRange(array);
    13.             return result;
    14.         }
    15.  
    16.         if ((changeType & ChangeType.Removed) == ChangeType.Removed
    17.             && (!hasCurrent && hasOld))
    18.         {
    19.             var array = chunk.GetNativeArray(diffType);
    20.             var result = new NativeList<T1>(array.Length, Allocator.TempJob);
    21.             for (int i = 0; i < array.Length; i++)
    22.             {
    23.                 result.Add(array[i].Value);
    24.             }
    25.             return result;
    26.         }
    27.         if ((changeType & ChangeType.Updated) == ChangeType.Updated && hasCurrent && hasOld)
    28.         {
    29.             var result = new NativeList<T1>(Allocator.TempJob);
    30.             var currentArray = chunk.GetNativeArray(valueType);
    31.             var oldArray = chunk.GetNativeArray(diffType);
    32.             for (int i = 0; i < currentArray.Length; i++)
    33.             {
    34.                 if (currentArray[i].Equals(oldArray[i].Value) == false)
    35.                 {
    36.                     result.Add(currentArray[i]);
    37.                 }
    38.             }
    39.  
    40.             return result;
    41.         }
    42.  
    43.         return new NativeList<T1>();
    44.     }
    I didn't tested it yet, so take it with a grain of salt. But the idea would be to filter the chunk for changed components. In this case the Filter method returns a NativeList of component data, in some cases it would be better to return a NativeList of ints / indexes, so that if you have other components you need to access in the chunk you can do it by index. BTW you could still use the `FilterChanged` option on EntityQuery if you are interested only in updated components. Added and Removed are problematic though.

    PS: with using `FilterChanged` option on EntityQuery you might get lots of false positive results as a chunk is marked changed if at least one of the components was changed (see my blog post). So it is IMHO still good to use the Filter method I described in this post, to not process unchanged components.
     
    Last edited: Jul 25, 2019
  46. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    Imagine you have systems A, B, C, and D which all care about a component named Care.

    System A writes new values to Care.

    System B applies a change filter to Care and reads the cahnged values and logs them in a database.
    Because System B doesn't want to accidentally log these changes again next frame, it resets the diff component to match the current value.

    System C also applies a change filter to Care. It sees nothing because B had to wipe the state.

    System D also writes to Care using a different EntityQuery from A. These are changes that B and C want to pick up on next frame.

    Do you see the problem with only storing one copy of the diff rather than a copy per system?

    The solution to this problem is to apply the same change version approach Unity uses on chunks, except you apply them to the individual entities. You simply have to add an int changeVersion variable to a component and make sure you update that variable whenever you write to it using the global change version which you can get from the system.

    This solution is really easy to do, but it is also really easy to forget to do and introduce hard-to-track bugs. So creating some sort of API that makes it hard to screw up without being annoying is real challenge.
     
  47. maxim-zaks

    maxim-zaks

    Joined:
    Jul 24, 2019
    Posts:
    9
    Yeah I mentioned it in my first reply as the order problem and that you would need to place the write systems before read and put the dedicated balance system at the end. You can use Component System Groups for it or just define annotate the systems with `[UpdateBefore]`. Which can be tricky I agree. It would be nice to get some tooling which analyse the queries for read write and put write systems before reactive systems. As a last resort I would start introducing multiple diff components.

    Your solutions is also feasible with a specialise interface which forces user to have a version property and force them to implement an `Update` method where they provide new version of data and the version will be increased automatically, however you have a problem of remove component. You need force people not to cal remove component and have a flagcinstead. You also have to prohibit direct destruction of entities, or force people to use SystemData.

    Something like this comes to mind:
    Code (CSharp):
    1. interface IVersionedDataComponent<T> : ISystemStateComponentData
    2.     where T: struct
    3. {
    4.     T Value { get; set; }
    5.     int Version { get; set; }
    6.     bool Removed { get; set; }
    7.     void Update(T value);
    8. }
    9.  
    10. struct Position2 : IVersionedDataComponent<float2>
    11. {
    12.     public float2 Value { get; set; }
    13.     public int Version { get; set; }
    14.     public bool Removed { get; set; }
    15.     public void Update(float2 value)
    16.     {
    17.         this.Value = value;
    18.         this.Version += 1;
    19.     }
    20. }
    But you get still lots of maintenance issues with it and don't forget that every system needs to maintain an NativeDictionary<Entity, Int> to store the last version of the component it handled.
    All in all I think it is a pretty heavy implementation, with lots of steps you need to keep an eye on.
    I think I would rather go with IDiffComponent and figuring out the system order, but it is up to the actual use case and developer preference.
     
  48. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    The way a system would react to changed components versus added or removed components is never going to be the same. So because of that, I think it is a bad idea to conflate the problems.

    For adding and removing, you generally only want to do once during your frame anyway due to the sync point it creates. So by intelligently reducing your structural changes to a single sync point, you promise all other systems to get correct reactive queries if you use ISystemStateComponentData (you can make these tags for even more performance goodness). But in the case you can't do that, you can create an additional ISystemStateComponentData tag for each sync point so that you can ensure the add or remove change can be reacted to for a full cycle.

    Now as for reacting to changes to components, the solution I propose does not require a NativeDictionary per system at all. Actually, it just uses what is already built into ECS.

    A system has two member properties called GlobalSystemVersion and LastSystemVersion. GlobalSystemVersion is a counter that lives in a World and increments every time any system gets updated. LastSystemVersion is what the GlobalSystemVersion was at the last time the current system was run (usually a frame ago). Whenever you write to a component, you also write to it the GlobalSystemVersion. Whenever you read from a component, you compare its version against your LastSystemVersion. If it is less than or equal to your LastSystemVersion, then you know the component hasn't changed since the last time you saw it. If it is greater than your LastSystemVersion, then you know some other system wrote to that component since the last time you processed it.

    So in order to react to all changes, I simply need a changeVersion field in my IComponentData, a tag ISystemStateComponentData, and an additional tag ISystemStateComponentData for each sync point if for some reason I have more than one sync point per frame. System order can otherwise be or do whatever it's feeling that day.

    I can think of really easy APIs to force a user to remember to update the changeVersion when writing. Just make the data private and make a public set function which takes the GlobalSystemVersion as the second argument. But that's kinda annoying because you either have to write that boilerplate for each component or use a generic component and then register each used type with the type manager. The new code analysis stuff which is coming will hopefully solve this problem completely.
     
  49. maxim-zaks

    maxim-zaks

    Joined:
    Jul 24, 2019
    Posts:
    9
    My experience tells me otherwise. Add and Update very often is handled equally. Remove is mostly a special case, but sometimes you just need a trigger, you execute logic in case something changed doesn't matter if added, updated, or removed because the system has its own "entity evaluation process".

    That sad I guess it is generally better to avoid removing components for performance reasons, but there are enough cases where code clarity has higher priority.

    Again can't agree to 100%. Many things can kill an entity. And also many things can spawn an entity. I agree that it is desirable to tight things up and reduce the sync points, but we also should not forget that Unity is used not only by "ace programmers" therefor a solution, which punishes every "small stupidity" is very painful and can cost lots of debugging time.

    I was not aware of GlobalSystemVersion and LastSystemVersion, this is quite helpful. I didn't really understand what are the two ISystemStateComponentData are for. Maybe this is also why I think there is a system order problem in your solution. If you have systems S1, S2, S3 in the given order and S3 is changing the component S1 and S2 what to react to, they will not trigger because the GlobalSystem order of S3 from previous frame could be equal to LastSystemVersion of S1 and S2 in current frame. Maybe to solve this you propose to introduce the ISystemStateComponentData flags, but I didn't get it. I would really appreciate if you could give a short example like with Position component and Move, Gravity, Wind system which might change the position component, and Collision and FindTarget systems which react to Position change.
     
    e199 likes this.
  50. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,043
    I'm not saying you should force add/remove and value change to be handled separately. I'm saying your API should not forcefully couple them when it doesn't need to. For cases where you want to process add or changed, this is plenty simple to write (and you can even simplify it with some methods that do the full check for you):
    Code (CSharp):
    1. if (care.changedVersion - lastChangedVersion <= 0 || careTrackingTag.Exists(entity))
    2.     return;
    Are you telling me you have an EntityCommandBuffer running after every system? Sure a lot of things can kill an entity, but there are very few places where the entity will actually die. It is very easy to make the number of those places 1, which is ideal. But even if you can't do that, I proposed an alternate solution for two or more sync points, and if that still doesn't meet your fancy, you could always do strategic placement of a balancing system like what your solution requires.

    That's not how it works. Every OnUpdate of every system will get a unique GlobalSystemVersion. So imagine you have Systems A, B, and C which update every frame.

    System A OnUpdate -> GlobalSystemVersion = 1
    System B OnUpdate -> GlobalSystemVersion = 2
    System C OnUpdate -> GlobalSystemVersion = 3

    Next frame

    System A OnUpdate -> GlobalSystemVersion = 4, LastSystemVersion = 1
    System B OnUpdate -> GlobalSystemVersion = 5, LastSystemVersion = 2
    System C OnUpdate -> GlobalSystemVersion = 6, LastSystemVersion = 3
     
    maxim-zaks likes this.
unityunity