Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

I created a ReactiveComponentSystem

Discussion in 'Data Oriented Technology Stack' started by tertle, Jul 9, 2018.

  1. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    Full reactive system added that only updates when a ComponentData changes.
    Read here: https://forum.unity.com/threads/i-created-a-reactivecomponentsystem.539581/#post-3566124

    -Original-

    I got sick of repeatably creating new ISystemStateComponentData for various systems, so I create my own ReactiveComponentSystem that can automatically track when components are added or removed.

    Demo
    Code (CSharp):
    1. using BovineLabs.Toolkit.ECS;
    2. using Unity.Entities;
    3.  
    4. namespace BovineLabs.Systems
    5. {
    6.     public struct TestComponent : IComponentData
    7.     {
    8.  
    9.     }
    10.  
    11.     public class TestReactiveSystem : ReactiveComponentSystem
    12.     {
    13.         private ComponentGroup _addGroup;
    14.         private ComponentGroup _removeGroup;
    15.  
    16.         protected override void OnCreateManager(int capacity)
    17.         {
    18.             _addGroup = GetReactiveAddGroup(typeof(TestComponent));
    19.             _removeGroup = GetReactiveRemoveGroup(typeof(TestComponent));
    20.         }
    21.  
    22.         protected override void OnReactiveUpdate()
    23.         {
    24.             var addedEntities = _addGroup.GetEntityArray();
    25.             var removedEntities = _removeGroup.GetEntityArray();
    26.  
    27.             // DO WORK
    28.         }
    29.     }
    30. }
    31.  
    32.  
    Usage
    - Implement ReactiveComponentSystem
    - Use GetReactiveAddGroup(Type) to create a ComponentGroup to track when a component is added
    - Use GetReactiveRemoveGroup(Type) to create a ComponentGroup to track when a component is removed.
    - Instead of using OnUpdate(), which I have sealed, use OnReactiveUpdate()

    How
    Basically the system just automatically creates new types of ISystemStateComponentData at runtime with a little inverse generic magic to allow the system to use EntityCommandBuffers to add and remove them without knowing the details.

    Limitations
    - Can currently only track a single component. I can easily add support to track multiple conditions for when a component is added, but inversing the condition is a little more difficult. I might look at it later. - Added here: https://forum.unity.com/threads/i-created-a-reactivecomponentsystem.539581/#post-3557084
    - Only limited testing at the moment, might break?!
    - ComponentSystem only, I can probably easily create a ReactiveJobComponentSystem version

    Where to get?
    https://github.com/tertle/ReactiveComponentSystem/
     
    Last edited: Jul 25, 2018
  2. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    Just after writing this post, I immediately ran into a situation where I wanted to track a group of components so I've updated it.

    GetReactiveAddGroup and GetReactiveRemoveGroup both now take params ComponentType[] componentTypes just like GetComponentGroup does.

    Code (CSharp):
    1. using BovineLabs.Toolkit.ECS;
    2. using Unity.Entities;
    3.  
    4. namespace BovineLabs
    5. {
    6.     public struct Test1 : IComponentData
    7.     {
    8.    
    9.     }
    10.  
    11.     public struct Test2 : IComponentData
    12.     {
    13.    
    14.     }
    15.  
    16.     public class TestReactive : ReactiveComponentSystem
    17.     {
    18.         private ComponentGroup _addGroup;
    19.         private ComponentGroup _removeGroup;
    20.         protected override void OnCreateManager(int capacity)
    21.         {
    22.             _addGroup = GetReactiveAddGroup(ComponentType.ReadOnly<Test1>(), ComponentType.Subtractive<Test2>());
    23.             _removeGroup = GetReactiveRemoveGroup(ComponentType.ReadOnly<Test1>(), ComponentType.Subtractive<Test2>());
    24.         }
    25.         protected override void OnReactiveUpdate()
    26.         {
    27.             var addedEntities = _addGroup.GetEntityArray();
    28.             var removedEntities = _removeGroup.GetEntityArray();
    29.  
    30.             // DO WORK
    31.         }
    32.     }
    33. }
    Note, remove is inverted, this might cause unwanted behaviour for Subtractive components
    - the first time Test1 is added and Test2 does not exist on a component it'll run.
    - the first time Test1 was removed and test2 has been added the remove will run

    I'm not sure if this ideal behaviour. Maybe I should treat subtractive different and not invert it?


    -edit-

    Has been updated, detailed here: https://forum.unity.com/threads/i-created-a-reactivecomponentsystem.539581/#post-3565997
     
    Last edited: Jul 16, 2018
  3. Deadcow_

    Deadcow_

    Joined:
    Mar 13, 2014
    Posts:
    125
    Fantastic! Thanks a lot.
    It'll be cool to have GetReactiveUpdateGroup or something like this for entries with changed/replaced component values too. It wont use ISystemStateComponentData but it feels right to have it in Reactive System
     
  4. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    I forgot I had an update for this that I never pushed following up my previous discussion I was having with myself.

    I settled on a solution that I've been using for a week now and it's working how I like it.

    Few changes

    1. Can no longer use ComponentType.Subtractive<T> in the tracking types (can do in conditions I'll get to that). It'll throw an exception. It made no sense, doing subtractive just inverted the conditions (add became remove, remove became add).
    2. There are now 4 instead of 2 methods, the 2 existing methods and 2 new component group methods with conditions

    Code (CSharp):
    1. ComponentGroup GetReactiveAddGroup(params ComponentType[] componentTypes)
    2. ComponentGroup GetReactiveRemoveGroup(params ComponentType[] componentTypes)
    3. ComponentGroup GetReactiveAddGroup(ComponentType[] componentTypes, ComponentType[] conditionTypes)
    4. ComponentGroup GetReactiveRemoveGroup(ComponentType[] componentTypes, ComponentType[] conditionTypes)
    These new methods give you the option to add a condition to the reactive group.

    The first 2 work similar to how it previously worked, except for the no subtractive part.

    GetReactiveAddGroup(typeof(ComponentA), ComponentType.ReadOnly<ComponentB>())


    Would create a reactive group that will trigger add when both component A and B are added to an entity.
    It will only trigger remove when BOTH componentA and componentB are removed.
    The ComponentGroup for Add will contain both ComponentA and ComponentB.
    The ComponentGroup for Remove will contain nothing except an entity set.

    The 2 new methods allow you to specify a condition to the reaction.
    GetReactiveAddGroup(new ComponentType[] {typeof(ComponentA), ComponentType.ReadOnly<ComponentB>()}, new ComponentType[] {ComponentType.Subtractive<ComponentC>())


    In this case, it would create a reactive group when both component A and B are added to an entity BUT only if the entity does not have ComponentC

    It will only trigger remove when BOTH componentA and componentB and removed. It does not depend on the condition for removal, however remove can not trigger before an Add so the condition must have been triggered beforehand.
    This means if you add ConditionC to the entity, after the reactive Add has triggered, this will not trigger the remove. Only removing both components will. You'd need a second reactive group to track that behaviour.

    The ComponentGroup for Add will contain both ComponentA and ComponentB.
    The ComponentGroup for Remove will contain nothing except an entity set.

    Note about componentGroup for the conditional one. It will include the condition in the group. For instance.
    GetReactiveAddGroup(new ComponentType[] {typeof(ComponentA), ComponentType.ReadOnly<ComponentB>()}, new ComponentType[] {typeof(ComponentC))

    Add group will have ComponentA, ComponentB, ComponentC in the Add group but again only the Entity in the remove group.

    The purpose it was designed this way is because most of the time, at least when I want to track a remove is after an entity has been destroyed and in this case all components will have been removed hence I can only create a subtractive group.
     
    Last edited: Jul 16, 2018
  5. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    I actually have an idea how I could do that by extending what I have here. If I find some time today maybe I'll look into it.
     
  6. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    Here we go! Full reactive system. Reacts to value changes on a componentData.

    Here it is in action

    upload_2018-7-17_10-8-14.png

    Demo code

    Code (CSharp):
    1. using BovineLabs.Toolkit.ECS;
    2. using Unity.Entities;
    3. using UnityEngine;
    4.  
    5. public struct Test : IComponentData
    6. {
    7.     public int TestValue;
    8. }
    9.  
    10. public class TestSystem : ReactiveComponentSystem
    11. {
    12.     private ComponentGroup _group;
    13.  
    14.     protected override void OnCreateManager(int capacity)
    15.     {
    16.         _group = GetReactiveUpdateGroup(ComponentType.ReadOnly<Test>());
    17.     }
    18.  
    19.     protected override void OnReactiveUpdate()
    20.     {
    21.         var tests = _group.GetComponentDataArray<Test>();
    22.         for (var index = 0; index < tests.Length; index++)
    23.         {
    24.             Debug.Log("ReactiveComponentSystem: Test Changed, New Value: " + tests[index].TestValue);
    25.         }
    26.     }
    27. }
    28.  
    Usage
    - Implement ReactiveComponentSystem
    - Use GetReactiveUpdateGroup(Type) to create a ComponentGroup to track when a component is changed

    Limitations
    - Can currently only track a single component, I'd have to dynamically create some generic types to get around this - might look at it later
    - Only limited testing at the moment, might break?!
    - ComponentSystem only
    - Will not trigger on initial component being added, only on first change. To also look at first value, use the GetReactiveAddGroup.

    Note: literally just finished writing this and only did a single test to see if it works. Have some other stuff to do so will go back and do some more testing later.

    -edit-

    I've committed a few performance optimizations (turned ReactiveCompareSystem into a JobComponentSystem and a few other things.) Restructured as well.

    A few more things to note:

    It'll create a new ReactiveCompareSystem<T, TC> for every update group you create. So if you have a lot of reactive groups, it'll fill your debugger - nothing I can do about this.

    upload_2018-7-17_11-47-7.png

    This system is responsible for checking if the value has changed.

    For an Add or Remove group, it'll create both the add and remove groups even if you don't use one.
    For update, it'll create 3 groups, add, remove and an update group.
    These are cached so only created once if you use add and update for example.

    For add/remove it'll create some weird components in your debugger, this is because it's creating types on the fly at runtime.

    Example

    upload_2018-7-17_11-49-7.png

    Finally, for the update system I'm going to have a look into a way to passing the previous value in as well.

    Performance

    Comparing 10000 components with an int field.

    upload_2018-7-17_12-3-54.png

    Little over 3 ms on my 3570k (little old, only 4 core)
     
    Last edited: Jul 17, 2018
  7. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    Couldn't attach anymore images to previous post, but here is 1million component comparisons

    upload_2018-7-17_11-57-29.png

    377ms, probably not feasible.

    I'm have to do a compare every frame in a job to see a change.

    Code (CSharp):
    1. // [BurstCompile] Burst does not support EntityCommandBuffer yet
    2. private struct CompareJob : IJobParallelFor
    3. {
    4.     public EntityCommandBuffer.Concurrent CommandBuffer;
    5.  
    6.     [ReadOnly] public EntityArray Entities;
    7.     [ReadOnly] public ComponentDataArray<T> Components;
    8.     [ReadOnly] public ComponentDataArray<TC> Previous;
    9.  
    10.     public void Execute(int index)
    11.     {
    12.         // Hasn't changed
    13.         if (Previous[index].Equals(Components[index]))
    14.             return;
    15.  
    16.         var previous = Previous[index];
    17.         previous.Set(Components[index]);
    18.         CommandBuffer.SetComponent(Entities[index], previous);
    19.         CommandBuffer.AddComponent(Entities[index], new ReactiveChanged());
    20.     }
    21. }
    I could do this much faster as well faster if I enforced using custom extension methods
    EntityManager.SetComponentNotify()
    EntityCommandBuffer.SetComponentNotify()

    Unity devs could do this much faster by simply firing it off within the EntityManager when Set() is called.

    Does fill your worker threads up nicely though

    upload_2018-7-17_12-3-5.png
     
    Last edited: Jul 17, 2018
    Deadcow_ and FROS7 like this.
  8. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    Significantly updated. Fixed all issues I found with original implementation - if you had 2 different systems tracking same entity but different component, if 1 component changed both systems would trigger. Now only the correct system triggers.

    I also added a ReactiveJobComponentSystem

    The only thing to note about this is you need to call base.OnCreateManager

    Code (CSharp):
    1. protected override void OnCreateManager(int capacity)
    2. {
    3.     base.OnCreateManager(capacity);
    It will throw an exception and warn you if you forget.

    I might change this and seal it and add a OnCreateReactiveManager instead or something

    upload_2018-7-19_10-46-17.png

    This picture somewhat explains how it works atm.

    This is what it looks like if you have 4 different reactive systems going at once (2 job component, 2 component).

    Basically you have a separate reactivecompare system that runs early to check if the component being tracked changed since last frame, then you have a late system that cleans up the entity.

    It'll also create a ReactiveBarrierSystem which is shared by all ReactiveJobSystems.

    As per previous post, I probably wouldn't use the change reactive system on components with more than 1000 entities.

    Also override your Equals on componentdata for (much) better performance
     
    Last edited: Jul 19, 2018
    rigidbuddy, Deadcow_ and Alverik like this.
  9. Deadcow_

    Deadcow_

    Joined:
    Mar 13, 2014
    Posts:
    125
    I'm not tested it much but as soon as I create one entity my ReactiveAddGroup system constantly spamming with this error (every frame)

    Code (CSharp):
    1. InvalidOperationException: The NativeArray AddJob.Entities must be marked [ReadOnly] in the job ReactiveGroup`1:AddJob, because the container itself is marked read only.
    2. Unity.Jobs.LowLevel.Unsafe.JobsUtility.ScheduleParallelFor (Unity.Jobs.LowLevel.Unsafe.JobsUtility+JobScheduleParameters& parameters, System.Int32 arrayLength, System.Int32 innerloopBatchCount) <0x3690d110 + 0x0007a> in <c53fd6a30f3a48df8726a5828f86721f>:0
    3. Unity.Jobs.IJobParallelForExtensions.Schedule[T] (T jobData, System.Int32 arrayLength, System.Int32 innerloopBatchCount, Unity.Jobs.JobHandle dependsOn) (at C:/buildslave/unity/build/Runtime/Jobs/Managed/IJobParallelFor.cs:51)
    4. BovineLabs.Toolkit.Reactive.ReactiveTypeHelper+ReactiveGroup`1[T].CreateAddJob (Unity.Jobs.JobHandle inputDeps, Unity.Entities.EntityArray entities, Unity.Entities.EntityCommandBuffer commandBuffer) (at Assets/Plugins/MyBox/ECS/Reactive/ReactiveTypeHelper.cs:189)
    5. BovineLabs.Toolkit.Reactive.ReactiveComponentSystem.OnUpdate () (at Assets/Plugins/MyBox/ECS/Reactive/ReactiveComponentSystem.cs:71)
    6. Unity.Entities.ComponentSystem.InternalUpdate () (at C:/Users/deadc/AppData/Local/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.12-preview.8/Unity.Entities/ComponentSystem.cs:294)
    7. Unity.Entities.ScriptBehaviourManager.Update () (at C:/Users/deadc/AppData/Local/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.12-preview.8/Unity.Entities/ScriptBehaviourManager.cs:82)
    8. Unity.Entities.ScriptBehaviourUpdateOrder+DummyDelagateWrapper.TriggerUpdate () (at C:/Users/deadc/AppData/Local/Unity/cache/packages/staging-packages.unity.com/com.unity.entities@0.0.12-preview.8/Unity.Entities/ScriptBehaviourUpdateOrder.cs:734)
    It seemed that this one is easy to fix, I'll check it out tomorrow
     
  10. Deadcow_

    Deadcow_

    Joined:
    Mar 13, 2014
    Posts:
    125
    I finally got to look at this error closely and I just don't get it, AddJob.Entities is already marked ReadOnly. Probably ECS analyzer lies because of reflection stuff and I don't get it much. Anyone else is getting this error? Maybe I'm doing something wrong in the first place :confused:
     
  11. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,198
    I can't reproduce it. What version of ecs, burst etc.
     
  12. madks13

    madks13

    Joined:
    May 8, 2016
    Posts:
    168
    Sorry about the Graph talk, i didn't see i was in the wrong tab.
     
    Last edited: Aug 8, 2018
  13. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,289
    madks13
    I presume, you means some form of template/blueprint?
     
  14. madks13

    madks13

    Joined:
    May 8, 2016
    Posts:
    168
    My tab with ShaderGraph feedback is beside this one, mixed up the 2, sorry.
     
  15. rigidbuddy

    rigidbuddy

    Joined:
    Feb 25, 2014
    Posts:
    36
    tertle
    Thank you for the share! But update system doesn't detect component change - it's not running and OnReactiveUpdate never got called in last commit 12155280.

    Though it worked in previous commit (2cd516c1). But there're issues when component got removed (reproduced in TwoStickShooter sample project)
     
    Last edited: Sep 27, 2018
unityunity