Chaining filtering jobs using component tags, one after the other

Let's say I have a bunch of objects called interactors that will be managed with multiple jobs similar to the one below. Those jobs need to be chained one after the other and will add/remove a ValidInteractorTag to filter in/out which entities need to be processed in the subsequent job. The system below will be run after a DistanceSystem that filters interactors that are too far away. It will in turn filter interactors that are not in view for subsequent processes.

Code (CSharp):

1. using Unity.Collections;
2. using Unity.Entities;
3. using Unity.Jobs;
4.  
5. namespace InteractionSystem.Runtime
6. {
7.     [UpdateAfter(typeof(DistanceSystem))]
8.     public class VisibilitySystem : JobComponentSystem
9.     {
10.         [RequireComponentTag(typeof(ValidInteractorTag))]
11.         private struct IsInFrustrumJob : IJobForEachWithEntity<Interactor>
12.         {
13.             [WriteOnly] public EntityCommandBuffer.Concurrent CommandBuffer;
14.            
15.             public void Execute(Entity entity, int index, ref Interactor c0)
16.             {
17.                 // determine if filtered out
18.                 if (filteredIn) {
19.                     // Remove ValidInteractorTag command on CommandBuffer?
20.                 }
21.             }
22.         }
23.    
24.         protected override JobHandle OnUpdate(JobHandle inputDeps)
25.         {
26.             // create some command buffer?
27.             var job  = new IsInFrustrumJob
28.             {
29.                 CommandBuffer = someCommandBuffer
30.             }.Schedule(this, inputDeps);
31.             // Manage command buffer somehow?
32.             return job;
33.         }
34.     }
35. }
36.  

Every kind of command buffer I came across don't seem to be removing ValidInteractorTag before the next system is updated.

I could switch booleans on the interactor component, but it would be faster to do it this way, since only a really small portion of possibly thousands of interactors will be close enough to get a ValidInteractorTag in the DistanceSystem step. Also, the last steps at the end will be pretty costly, with raycasts and such, so I'd like to keep those to a minimum.

I'd really like if it was possible for all those system to be run one after the other, with the previous system results, on the same frame. Is there a way to do this?

 

namespace InteractionSystem.Runtime {
// Here declare a command buffer to use.
// There are several to use depending upon where in the system you want it to run.
EndSimulationEntityCommandBufferSystem _someCommandBuffer;

protected override void OnCreate() {
// get a ref to the command buffer
_someCommandBuffer = World.GetOrCreateSystem<EndSimulationEntityCommandBufferSystem>();
}

..in job..
public EntityCommandBuffer.Concurrent SomeCommandBuffer; // no [writeonly] !!!

..setting up job..
var job = new IsInFrustrumJob {
SomeCommandBuffer = _someCommandBuffer.CreateCommandBuffer().ToConcurrent()
},schedule(this, inputdeps);
_someCommandBufferSystem.AddJobHandleForProducer(job);
return job;


Hope that helps.
https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/entity_command_buffer.html
https://docs.unity3d.com/Packages/com.unity.entities@0.0/manual/system_update_order.html

 

Oh, and...

..in job...
SomeCommandBuffer.RemoveComponent<ValidInteractorTag>(entity);

 

And it’s wrong Cos EndSimulationEntityCommandBufferSystem runs at the end of sim group and it doesn’t give results between systems.
@mxmcharbonneau you should declare your own barriers between systems, set order by UpdateBefore/UpdateAfter, and use them.

Code (CSharp):

1. public class SomeMono: MonoBehaviour
2. {
3.     private void Awake()
4.     {
5.         var archetype = World.Active.EntityManager.CreateArchetype(typeof(SomeComp));
6.         var arr = new NativeArray<Entity>(100, Allocator.TempJob);
7.         World.Active.EntityManager.CreateEntity(archetype, arr);
8.         arr.Dispose();
9.     }
10. }
11.  
12. public struct SomeComp: IComponentData {}
13. public struct ExcludeComp: IComponentData {}
14.  
15. public struct ReduceJob : IJobForEachWithEntity<SomeComp>
16. {
17.     public EntityCommandBuffer.Concurrent buff;
18.     public void Execute(Entity e, int index, [ReadOnly] ref SomeComp c)
19.     {
20.         if(index % 2 == 0)
21.             buff.AddComponent(index, e, new ExcludeComp());
22.     }
23. }
24.  
25. public class SystemA : JobComponentSystem
26. {
27.     private EntityQuery _query;
28.     private BarrierAfterA _barrier;
29.  
30.     protected override void OnCreateManager()
31.     {
32.         _barrier = World.GetOrCreateSystem<BarrierAfterA>();
33.         _query = GetEntityQuery(typeof(SomeComp), ComponentType.Exclude<ExcludeComp>());
34.     }
35.  
36.     protected override JobHandle OnUpdate(JobHandle inputDeps)
37.     {
38.         Debug.Log(_query.CalculateLength());
39.         _barrier.AddJobHandleForProducer(new ReduceJob()
40.         {
41.             buff = _barrier.CreateCommandBuffer().ToConcurrent()
42.         }.Schedule(_query, inputDeps));
43.         return inputDeps;
44.     }
45. }
46.  
47. [UpdateAfter(typeof(SystemA))]
48. public class BarrierAfterA: EntityCommandBufferSystem {}
49.  
50. [UpdateAfter(typeof(BarrierAfterA))]
51. public class SystemB : JobComponentSystem
52. {
53.     private EntityQuery _query;
54.     private BarrierAfterB _barrier;
55.  
56.     protected override void OnCreateManager()
57.     {
58.         _barrier = World.GetOrCreateSystem<BarrierAfterB>();
59.         _query = GetEntityQuery(typeof(SomeComp), ComponentType.Exclude<ExcludeComp>());
60.     }
61.  
62.     protected override JobHandle OnUpdate(JobHandle inputDeps)
63.     {
64.         Debug.Log(_query.CalculateLength());
65.         _barrier.AddJobHandleForProducer(new ReduceJob()
66.         {
67.             buff = _barrier.CreateCommandBuffer().ToConcurrent()
68.         }.Schedule(_query, inputDeps));
69.         return inputDeps;
70.     }
71. }
72.  
73. [UpdateAfter(typeof(SystemB))]
74. public class BarrierAfterB: EntityCommandBufferSystem {}
75.  
76. [UpdateAfter(typeof(BarrierAfterB))]
77. public class SystemC : JobComponentSystem
78. {
79.     private EntityQuery   _query;
80.  
81.     protected override void OnCreateManager()
82.     {
83.         _query   = GetEntityQuery(typeof(SomeComp), ComponentType.Exclude<ExcludeComp>());
84.     }
85.  
86.     protected override JobHandle OnUpdate(JobHandle inputDeps)
87.     {
88.         Debug.Log(_query.CalculateLength());
89.         return inputDeps;
90.     }
91. }

 

[...and that's why you don't answer questions on your phone at midnight]

thanks @eizenhorn i was thinking of my spawning stuff

 

I've been experimenting with that method for a while, and it works as expected. However, I have an issue with both CommandBuffers Playback being heavier than I expected (about 0.66 ms, profiler shown below). Note that in the example I'm testing, I've got probably 60 AddComponents in my command buffer.

What I have is this:

Code (CSharp):

1.     public class DistanceSystem : JobComponentSystem
2.     {
3.         private const float MaxDistance = 3;
4.        
5.         private DistanceBarrierSystem barrier;
6.         private EntityQuery deleteQuery;
7.        
8.         private struct IsInDistanceJob : IJobForEachWithEntity<Interactor, Translation>
9.         {
10.             public EntityCommandBuffer.Concurrent CommandBuffer;
11.             public float3 CameraPosition;
12.             public float MaxSqrDistance;
13.            
14.             public void Execute(Entity entity, int index,
15.                 [ReadOnly] ref Interactor interactor, [ReadOnly] ref Translation translation)
16.             {
17.                 var vector = translation.Value - CameraPosition;
18.                 var sqrDistance = math.dot(vector, vector);
19.                 if (sqrDistance < MaxSqrDistance)
20.                 {
21.                     CommandBuffer.AddComponent(index, entity, new ValidInteractorTag());
22.                 }
23.             }
24.         }
25.  
26.         protected override void OnCreate()
27.         {
28.             base.OnCreate();
29.  
30.             barrier = World.GetOrCreateSystem<DistanceBarrierSystem>();
31.         }
32.  
33.         protected override JobHandle OnUpdate(JobHandle inputDeps)
34.         {
35.             var jobHandle = new IsInDistanceJob
36.             {
37.                 CommandBuffer = barrier.CreateCommandBuffer().ToConcurrent(),
38.                 MaxSqrDistance = math.lengthsq(MaxDistance),
39.                 CameraPosition = Camera.main.transform.position,
40.             }.Schedule(this, inputDeps);
41.            
42.             barrier.AddJobHandleForProducer(jobHandle);
43.             return jobHandle;
44.         }
45.     }

Code (CSharp):

1.     [UpdateAfter(typeof(DistanceSystem))]
2.     public class DistanceBarrierSystem : EntityCommandBufferSystem {}

Both Interactor and ValidInteractorTag are empty IComponentData. I have other, heavier calculations down the line, after the filtering. Is there any way I can speed this up, or as soon as I AddComponent, the memory copy from chunk to chunk will add that kind of overhead? If not, is there a better way I could filter my data, so I don't have to use the entire set of data for my subsequent calculations?

 

You are creating structural changes where if I understand what you are doing correctly, just a simple boolean field on Interactor would probably suffice. Iterating over all entities and just filtering by the bool is likely so much more performant then command buffers that it would take a few thousand iterations before command buffers win.

Also factor in you likely don't need this run every frame. For a lot of distance related logic simply not running every frame is really the best optimization.

Filling some sort of collection of entities in range to be used by later jobs is always going to win over an approach that requires a structural change per entity. Lots of structural changes on a constant basis you want to find alternative designs for. It's not an approach that will scale well as your project grows.

So I would first try to get away with filling a NativeArray or using DynamicBuffer. You are already using a linear approach to distance filtering so that would fit well.

Once you get to where you need a spatial structure, Unity.Physics is actually pretty good here, albeit a bit of a learning curve. Although if linear performs fine I would stick with it, use the simplest approach that works IMO.