How to delete entities or remove components in IJobProcessComponentData Jobs

Hi everyone!

Currently my approach to deleting entities or adding/removing IComponentData to an entity is really slow.
For example destroying an entity:
I have JobComponentSystems that schedule IJobProcessComponentData jobs. For entities that are destructible I have an additional component "Destructible" I pass to the IJobProcessComponentData job. When my logic determines that the entities passing the job should be destroyed (for example exceeding a certain lifetime), I set a value in the Destructible component data from 0 to 1. (death flag).

In the regular ComponentSystem that runs on the main thread I then iterate through all Destructible components to see if any entity is marked to be destroyed by checking the flag. If it is 1 I schedule a PostUpdateCommand. But as you may know, iterating through 1000s of entities and returning the struct by value each time (since ref returns are not supported) can take quite some time, especially if the components have more data fields. So it is a very slow approach overall.

My question is if there is a better way to do this. destroying entities or using PostUpdateCommand may not be supported by IJobProcessComponentData jobs but I thought of having a global thread safe collection (like a ConcurrentBag) where I store commands that get executed in the ComponentSystem OnUpdate without having to iterate through all components again.

So my approach would be:
- Storing the entity ID in according component data structs,
- performing some logic in the IJobProcessComponentData job, if the logic determines that the entity should be destroyed, I issue something like a command struct to the thread safe command buffer list or queue.
- In the ComponentSystem I just have to iterate through the command queue and do not have to iterate through all components that implement a "Destructible" component. Then I apply the according commands with PostUpdateCommand calls.

But there are some problems:
1. I couldn't find a way to retrieve an entity solely by the entity ID from the EntityManager. Since entities are structs too, would it be legit to store the whole entity instead in my component data that is used by logic that can determine if the entity should be destroyed?
2. I don't know how I could how I could implement generic removal or add functions for components on entities: for example:

Code (CSharp):

1.  PostUpdateCommands.RemoveComponent<DeathCheckComponent>(bulletGroup.entityArray[i]);

requires a generic type argument. I have no idea how to make a modular command that would enable me to remove specific component or add a specific component. But maybe this isn't necessary in most, since I could make one command buffer for every ComponentSystem. RemoveComponent (always the component on which the system operates) and DestroyEntity would be supported then, but not AddComponent.


Reading through the Twin Stick Shooter tutorial page on github
https://github.com/Unity-Technologi...er/Documentation/content/two_stick_shooter.md
there is the statement that ref returns are not implemented yet, but it is planned to support the newest version of C# in the future and support the feature. This would make the iterations through all components significantly faster.

 

Why not use BarrierSystem? Inject a BarrierSystem, create a command buffer, set it as WriteOnly insde the IJob and write to it whenver you need to delete an entity. It would be something like this:

Code (CSharp):

1.  
2. using Unity.Collections;
3. using Unity.Entities;
4. using Unity.Jobs;
5. using UnityEngine.Jobs;
6. public class MyBarrier : BarrierSystem { }
7. public class MyJobSystem : JobComponentSystem
8. {
9.     public struct Data
10.     {
11.         public int Length;
12.         [ReadOnly] public EntityArray Entities;
13.         [ReadOnly] public ComponentDataArray<Destructible> Destructibles;
14.     }
15.     [Inject] private Data m_Data;
16.  
17.     [Inject] private MyBarrier m_Barrier;
18.  
19.     protected override JobHandle OnUpdate(JobHandle inputDeps)
20.     {
21.         var job = new DestructionJob()
22.         {
23.             Cmd = m_Barrier.CreateCommandBuffer(),
24.             Entities = m_Data.Entities
25.         }.Schedule(m_Data, 1, inputDeps);
26.         return job;
27.     }
28.  
29.     [ComputeJobOptimization]
30.     struct DestructionJob : IJobParallelFor
31.     {
32.         //Since it's a write only, it should have no problem running in parallel
33.         [WriteOnly] public EntityCommandBuffer Cmd;
34.         [ReadOnly] public EntityArray Entities;
35.         public void Execute(int index)
36.         {
37.             Cmd.DestroyEntity(Entities[index]);
38.         }
39.     }
40. }

Note: I didn't tested te above code, just wrote it to you have an idea of what I'm talking about.

 

Thanks for helping once again! I'll try this approach.

 

InvalidOperationException: DestructionJob.Data.Cmd is not declared [ReadOnly] in a IJobParallelFor job. The container does not support parallel writing. Please use a more suitable container type.
Unity.Jobs.LowLevel.Unsafe.JobsUtility.CreateJobReflectionData (System.Type wrapperJobType, System.Type userJobType, Unity.Jobs.LowLevel.Unsafe.JobType jobType, System.Object managedJobFunction0) (at C:/buildslave/unity/build/Runtime/Jobs/ScriptBindings/Jobs.bindings.cs:101)
Unity.Entities.JobProcessComponentDataExtensions+JobStruct_Process2`3[T,U0,U1].Initialize (Unity.Job).

So no, sadly it's not working like that. I'll try my approach again.

 

Okay, here is my hack. In order to access the ConcurrentPack, [ComputeJobOptimization] has to be disabled.

Code (CSharp):

1. public struct EntityInformation : IComponentData
2.     {
3.         public Entity parentEntity;
4.     }
5.  
6. public struct EntityChangeCommand
7.     {
8.         public Entity entity;
9.         public IComponentData component;
10.         public EEntityCommandAction action;
11.     }
12.  
13. public enum EEntityCommandAction
14.     {
15.         None = 0,
16.         Remove = 1,
17.         Create = 2,
18.         DeleteEntity = 3
19.     }
20.  
21. //[ComputeJobOptimization] //static field not supported by burst!!
22.     public struct DestructionJob : IJobProcessComponentData<DeathCheckComponent, EntityInformation>
23.     {
24.  
25.         public void Execute(ref DeathCheckComponent data, ref EntityInformation entityInfo)
26.         {
27.             if (data.deathFlag > 0)
28.             {
29.                 BulletBDSystem.entityCommands.Add(new EntityChangeCommand
30.                 {
31.                     action = EEntityCommandAction.DeleteEntity,
32.                     entity = entityInfo.parentEntity
33.                 });
34.             }
35.         }
36.  
37.     }
38.  
39.     public class BulletBDDestructionJobSystem : JobComponentSystem
40.     {
41.  
42.         protected override JobHandle OnUpdate(JobHandle inputDeps)
43.         {
44.             var job = new DestructionJob();
45.             return job.Schedule(this, 30, inputDeps);
46.         }
47.     }
48.  
49.  
50. //In my ComponentSystem  BulletBDSystem:
51.  
52. public static ConcurrentBag<EntityChangeCommand> entityCommands = new ConcurrentBag<EntityChangeCommand>();
53.  
54. //in the OnUpdate()
55.  
56. EntityChangeCommand command;
57.             while (!entityCommands.IsEmpty)
58.             {      
59.                 if (entityCommands.TryTake(out command))
60.                 {
61.                     if (command.action == EEntityCommandAction.DeleteEntity)
62.                     {
63.                         PostUpdateCommands.DestroyEntity(command.entity);
64.                     }
65.                 }
66.                 else {
67.                     //bag empty
68.                 }
69.                        
70.             }  

 

In the future they said statics from inside the job would be disallowed completely, so I don't recommend this approach.

If it's me the death flag would not be a 0-1 state but just an empty separated ISharedComponentData attached to parentEntity. Then a normal ComponentSystem could inject all of them with EntityArray, then use PostUpdateCommand DestroyEntity to destroy them. I have something like this to clean up all that was attached with a certain ISharedComponentData. Should not be slow as all the injects are still pointers to value and ISharedComponentData should make all of them in the same chunk.

Code (CSharp):

1.     [UpdateBefore(typeof(Initialization))]
2.     public class DestroyReactivesSystem : ComponentSystem
3.     {
4.         public struct ReactiveEntity : ISharedComponentData { }
5.  
6.         struct AllReactives
7.         {
8.             [ReadOnly] public SharedComponentDataArray<ReactiveEntity> reactiveEntities;
9.             [ReadOnly] public EntityArray entities;
10.         }
11.         [Inject] AllReactives allReactives;
12.         protected override void OnUpdate()
13.         {
14.             for (int i = 0; i < allReactives.entities.Length; i++)
15.             {
16.                 PostUpdateCommands.DestroyEntity(allReactives.entities[i]);
17.             }
18.         }
19.     }

 

Hi, thanks for the hint,

That's very unfortunate, my approach with the static ConcurrentBag works really well (you just can't use the burst compiler with it, hence no [ComputeJobOptimization]

Your implementation misses something: When//where do I attach the reactive component that results in destruction?
Because I can't do that inside a IJobProcessComponentData job. If postUpdateCommands are supported in jobs in the future, this would be another story. I get the architecture you are going for, but I can't really find a good way to use it with the exception that you change and operate on entities in a ComponentSystem. But I want to use jobs as much as possible for concurrent execution of tasks, if I had the logic inside the ComponentSystem, it would only use the main thread (because I still have to make checks and then attach the ReactiveEntity, can't do that in a Job.

 

Ah right, I attach that component in a regular IJob (where RW EntityCommandBuffer works) but not a parallel job.

If I am to parallelize the removal command without statics, I would send the job with NativeArray<(Entity,EEntityCommandAction)> and IJobParallelFor where we can use the index. I am not sure about NativeList whether it supported parallelized Add or not. If it is then you could also use IJobProcessComponentData?

NativeArray<(Entity,EEntityCommandAction)> would be a public member belongs to the system running that job. Then later at the correct time other system could inject this system, then get the public field and loop through them to execute all queued commands. (You might need to .Complete the job that process the native array to make sure, because the removal system will be in the main thread and does not know about inputDeps)

 

Oh wait, NativeQueue/NativeHashMap/NativeMultiHashMap all have a Concurrent version. I think this is exactly what you would want to use. (They are castable to their .Concurrent version with reduced functionality to just .Enqueue/.Add)

Code (CSharp):

1. var nq = new NativeQueue<(Entity target, EEntityCommandAction command)>(Allocator.Persistent);

In a job :

Code (CSharp):

1.  
2.     struct Job : IJobProcessComponentData<Data1, Data2>
3.     {
4.         public NativeQueue<(Entity, EEntityCommandAction)>.Concurrent nqConcurrent;
5.         public void Execute(ref Data1 d1, ref Data2 d2)
6.         {
7.             nqConcurrent.Enqueue((d1.Entity, EEntityCommandAction.DeleteEntity));
8.         }
9.     }
10.  

Later :

Code (CSharp):

1.         while (nq.TryDequeue(out var pair))
2.         {
3.             pair.target, pair.command ....
4.         }

 

That's a really cool find!
But I don't know how I could access the NativeQueue from a IJobProcessComponentData since it is basically schedule the job and forget. I think it would work in an IJobParallelFor though. For now I'll leave my system as it is, but I would definitely go with your system if they are to scratch static member access inside jobs.

I really hope Unity won't remove too many features of what is currently possible with the ECS. Because they stated that they allow for more experimentation first and then settle to the approach that is most commonly used and restrict the others. At least that was my interpretation of it.

 

When creating a job and before scheduling you can setup a job with that NativeQueue. In all it might looks like this :

Code (CSharp):

1. public class Sample : JobComponentSystem
2. {
3.     public NativeQueue<int> nq;
4.     public JobHandle jh;
5.     protected override void OnCreateManager(int capacity)
6.     {
7.         nq = new NativeQueue<int>(Allocator.Persistent);
8.     }
9.    
10.     protected override void OnDestroyManager()
11.     {
12.         nq.Dispose();
13.     }
14.  
15.     protected override JobHandle OnUpdate(JobHandle inputDeps)
16.     {
17.         var job = new Job()
18.         {
19.             cnq = nq
20.         };
21.         jh = job.Schedule(this, inputDeps);
22.         return jh;
23.     }
24.    
25.     struct Job : IJobProcessComponentData<Data>
26.     {
27.         public NativeQueue<int>.Concurrent cnq;
28.         public void Execute(ref Data d)
29.         {
30.             cnq.Enqueue(...);
31.         }
32.     }
33. }
34.  
35. [UpdateAfter(typeof(Sample))]
36. public class Sample2 : ComponentSystem
37. {
38.     [Inject] Sample sampleSystem;
39.     protected override void OnUpdate()
40.     {
41.         sampleSystem.jh.Complete();
42.         sampleSystem.nq.Dequeue(); //use it
43.     }
44. }

 

That approach works. Great, thanks!

 

As for new entity package there is an

EntityCommandBuffer.Concurrent

. Maybe you should try.

 

Wow! You are so lucky that the perfect solution arrives as soon as you want it.. (https://github.com/Unity-Technologies/EntityComponentSystemSamples/blob/master/ReleaseNotes.md)