Help Wanted, How to efficiently divide entities into different chunks?

Hi,

I tried using the shared component, which only works on the mainthread. The problem is that it cannot be burst compiled or multithreaded.

In my case the entities can exists anywhere, so they cannot be divided on creation.

So the performance penalty comes from the mainthread.
If there would be faster way to divide entities into chunks it would boost performance significantly.

I have checked that I'm not creating too many chunks. I have about 60 chunks in scene where is spawning, moving and despawning thousands of entities.

Code (CSharp):

1.  
2. List<TargetChunkSharedValues> cohorts = new List<TargetChunkSharedValues>();
3. EntityManager.GetAllUniqueSharedComponentData(cohorts);
4. float3 newChunkPosition;
5. foreach (TargetChunkSharedValues sharedGroups in cohorts)
6. {
7.     var newChunkSharedData = new TargetChunkSharedValues();
8.     var currentChunkIndexIn = sharedGroups.ChunkIndex;
9.     Entities
10.         .WithoutBurst()
11.         .WithName("InitSharedComponentData")
12.         .WithSharedComponentFilter(sharedGroups)
13.         .WithStructuralChanges()
14.         .ForEach((int entityInQueryIndex,
15.             ref Target targetIn,
16.             in Translation translation) =>
17.         {
18.        
19.             var newChunkIndex = GeometryVisionEntityUtilities.ChunkIndexCalculator(targetIn.position, GeometryVisionSettings.TargetChunkSize);
20.             //Moving to different chunk
21.             if ((newChunkIndex != currentChunkIndexIn).x)
22.             {
23.                 newChunkSharedData.ChunkIndex = newChunkIndex;
24.                 EntityManager.SetSharedComponentData(targetIn.entity, newChunkSharedData);
25.             }
26.             //Copy settings to chunk and push changes
27.             var chunkTargetComponent = EntityManager.GetChunkComponentData<TargetChunk>(targetIn.entity);
28.             newChunkPosition = GeometryVisionEntityUtilities.ChunkIndexToPositionCalculator(newChunkIndex, GeometryVisionSettings.TargetChunkSize);
29.             chunkTargetComponent.ChunkIndex = newChunkIndex;
30.             chunkTargetComponent.ChunkPosition = newChunkPosition;
31.             chunkTargetComponent.BoundsMin = GeometryVisionEntityUtilities.ChunkAABBGeneratorMin(newChunkPosition, GeometryVisionSettings.TargetChunkSize);
32.             chunkTargetComponent.BoundsMax = GeometryVisionEntityUtilities.ChunkAABBGeneratorMax(newChunkPosition, GeometryVisionSettings.TargetChunkSize);
33.             EntityManager.SetChunkComponentData(EntityManager.GetChunk(targetIn.entity),chunkTargetComponent);
34.         }).Run();
35.  
36.  
37.              

 

Here is the target creation.

Code (CSharp):

1. if (entitiesWithoutTargetComponent.IsEmptyIgnoreFilter == false)
2. {
3.     var entitiesWithOutComponent = entitiesWithoutTargetComponent.ToEntityArray(Allocator.Temp);
4.     entityManager.AddChunkComponentData(entitiesWithoutTargetComponent, new TargetChunk() {ChunkIndex = new int3(-10000000,-10000000,-1000000)});
5.     entityManager.AddSharedComponentData(entitiesWithoutTargetComponent, new TargetChunkSharedValues(){ChunkIndex = new int3(-10000000,-10000000,-1000000)});
6.     entityManager.AddComponent<Target>(entitiesWithOutComponent);
7.     entitiesWithOutComponent.Dispose();
8. }

 

So currently the system is being bottlenecked by Unitys entitymanager, which starts to slow down the more there are entities.

The more these entities are spawned/destroyed the more Unity needs to arrange them to groups until the fps starts to drop.

 

Here is my test scene.


I have no idea how is ecs suppose to handle this scene. to me it looks like its not using all the resources, but still fps is only 40-60

 

Here is my profiler. 74% goes to the structural changes. 26% is divided between rendering the transform system and

the actual code of the targeting system only takes 5%!

Maybe its better to use the hashmap approach rather than go for the chunks. Documentation should have warning or example how this chunk processing is suppose to work. I got the idea that the chunk system is made for cases like this

 

Well I guess the only way is to assign these to the entity archetypes before they are spawned. Then disable this automatic chunk organizer feature with strong mention on the plugins documentation

 

Ok I tried putting the values directly to the spawned entities and it seems in order this to work I would need to update the values constantly, which kill all the performance. I think I'm going to report this as a bug

 

I'm not sure I understand what you are trying to do, much less why you believe a bug exists when there is no expectation/reality comparison here.

Are you trying to split up your entities into chunks to coincide with some spatial partitioning structure? If so, I think what you actually want is to have a Burst job that creates tuples of Entities and shared component values for all entities that need their shared component changed and store that in a NativeList or something. Then have a non-Bursted job dump that into an ECB. Or just process those changes on the main thread.

 

So the question is why do you need your entities split into chunks?
Is it spatial querying? If so people underestimate how fast it is to store entities in a hashmap spatially.

On my machine, the job takes 1.20ms to generate a hashmap with 100,000 entities per frame and the work is entirely in worker threads

(x all the threads)

Most games aren't even going to be close to this count of entities. If you have a reasonable amount, let's say 2,000 entities we're talking a fraction of a ms

(x all the threads)


really though don't use SCD if you can't set it up on initialization. It's never performant to change SCD at runtime.

 

Yea so it seems.

I don't know why but I cant get the enitties divided in to correct chunks on the way you are suggesting(assign scd and chunk data before spawn).

I wanted to go chunks because it would have been ideal and easier to set up + convert the current system to it. + according to documentation this is ideal case for chunks.

The problem here I think is not just the scd. Its more that i can't put the Bounds min and max value to chunks without updating scd first and for some reason it requires constant update.

I have to update scd OnUpdate and use the same value for chunk component. If I try to do anything else, the chunks don't get correct values resulting the systems not being able to see their chunks.

Currently I have been looking at why this happens and I'm starting to lean on that the entitymanager messes up the data. For example I update the scd and chunk. Then entitymanager moves the entity to correct chunk, but its not the same chunk it was before and where I assigned the data. But even this assumption might be wrong since I have been trying to correct the chunk values afterwards. Maybe it ads th same value for all the chunks, but then again why does the constant update method work?

 

Here is the quick code I use to build entity before spawning. I wonder why it doesn't work.

Code (CSharp):

1. public void Convert(Entity entity, EntityManager entityManager, GameObjectConversionSystem conversionSystem)
2. {
3.     StreamSpawnPoint = this.transform.position;
4.     var primeEntity = conversionSystem.GetPrimaryEntity(Prefab);
5.  
6.     if (entityFilterComponentType != null)
7.     {
8.         entityManager.AddComponent(primeEntity, entityFilterComponentType);
9.     }
10.  
11.     var chunkIndex = GeometryVisionEntityUtilities.ChunkIndexCalculator(StreamSpawnPoint,
12.         GeometryVisionSettings.TargetChunkSize);
13.     entityManager.AddSharedComponentData(primeEntity, new TargetChunkSharedValues()
14.     {
15.         ChunkIndex = chunkIndex,
16.         ChunkAssignedAndLocked = GeometryDataModels.Boolean.False
17.     });
18.     var position = GeometryVisionEntityUtilities.ChunkIndexToPositionCalculator(chunkIndex, GeometryVisionSettings.TargetChunkSize);
19.  
20.     entityManager.AddChunkComponentData<TargetChunk>(primeEntity);
21.     var chunk = entityManager.GetChunkComponentData<TargetChunk>(entityManager.GetChunk(primeEntity));
22.     chunk.ChunkIndex = chunkIndex;
23.     chunk.ChunkPosition = position;
24.     chunk.BoundsMin = GeometryVisionEntityUtilities.ChunkAABBGeneratorMin(position, GeometryVisionSettings.TargetChunkSize);
25.     chunk.BoundsMax = GeometryVisionEntityUtilities.ChunkAABBGeneratorMax(position, GeometryVisionSettings.TargetChunkSize);
26.     chunk.ChunkAssignedAndLocked = GeometryDataModels.Boolean.False;
27.     entityManager.SetChunkComponentData(entityManager.GetChunk(primeEntity), chunk);
28.     entityManager.AddComponent(primeEntity,  typeof(MarkedForDestruction));
29.     entityManager.AddComponent(primeEntity,  typeof(Target));
30.     entityManager.SetComponentData( primeEntity, new MarkedForDestruction {AmountOfLifeTimeLeft = LifeTime});
31.     entityManager.AddComponent(primeEntity, typeof(StreamMovement_MoveStreamComponent));
32.  
33.     var spawnerData = new SpawnStreamData
34.     {
35.         ChunkIndex = chunkIndex,
36.         EntityFromPrefab = primeEntity,
37.         StreamSpawnPoint = this.StreamSpawnPoint,
38.         CountX = this.CountX,
39.         CountY = this.CountY,
40.         separationMultiplier = this.separationMultiplier,
41.         StreamInterval = this.StreamInterval,
42.         Direction = this.SpawnerOrietation,
43.         Speed = this.Speed,
44.         MoveDirection = this.MoveDirection,
45.         LifeTime = this.LifeTime
46.     };
47.  
48.     entityManager.AddComponentData(entity, spawnerData);
49.  
50.     var timer = new Timer
51.     {
52.     };
53.  
54.     entityManager.AddComponentData(entity, timer);
55. }

I think the issue must be that changing a one chunk value changes others, but somehow if I do it with the scd ont the OnUpdate the chunks go correctly, but cause major performance reduction

 

So what I want is chunk component that defines a chunk type. Then split up those entities in to a different chunks, but still having the same chunk type.

For example chunk type house. You have one chunk type but multiple houses. These can be identified by checking them against cameras frustum

 

In my case one system can handle 400k entities, when I start to scale up the systems performance drops.
So if someone wants to use this plugin for example tower defense like games that require hundred or even thousands of enemies and towers this needs to work

 

I have no idea what you mean by "chunk type" here. That could mean one of several things. If you are just trying to optimize the original code you posted, then you need to make that job an IJobEntityBatch and defer the SetSharedComponent to an IJob reading from a NativeList like I mentioned earlier. ISCD by itself can be used in a Burst job if its fields meet the Burst criteria. It is the API that works with ISCD that is not Burst-compatible.

 

Well thanks for the help.

Code (CSharp):

1.     public struct TargetChunk : IComponentData
2.     {
3.         public int3 ChunkIndex;
4.         public float3 ChunkPosition;
5.         public GeometryDataModels.Boolean ChunkAssignedAndLocked;
6.         public float3 BoundsMin;
7.         public float3 BoundsMax;
8.     }

So this is what I mean by chunk type.

So every entity that has this component I want to divide in to different groups.

 

Ok I found out that for me the issue was that I was using the shared component data to divide them to chunks. Its possible to just use chunk components.

rookie mistake obviously.