Creating & Destroying 1000's of entities during one frame

My goal is to generate light entities with several ( 3 to 6+ ) ComponentDatas depending on the requirements, for ten to hundred of agents per frame.

Then I have multiple systems that:
- init a NativeArray based on one main ComponentData. Store it somewhere.
- transform the data based on ComponentDatas.
- output few simple result values which I store in the agents that created it.
- deallocate all the NativeArrays
- destroy the entity (because I no longer need any of that data).

I know that I could write it in ECS with no issues, but I'm afraid that I'm over-engineering things and short-lived entities would cause performance issues.

So my question is:
Is creating/destroying entities during one frame the way to go?
Or should I create a specialized system that handles this ( without using entities at all ) ?

 

It's generally not advisable to do structural changes (sync points) every frame. The only exception is adding/removing Tag/Empty components. Even in that case, it's better to not do it every frame. I use them, for example, for input handling (so can be very frequent, but not every frame).

 

This is false on several levels. While doing many sync points per frame (like once after every system) can bring down your efficiency, once per frame is pretty good, and I think a lot of games will have 2 - 4 sync points per frame as the norm.

The main thing is to not use EntityCommandBuffers when creating and destroying a large number of entities. Instead you want to use the EntityManager to instantiate a large number of entities on the main thread in a single step and also collect all the entities you want to destroy into a NativeArray to destroy them all at once with the EntityManager.

You can hide the cost of a sync point by scheduling jobs right before the sync point which are not dependent on the ECS world. If you know of certain chunks that won't be affected by the added and deleted entities and they have some heavy work that needs to be done, you can use ToComponentDataArray to get the data out of ECS and then schedule your jobs. Then you can sync and do your adds and deletes and then write the results back to the ECS world. So it all looks something like this:

1. ToComponentDataArray unrelated entities
2. Schedule heavy jobs on arrays and store job handle somewhere
3. Use EntityManager to delete entities
4. Use EntityManager to batch remove some NewEntity component on all the entities created last frame
5. Instantiate new entities with NewEntity component
6. Schedule a job to write the results from (2) with the job handle as an input dependency
7. Schedule a job to process the NewEntity entities with whatever variations are required.

Of course if you have other types of jobs outside of ECS, you could also just schedule those too before a sync point and skip the back and forth dance of To/FromComponentDataArray.

There's also the ability to generate entities in a different world from a job using ExclusiveEntityTransaction and then merge that into the main world with a short sync point. Same rules apply but you get a little more control over the generation of each entity rather than instantiating a prefab with a count of 1000.

 

If you need to create/destroy thousands of entities each frame, it may be reasonable to use a pull, instead of removing an entity you can disable it and put a reference to NativeQueue, and instead of creating grab one of the disabled entities from that queue reconfigure it and enable. This way you save lots of main tread time by avoiding structural changes.

 

Yeah batch apis & ExclusiveEntityTransaction should be the cookie cutter response. But in practice I find a lot of things hard to batch and inter-worlds transactions can over-complicate things (especially when the OP said he doesn't want to over-engineer). So instead I just wanted to emphasize the overhead of structural changes. And structural changes by themselves (without batching) are expensive operations.

 

Then why are the unity examples telling us to use the CommandBuffer?

I just want to make sure i understand you correctly. Currently iam doing this:

1. Use EntityManager.CreateEntity() & EntityManager.SetComponentData() to create flags

Code (CSharp):

1. foreach (var obj in list)
2.         {
3.             var entity = _manager.CreateEntity(flagArchetyp);
4.             _manager.SetComponentData(entity, obj);
5.         }

2. Have a job to iterate over each flag and create objects depending on the flag via EntityCommandBuffer

I originally wanted to create a new thread asking how i can move step 1. into a job as i thought it would be the bottleneck, but as it looks like the bottleneck is happening on step 2?

Sorry for highjacking this thread

 

The ECS devs have confirmed multiple times that they will improve EntityCommandBuffer's performance. So I'm guessing it'll be at least on par with the batch api. I haven't tested myself but p33 saw some improvements:

 

quote from documentation

in a job you can't do any structural changes, CommandBuffer exists to schedule that changes for the main thread.
All structural changes (including Entity construction) always happen on the main thread.

so the answer is: you can't move EntityManager.CreateEntity() or EntityManager.AddComponentData() to the job, but you can schedule it from a job using CommandBuffer (what do not give you any gain in performance) jobs are good for doing in parallel massive non-structural changes.

 

I understand why we need to use them, i just wonder why they give us an example with the CommandBuffer if its not the best way to do it.

After seeing the examples i would have guess that this is the best way, assuming i have already a List<T> with all the information on the main thread:

1. Gather all "spawn information" in a NativeArray
2. Let job create the stuff via CommandBuffer
3. Done

But it looks like the best way to do it is this:

Let the EntityManager create all the objects & their nesting (childs etc) from the list

eg this:

Code (CSharp):

1. foreach (var obj in list) {      
2.       var parent = EntityManager.CreateEntity(flagArchetyp);
3.       var child1 = EntityManager.Instantiate(Cube);      
4.       var child2 = EntityManager.Instantiate(Cube);      
5.       EntityManager.SetComponentData(parent , obj);
6.       EntityManager.SetComponent(child1 , new Parent() = parent)
7.       EntityManager.SetComponent(child2 , new Parent() = parent)
8.  
9.     etc
10.  
11. }

isnt this a little bit disappointing that we have all this multithreading going on and the whole job system, if the best solution is still just creating everything on the mainthread?

 

it is not disappointing at all it is impossible (you can't change data while iterating it)
the same way you can fill disappointed because people can't teleport in real life

yes main thread is the right place for constructing entities, but some times you need to be able to schedule a structural change form a job
for example: if you have a game with a machine gun and 1000000 of bullets on the screen, it is reasonable to process those massive number of bullets using a job, but when bullet hit's something you need to destroy it, but this something you can't do in a job, and this is the right place to use CommandBuffer

 

Concurrency means you have to deal with synchronization.

With say .Net concurrent collections, they synchronize on pretty much every call, you pay the cost at all times even when you don't need to.

The job system and ECS create flows that don't need to synchronize for most work. You just don't create flows that write to the same data concurrently (for the common case). And you can setup concurrent reading without paying the synchronization cost concurrent collections impose.

It's a far more performant approach, but it requires you to have to reason about your flows and what and where you synchronize.

 

This is a good example, so let me ask again:

What is the best approach if i have the directions of 10000000 Bullets in a List<float3> on the mainthread:

1. Just take the list and create everything from EntityManager.Instantiate etc
2. Take the list into a job and use CommandBuffer to create them

What is the best approach if i want to remove them if they are out of sight/hit something?

Assume that i use a job and iterate over each bullet and check if they should be removed or not.

1. In the same job use CommandBuffer.DestroyEntity to each destroy the entity
2. In the job mark the entity with another component "DestroyThis" and iterate on mainthread over it and destroy it via EntityManager
3. In the job mark the entity with another component "DestroyThis" and iterate on a job over it and destroy it via CommandBuffer

 

IMHO

What is the best approach if i have the directions of 10000000 Bullets in a List<float3> on the mainthread:
1) instantiate bullet and set initial properties right on the main thread
What is the best approach if i want to remove them if they are out of sight/hit something?
1) in a job where you check new position against scene/targets bounds use CommandBuffer.DestroyEntity to schedule bullet destruction

 

Can you talk about your reasoning here?

 

all other variants are overengineering
in any case with CommandBuffer or without it those operations going to happen on the main thread

Qeustion1 variant2. Take the list into a job and use CommandBuffer to create them
job does unnecessary work in this case, it creates a command buffer of commands to be executed later on the main thread instead of executing command right away, you pay job scheduling cost and command buffer cost for nothing
(if you need to postpone structural changes till the end of frame you can use CommandBuffer right on the main thread)

Qeustion2 variant2and3. mark the entity with another component "DestroyThis" is an unnecessary extra step you can skip

 

Thank you for your answer

 

Also if you already have to tag the object as required by your game logic, (so it is not `DestroyThis` but rather something like `Hitted`, so you apply damages before destroy, etc.) Then you can piggyback the fact that `Hitted` separated chunks for you and use EntityQuery overload of EntityManager/EntityCommandBuffer on the main thread. That way you iterate equal to no. of chunks with `Hitted` and not each entity, as would be the case if destroy commands was enqueued from the job individually.

There is also a potential of batch-setting initial value with `EntityQuery.CopyFromComponentDataArray`. So you have to maintain an array of data sized exactly as all bullet entities linerized (maybe easy if you just created all the bullets with no prior bullets) and also for correct ordering where each data lands on which entity you may have to do math with chunk offset.. I touched on them a bit here but it may not worth the trouble to setup. (But if you really have 10000000 bullets maybe it could be significant)

 

I saw this, but iam confused here: where do i get the NativeArray<Entity> from? It looks like i need to create the entities, and put them inside a nativearray to use the batch calling, but then i already created the entites anyway, so why do the batchcalling?

 

thank you

 

I can't use instantiate because I do not know in advance how many agents are gonna need to calculate/transform/output, nor do I know those values until an agent requests them.

So all I can think of is this ( if I need to use Instantiate ) :
- agent requests calculation by adding the requirements to a queue/list inside a system ( guess it would be like a light ECB )
- that system consumes the queue and uses instantiate to create all the temp entities
- apply calculations by various systems
- delete all those temporary entities