Question Best way to organize identical groups of Entities with same systems running in parallel

Hello everyone! I am looking to use DOTS for a machine learning project and trying to figure out a good ECS-centric design for having multiple identical small simulations running simultaneously, something similar to what is shown in Unity MLAgents docs (image attached). Each simulation has an identical group of entities and uses the same overall logic/systems, but entities from different simulations should not affect each other. A controller/server sends an input to each simulation, the simulations run, and the server waits for the results.


My first thought is to create one simulation World, then duplicate that same World with adjusted parameters in different positions (so N same Worlds in the same scene). But each World needs to take parameters in and output the simulation results. How would I communicate to/from the Worlds?
Also, how would I indicate the different World positions? I'm thinking loading a subscene but would still need to programmatically create and set the position for subscenes which I do not know if it is possible.

I am planning to have 100+ separate groups/simulations running at once, so having some sort of tag indicating what simulation each entity is in to filter queries for the systems to filter based on would need too many tags in my opinion. It would also require creating multiple separate systems which have a different query for each unique tag/simulation. Would this be a better approach? Or is there a better design?

 

Shared components might be an option.

 

The main issue is querying entities only within a simulation without creating duplicate systems/queries. If the idea is using a shared component as the "tag" to indicate which simulation each entity is in, my understanding is I will have to create 100+ near-identical systems which only vary with what the shared component "tag" they use in their queries. Hmm. Would it be the best idea to have System X1, System X2, System X3,... which all have basically the same logic besides adding a filter for the shared component tag 1/2/3/... in their queries? Wouldn't there be significant computation cost keeping track of 100+ systems running all at once?

 

You could make each system loop through each tag value and set the filter on the queries it uses. So each system would handle all 100 groups, and then the next system would handle the same100 groups with the next logic and so on.

 

have you used shared components before, do you know how they work? https://docs.unity3d.com/Packages/com.unity.entities@1.1/manual/components-shared-introducing.html

In my words: This can be used / is a "filter component" that you can query and that organizes the entities in separate chunks for each different shared component value. So say you have 1000 entities of the same archetype that includes the shared component (and the shared component is a simple int value). If you set for say 500 of those the int to 1 and for the remaining 500 the int to 2, you would split the "entities" (components belonging to the archetype) in two separate chunks (linked chunks up to their respective capacity).... and you can query them by the shared component value.

edit: overlap...with Dreaming's post...

 

I didn't know shared components organize the entities into separate chunks. It seems having some sort of shared component associated with each Entity to indicate the simulation/group it is in would be the best method.

I found Chunk components in the docs. I think this would be a viable "substitute" for the shared component - my simulations/groups will have a small number of entities each (there are just a lot of groups). I am thinking to have a single chunk component attribute associated with each entity indicating which group it is part of. Then, when my systems query entities and need to find other entities in the same chunk as the entity matching the query, I can do a IJobChunk to only search entities in the same group/chunk. Does this logic make sense? Can it be improved?

The only issue I see is that I still have to create a new chunk component attribute for every unique group/simulation, but seems there is no way to avoid that since cannot create duplicate Worlds easily.

 

Chunk components are meant for chunk-level caches. Aside from being a part of the entity archetype, they aren't associated to the entities in any meaningful way. They are instead associated with the chunk. If you want separation of simulations, a shared component is the much better way to go.

 

I see. Shared components it is then! I assume IJobChunk would still be useful to query entities that I know are in the same chunk based on the shared component.

Let's say I have 100 groups/simulations, so 100 tags. Then each system would have to have a separate query for every tag (so 100 queries per system) correct? Is there any way to simplify this?

I imagine having so many queries would be computationally taxing - if only 1 entity in 1 group changed, the system would have still have queried every group to find which triggered the system update (unless I create a separate system per tag, so 100x more systems). It would also be a pain to code, although I imagine I could try having a template Job/Query to simplify the process.

Is it possible to query for a parent/interface attribute, then match it to a child of the parent? For example, say I have a generic

SimulationGroup : IComponentData

. Can I then do

SimulationGroup100 : SimulationGroup

and then query for SimulationGroup? So my system would just query "entity with a

.WithAny<SimulationGroup>()

" and later I can do "if the SimulationGroup component is SimulationGroup100, do logic on SimulationGroup100 entities".

 

Shared components group by value, so you want a shared component that has a byte field, and for each simulation group, you give all entities in that group a shared component with a specific value pertaining to that group. Then all your systems only query for that shared component and inside your loop of all groups, you set a shared component filter on your query.

In practice though, you don't need to do this for systems where each entity minds its own business. You can do all the groups at once. It is only when you have to deal with aggregates that you concern yourself with this.

 

Q1: Do you need ECS or Burst + Jobs only?
Q2: Do you need to split in groups (i.e. for aggregates, etc. or simulation is independent per entity just with a different logic per group)

ECS offers (1) shared components (2) Dictionary like structures (NativeMultiHashMap, Hashmap i.e. combined with DynamicBuffers, ...)

Without ECS, you could possibly just have a big array and run bursted jobs over it. (i.e. flattened 2d with dimension 1 for group, dimension 2 for the "entities")

 

Makes sense! If shared components group by value, then maybe I can assume the same-value shared components use the same chunk (so I can use something like IJobChunk)? I was planning to just create a shared component for every group to force same-group entities to be in the same chunk. I have got a good idea of how to progress now, thank you so much for the help!

A1: I am deliberately trying to use ECS for this project, want to evaluate the paradigm's usefulness for ML simulations.
A2: I need to split in groups. The entities within a group interact with each other, but do not interact cross-groups (intra-group, not inter-group).

 

All entities in a chunk will have the same shared component value for any given shared component type. They won't necessarily be all the entities with that shared component value, but there will only ever be one value per type in the chunk.