Entities.ForEach in another Entities.ForEach

Hello Unity Community,

I have recently released a game for "Brackeys Game Jam 2020.1" on itch.io with the title "A Hole In Space" and used Unity's DOTS feature for the first time. I watched and readed multiple tutorials about this topic, but due to the development progression, some tutorials are already deprecated and I don't know how to use it correctly with the latest DOTS version.

Anyways, with DOTS I was able to create a whole galaxy with up to 1 million stars. Even though, my DOTS code is not as performant as it could be. The main reason for this is, I couldn't use an Entities.ForEach loop in another Entities.ForEach loop and couldn't use the .Schedule() at the end of it.

Nevertheless, this is necessary to apply the gravity force of each entity (star/black hole) to another entity.

So I tried with, run one Entities.ForEach loop and get all entites in one NativeList, which I could loop with a normal ForEach in the second Entities.ForEach loop through. But Burst only wanted ReadOnly Lists.
So, the only way to make the NativeList ReadOnly is to define it as a global variable with that attribute, but Burst don't like global variables.

Last but not least, I ended up defining the black holes by hand and loop only through the suns, which were affected by the black holes. This makes the code not easily expandable and clunky.


My questions are the following:

1. How can I loop through all entities in an Entities.ForEach loop to apply interaction calculations?
2. How am I supposed to use the NativeList correctly with Burst Compiler?
3. How can I increase a variable in a .Schedule() Entities.ForEach loop instead of a .Run()?

With these information I could probably optimize my code further.

Thank you very much for reading this big post.

 

I also want to know

 

Hello

As far as i know there can be no nested ForEach for the time being, ComponentSystem classes can do that but they offer no parallelism.

You can however use EntityQuery to get the entities you are interested and iterate them in in your ForEach, so something like:

Code (CSharp):

1.  
2.     protected override JobHandle OnUpdate(JobHandle inputDeps){
3.         var entityArray = GetEntityQuery(typeof(StartTag)).ToEntityArray(Allocator.TempJob, out JobHandle firstJobHandle);
4.      
5.         inputDeps = Entities.ForEach(()=>{
6.            //Your logic
7.         }).WithDeallocateOnJobCompletion(entityArray).Schedule(firstJobHandle);
8.  
9.         return inputDeps;
10.    
11.     }
12.  

You can use GetComponentDataFromEntity<T> and GetBufferFromEntity<T> in your OnUpdate to acess your entities' data, if you only want to read it component data you need to pass true to those functions and set them ReadOnly in your ForEach lambda:

Code (CSharp):

1.  
2.     protected override JobHandle OnUpdate(JobHandle inputDeps){
3.         var entityArray = GetEntityQuery(typeof(StartTag)).ToEntityArray(Allocator.TempJob, out JobHandle firstJobHandle);
4.         var translations = GetComponentDataFromEntity<Translation>(true);
5.         inputDeps = Entities.ForEach(()=>{
6.            //Your logic
7.         })
8.         .WithDeallocateOnJobCompletion(entityArray)
9.         .WithReadOnly(translations)
10.         .Schedule(firstJobHandle);
11.  
12.         return inputDeps;
13.    
14.     }
15.  

If you want to manipulate the data returned from GetComponentDataFromEntity and Buffer, you need to pass false, which is by default, to those functions and set them Parallelly accessiable:

Code (CSharp):

1.  
2.     protected override JobHandle OnUpdate(JobHandle inputDeps){
3.         var entityArray = GetEntityQuery(typeof(StartTag)).ToEntityArray(Allocator.TempJob, out JobHandle firstJobHandle);
4.         var translations = GetComponentDataFromEntity<Translation>();
5.         inputDeps = Entities.ForEach(()=>{
6.            //Your logic
7.         })
8.         .WithDeallocateOnJobCompletion(entityArray)
9.         .WithNativeDisableParallelForRestriction(translations )
10.         .Schedule(firstJobHandle);
11.  
12.         return inputDeps;
13.    
14.     }
15.  

Please note that, you cannot use the component type in your foreach that you requested with GetComponentDataFromEntity or buffer, so you cannot have both GetComponentDataFromEntity<Translation> and Entities.ForEach((ref Translation translation)=>{});

As for your 2nd question, you simply need to set them read only just like in the example above with "WithReadOnly(variableName)"

As for your 3rd question, you can increase a variable if you put it into a Native container, please note that you need to catch it locally if your native container is a class field, something like:

Code (CSharp):

1.  
2. NativeArray<int> myVariable;
3. protected override OnCreate(){
4.     myVariable = new NativeArray(1, Allocator.Persistent);
5. }
6.  
7. protected override OnDestroy(){
8.     myVariable.Dispose();
9. }
10.  
11.     protected override JobHandle OnUpdate(JobHandle inputDeps){
12.         var entityArray = GetEntityQuery(typeof(StartTag)).ToEntityArray(Allocator.TempJob, out JobHandle firstJobHandle);
13.         var localMyVariable = myVariable;
14.         inputDeps = Entities.ForEach(()=>{
15.            //Your logic
16.            localMyVariable [0] = localMyVariable [0] + 1;
17.         }).WithDeallocateOnJobCompletion(entityArray).Schedule(firstJobHandle);
18.  
19.         return inputDeps;
20.    
21.     }
22.  

Hope it helps^^

 

Thank you very much Curlyone for the very detailed answer!

I will try this out and report whether it's working or not.

 

Yes it works! This reduced my code about 50 lines and makes it expandable for new content. Thank you very much!

Code (CSharp):

1. [BurstCompile]
2.     protected override JobHandle OnUpdate(JobHandle inputDeps)
3.     {
4.         //get all needed parameters
5.         float deltaTime = Time.DeltaTime;
6.         float gravityConstant = UniverseController.instance.universeData.gravityConstant;
7.         float gravityEffectiveRadius = UniverseController.instance.universeData.gravityEffectiveRadius;
8.  
9.         //setup the black holes
10.         NativeArray<Translation> blackHoleTranslations = GetEntityQuery(typeof(BlackHoleData), typeof(Translation)).ToComponentDataArray<Translation>(Allocator.TempJob);
11.         NativeArray<KeplerData> blackHoleKeplers = GetEntityQuery(typeof(BlackHoleData), typeof(KeplerData)).ToComponentDataArray<KeplerData>(Allocator.TempJob);
12.         NativeArray<float> blackHoleMasses = new NativeArray<float>(blackHoleKeplers.Length, Allocator.Persistent);
13.  
14.         //apply force to all entitites from black holes
15.         JobHandle jobHandle = Entities.ForEach((ref Translation translation, ref KeplerData kepler) =>
16.         {
17.             //add up the force by both black holes and get absorbed if to close
18.             float3 force = float3.zero;
19.             float distance = 0;
20.  
21.             for (int i = 0; i < blackHoleTranslations.Length; i++)
22.             {
23.                 //get once the start mass of the black holes
24.                 if (blackHoleMasses[i] == 0)
25.                     blackHoleMasses[i] = blackHoleKeplers[i].mass;
26.  
27.                 //get distance to black hole and add force only, if in gravity range
28.                 distance = math.distance(translation.Value, blackHoleTranslations[i].Value);
29.                 if (distance != 0 && distance <= gravityEffectiveRadius)
30.                 {
31.                     //if object to close to black hole, it's mass will be absorbed
32.                     if (0 < distance && distance <= (blackHoleKeplers[i].diameter + kepler.diameter) / 2.1f)
33.                     {
34.                         //first case, one black hole and one other object; second case, two black holes meet each other, the black hole with the higher mass absorb the smaller one
35.                         if (kepler.type != KeplerType.BlackHole || blackHoleKeplers[i].mass > kepler.mass)
36.                         {
37.                             blackHoleMasses[i] += kepler.mass;
38.                             kepler.mass = 0;
39.                         }
40.                     }
41.  
42.                     //negative, because the gravity force is attractive
43.                     force -= math.normalize(translation.Value - blackHoleTranslations[i].Value) * gravityConstant * (kepler.mass * blackHoleKeplers[i].mass) / math.distance(translation.Value, blackHoleTranslations[i].Value);
44.                 }
45.             }
46.  
47.             //apply the force, but check if velociy is not faster than max
48.             kepler.velocity += force / kepler.mass * deltaTime;
49.             if (math.length(kepler.velocity) > kepler.maxSpeed)
50.                 kepler.velocity = math.normalize(kepler.velocity) * kepler.maxSpeed;
51.  
52.             //move the object
53.             translation.Value += kepler.velocity * deltaTime;
54.         })
55.         .WithDeallocateOnJobCompletion(blackHoleTranslations)
56.         .WithDeallocateOnJobCompletion(blackHoleKeplers)
57.         .Schedule(inputDeps);
58.  
59.         //finish the job
60.         jobHandle.Complete();
61.  
62.         //then apply the masses to the corressponding black hole kepler data
63.         EntityManager entityManager = World.DefaultGameObjectInjectionWorld.EntityManager;
64.         NativeArray<Entity> blackHoles = GetEntityQuery(typeof(BlackHoleData)).ToEntityArray(Allocator.TempJob);
65.         for (int entityIndex = 0; entityIndex < blackHoles.Length; entityIndex++)
66.         {
67.             KeplerData keplerData = entityManager.GetComponentData<KeplerData>(blackHoles[entityIndex]);
68.             keplerData.mass = blackHoleMasses[entityIndex];
69.             entityManager.SetComponentData(blackHoles[entityIndex], keplerData);
70.         }
71.  
72.         //deallocate the native arrays
73.         blackHoles.Dispose();
74.         blackHoleMasses.Dispose();
75.  
76.         return default;
77.     }

 

Would you provide me a Zipped copy of your Project? I did very much the same and had issues with performance.

Link:

Maybe I can Improve my solution a bit

 

I think it should be on unity docs page, very helpful.

 

I think your performance looks reasonable. If you have a look at the Profiler or the ms time in the EntityDebugger, you will see that the most performance is not lost on your entity system, but the rendering itself.

This means the next step would be to implement multithreaded rendering (if it is not used as standard for entities). Unfortunately, I don't know how to do that at the moment. That's something I have to learn. But I think multithreaded rendering is something for another post and not this one.

EDIT: I saw that you use Unity's Physic components e.g. "Physic Shape". I don't use these, because I saw a massive performance lost (~600FPS -> ~220FPS) and micro stuttering in my project with these. So I created my own little physic system with things I really need.

Could you provide a link to the exact doc page please?

 

My point was that these examples are so clear, that would be good to add it to this page
https://docs.unity3d.com/Packages/com.unity.entities@0.6/manual/entities_job_foreach.html

 

Is this actually going to work as expected given inherent concurrency of

Entities.ForEach

? I'd agree if it was a

Job.WithCode

 

I need one last thing to know to improve my performance further with Unity's DOTS feature.



In the scene were 1 million enitites (8 different entity archetypes) rendered at the same time. It seems that the most performance is lost on the CPU for UpdateDynamicRenderBatches. So I have the following questions:

1. Is it possible to reduce the general time this process needed? Perhaps create static batches which does not change dynamically?
2. This process runs single threaded at the moment. Is it possible to make it multithreaded?

 

The hybrid renderer doesn't do so well with a lot of different dynamic entities. There are some steps you can take to try and improve it. There's supposed to be a big update coming soon-ish (according to Unity) which massively improves hybrid renderer performance with dynamic entities.

From the last snippet you posted it doesn't look like you can run the ForEach as a parallel job since you're modifying your entire array in every ForEach iteration, like Nicolas mentioned. But you can run it as single threaded a job, then schedule a second job to write your transformed data back in to your components. A few tips -

• If you derive your system from SystemBase you can run a ForEach on a single thread with .Schedule()
• Don't create entity queries in OnUpdate, cache them in OnCreate.
• You can dispose arrays at the end of OnUpdate via

  array.Dispose(Dependency)

  , they will be disposed after your jobs complete.
• Inside a ForEach you can use the entityInQueryIndex parameter to refer to the matching element in your arrays created from ToComponentDataArray - as long as the ForEach query matches and you haven't made any structural changes since they were created.

 

Thank you for that info!

I already changed it in a way, that I don't need to call jobHandle.Complete() anymore and also don't need to apply the new masses at the end. I saved the entityIndex instead in a ComponentData and in another system, which already existed and goes through all entities anyway, I looked for those entityIndices to apply the masses there. Turns out to be more performant that way.

1. Thanks for the info!
2. But my entites are changing during the game. So if I would cache them in OnCreate, wouldn't I used old data, because it is just a copy and not a reference of these?
3. Thanks again for that!
4. I had some issues on that. If I created a ToComponentDataArray before the Entity.ForEach, somehow the order of these objects changed in the same frame in Entity.ForEach. A simple case: I have 2 Black Holes. If I store data from them in a NativeArray, the diameter for instance, and then apply these diameters in the Entity.ForEach, somehow they swapped their diameters. The small one is now big and the big one is now small. This happens constantly during the game.

I also have problems with multithreaded counting. I am forced to used Run() instead of Schedule() to fix that problem.

Code (CSharp):

1. NativeArray<int> solarCount = new NativeArray<int>(1, Allocator.Temp);
2.         NativeArray<long> solarMasses = new NativeArray<long>(1, Allocator.Temp);
3.  
4.         Entities.ForEach((in SunData sunData, in KeplerData kepler) =>
5.         {
6.             solarCount[0]++;
7.             solarMasses[0] += kepler.mass;
8.         }).Run();

If I use Schedule(), the values are always wrong. I even had two different arrays which counts the same, but they were different at the end.

 

The array you get from ToComponentDataArray is the copy of your data, the query you get from GetEntityQuery() is how you access that copy. You should cache the result of GetEntityQuery. Not a huge deal but by not caching it you're doing work you don't need to do in every update.

According to Jaochim the order of ToComponentDataArray will match a ForEach as long as the queries match and as long as you're not making structural changes between when you copy your data and when you schedule your job. To ensure you're using the right query you can directly retrieve the query generated from a ForEach with WithStoreEntityQueryInField.

 

Unfortunately, it does not work for me so good. If I do this, somehow my simulation behaves very unnatural and some features are not working properly anymore. I have to investigate it more to fully understand what the issue is here.

Another nice info. Thank you very much!
But I can't use the EntityQuery in the same Entity.ForEach, can I? Probably not.

 

Unless you been advised already, you need sheduleSingle, or use IJobChunk job, to calculate masses in parallel. You just pass index to an array, per chunk. Array need be size of chunks count. Then run second job, to iterate through array, to get final result of summed masses.

I know it is been while, but It is unclear for me, why you need job inside job. What you tried to do exactly? Unless you solved that already?

 

Yes, I solved it already. But thanks anyway.

I needed a job inside a job to simulate gravity between each other entity. The first loop goes through all entites as gravity source. The second lop applies this gravity to all other entities.

 

May I ask, what solution did you choose?
Did you actually did multithreaded, or calculated all forces singlethreaded?
I assume, you didn't go with some form of nested job, as you already in a job.

 

First I get an entity array of all data components I needed. Then I put this array in a while loop in a multithreaded job and put it on schedule.

Code (CSharp):

1. protected override JobHandle OnUpdate(JobHandle inputDeps)
2. {
3.     //get all needed parameters
4.     float deltaTime = Time.DeltaTime;
5.     float gravityConstant = Kepler.universeData.gravityConstant;
6.  
7.     //setup all other native arrays
8.     NativeArray<KeplerData> otherKeplers = GetEntityQuery(typeof(KeplerData)).ToComponentDataArray<KeplerData>(Allocator.TempJob);
9.     NativeArray<Translation> otherTranslations = GetEntityQuery(typeof(KeplerData), typeof(Translation)).ToComponentDataArray<Translation>(Allocator.TempJob);
10.  
11.     //apply force to all entitites
12.     JobHandle jobHandle = Entities.ForEach((ref Translation translation, ref KeplerData kepler, in Entity entity) =>
13.     {
14.         //add up the force
15.         kepler.force = float3.zero;
16.         float distance = 0;
17.         int i = 0;
18.  
19.         while (i < otherKeplers.Length)
20.         {
21.             //get distance and add force
22.             distance = math.distance(translation.Value, otherTranslations[i].Value);
23.             if (distance > 0)
24.             {
25.                 //negative, because the gravity force is attractive
26.                 kepler.force -= math.normalize(translation.Value - otherTranslations[i].Value) * gravityConstant * otherKeplers[i].mass / (distance * distance);
27.             }
28.  
29.             //go to next entity
30.             i++;
31.         }
32.  
33.         //apply the force
34.         kepler.velocity += kepler.force * deltaTime;
35.  
36.         //move the object
37.         translation.Value += kepler.velocity * deltaTime;
38.     })
39.     .WithDeallocateOnJobCompletion(otherTranslations)
40.     .WithDeallocateOnJobCompletion(otherKeplers)
41.     .Schedule(inputDeps);
42.  
43.     return jobHandle;
44. }

 

Nice, you did pretty much in similar way I had in mind.
Only small difference, I would perhaps assign

Code (CSharp):

1. //move the object
2.         translation.Value += kepler.velocity * deltaTime;

as separate job.
Otherwise, you may sometimes get position of not yet moved body, and other time already moved within same job.
It may not be important in your case, as it probably has little difference. But for simulation purpose, it may be introducing significant inaccuracy.

 

That's correct. Thanks for pointing this out.

 

This has been extremely helpful thanks all