Resolved Bunch of questions -UnityPhysics with Burst/Parallel

Hello

Right now I'm working on a small car game in the dots + physics world. But my profiler showed, that my vehicleMechanic job is taking a lot of time ingame.

So far I did it in a pure ECS way, related to this approach of a vehicleMechanic job of the UnityPhysicsSamples RaycastCar, just a bit simplified.

Now I wanted to implement parallel execution & Burst to improve the performance, but then a lot of questions stacked up:

1. Solved: In the RaycastJob, it does a JobHandle.Schedule() for a IJobParallelFor job. Does it mean that it can be executed in parallel? Because in the Entities.ForEach() you have to write ScheduleParallel() instead of Schedule() to run it parallel, right!?
2. Open: If I execute the vehicleMechanic job with Burst, it will fail with
   "Burst error: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field is not supported" for
   
   Code (CSharp):

   1. JobHandle rayJobHandle = ScheduleBatchRayCast(world.CollisionWorld, rayInputs, rayResults);
   2. rayJobHandle.Complete();
   (this specific job is also mentioned here & in the previous linked UnityPhysicsSample job)
   I get the point, that I'm not allowed to get the results via pointer/reference on rayResults with Burst. But which options I have? Since Schedule() does not return something or so.. I have to go the way with a pointer I guess, so how do I fix it? I could create a extra job without Burst, just for the execution.. then save the results in a dynamicBuffer & access it again in the "vehicleMechanic" job.. would this be the right way?
3. Solved: Another thing is parallism. I have multiple cars in the scene, so it would be perfect if I could run the vehicleMechanic job in parallel. But it will already fail in this line:
   
   Code (CSharp):

   1. int ceIdx = world.GetRigidBodyIndex(ce);
   
   "The previously scheduled job VehicleMechanicSystem [...] writes to the NativeArray [...] CollisionWorld.DynamicTree.BranchCount. You are trying to schedule a new job [...] which reads from the same NativeArray. To guarantee safety, you must include Vehicl" (it does not display the full error).
   This ceIdx is from the chassis, which is the physicsEntityBlock of the car, so all calculations of all the wheels are depending on this. Does it mean physics logic in general is not possible with parallism or how can I solve this problem?

I appreciate your help

 

1. Schedule() for IJobParallelFor is the same as ScheduleParallel for other kinds of jobs you're used to. Not intuitive! See here:
https://forum.unity.com/threads/ijo...lelfor-schedule-differences-use-cases.895019/

So the RaycastJob from ScheduleBatchRayCast is already parallel.

2. I'm not sure why you're using a ptr here, I admit. You normally should be able to just iterate over the NativeArray<RaycastHit>. I'll leave others to answer this and possibly help you more D:

3. Putting the whole foreach in parallel has a number of possible traps, because the job may run longer than you expect it will - depending on the dependencies you use, it may run into the next frame. That is what the error you are getting suggests to me.
This part in particular: The previously scheduled job VehicleMechanicSystem [...] - did you rename the system to VehicleMechanicSystem from the VehicleMechanicsSystem in the sample, by any chance? If so, what's happening is probably that your job is still running, then you're scheduling it again next frame, and it therefore has a clash.

I'd recommend switching the VehicleMechanicSystem from ComponentSystem to SystemBase if you didn't already. You can then pass "Dependency" as the dependency for the Foreach, like so:

Code (CSharp):

1.         Dependency = Entities
2.             .WithBurst()
3.             .ForEach((  VehicleMechanics mechanics  ) => {
4. //code
5.             }).ScheduleParallel(Dependency);

[You don't need to declare Dependency, it's from SystemBase itself]

It might solve your problem already, because it makes sure you're scheduling your job after the dependencies of this system, and ensures that following jobs wait for this one.

Again, I'*m just assuming you're still using ComponentSystem because the sample uses it. If you don't and already manage dependencies, then sorry for my suggestion D:

 

Hey Zeffi, thank you so much for your time & effort!

1. Thanks for this one!
   
2. Normally it looks like this:
   
   
   Code (CSharp):

   1. [BurstCompile]
   2.     public struct RaycastJob : IJobParallelFor
   3.     {
   4.         [ReadOnly] public CollisionWorld world;
   5.         [ReadOnly] public NativeArray<RaycastInput> inputs;
   6.         public NativeArray<RaycastHit> results;
   7.  
   8.         public unsafe void Execute(int index)
   9.         {
   10.             RaycastHit hit;
   11.             world.CastRay(inputs[index], out hit);
   12.             results[index] = hit;
   13.         }
   14.     }
   15.  
   16.     public static JobHandle ScheduleBatchRayCast(CollisionWorld world,
   17.         NativeArray<RaycastInput> inputs, NativeArray<RaycastHit> results)
   18.     {
   19.         JobHandle rcj = new RaycastJob
   20.         {
   21.             inputs = inputs,
   22.             results = results,
   23.             world = world
   24.  
   25.         }.Schedule(inputs.Length, 4);
   26.         return rcj;
   27.     }
   You see? It writes the result of world.CastRay to hit, which we save in results, which is a pointer to the outer world. If I try to call this in my vehicleMechanicSystem job with enabled Burst:
   
   
   Code (CSharp):

   1.  
   2.  var rayCommands = new NativeArray<Physics.RaycastInput>(4, Allocator.TempJob);
   3.  var rayResults = new NativeArray<Physics.RaycastHit>(4, Allocator.TempJob);
   4.  [...]
   5.  var handle = ScheduleBatchRayCast(world, rayCommands, rayResults);
   6.  handle.Complete();
   7.  
   It will lead to: "Burst error: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field is not supported".
   So I don't know exactly how I should do it. I cannot schedule it via Burst -so what would be the best workaround for this? Like creating a new Burst-less job and save the results via DynamicBuffer for the upcoming VehicleMechanicSystem job?
   Sure, I could run "world.CastRay()" directly in my VehicleMechanicSystem job, but it would not be parallel anymore
3. First, you are right: VehicleMechanicSystem == VehicleMechanicsSystem
   And yes, I used SystemBase already
   Your idea makes perfectly sense for me, but it did not helped. I get still the same error. I broke it down to:
   
   
   Code (CSharp):

   1. [UpdateAfter(typeof(BuildPhysicsWorld)), UpdateBefore(typeof(StepPhysicsWorld))]
   2. public class VehicleMechanicSystem : SystemBase
   3. {
   4.     BuildPhysicsWorld CreatePhysicsWorldSystem;
   5.  
   6.     protected override void OnCreate() {
   7.         this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
   8.     }
   9.    
   10.     protected override void OnUpdate() {
   11.         this.CreatePhysicsWorldSystem.GetOutputDependency().Complete();
   12.         PhysicsWorld world = this.CreatePhysicsWorldSystem.PhysicsWorld;
   13.  
   14.         this.Entities.ForEach((
   15.             int entityInQueryIndex,
   16.             Entity entity,
   17.             in MechanicComponent mechanics
   18.         ) => {
   19.             Entity ce = mechanics.chassisEntity;
   20.             int ceIdx = world.GetRigidBodyIndex(ce); // Not working with Schedule() or ScheduleParallel()
   21.  
   22.         }).WithoutBurst().ScheduleParallel(this.Dependency);
   23.     }
   24. }
   25.  
   I renamed the class name to "Z" to see more of the error:
   "InvalidOperationException:
   The previously scheduled job Z:<>c__DisplayClass_OnUpdate_LambdaJob0
   writes to the Unity.Collections.NativeArray`1[System.Int32] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.world.CollisionWorld.Broadphase.m_DynamicTree.BranchCount.
   
   You are trying to schedule a new job Broadphase:AllocateDynamicVsStaticNodePairs,
   which reads from the same Unity.Collections.NativeArray`1[System.Int32]
   (via AllocateDynamicVsStaticNodePairs.dynamicBranchCount).
   To guarantee safety, you must include Z:<>c__DisplayClass_OnUpda"
   
   Without the UpdateAfter/UpdateBefore it gives me an additional error:
   "InvalidOperationException:
   The previously scheduled job Z:<>c__DisplayClass_OnUpdate_LambdaJob0
   writes to the Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] <>c__DisplayClass_OnUpdate_LambdaJob0.JobData.world.CollisionWorld.m_Bodies.
   
   You must call JobHandle.Complete() on the job Z:<>c__DisplayClass_OnUpdate_LambdaJob0,
   before you can deallocate the Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] safely."
   
   So since it wants that I "complete" Z (=VehicleMechanicSystem) I assume parallism is not working with dots physics?

 

Jumping in here. For 1., the answer is correct, not action required from me.
Number 2., your Burst job is fine, the error you linked in the second post is related to the fact you are using a static member somewhere I assume. It looks to me like it's not related to this job in particular.
Number 3., make sure you add your job as an input dependency for StepPhysicsWorld system (call AddInputDependency() on it and provide the handle). Also, you don't need to complete BuildPhysicsWorld output dependency, just provide it as an input for your ScheduleParallel call (combined with Dependency).

 

petarmHavok, feel you hugged, it finally worked with Burst & ScheduleParallel! Thank you so much!

I was actual completely unaware of these dependencies & how to work with them, so I was reading a bit about it first.
The only thing left is actual number 2. I called now

world.CastRay()

directly in a loop within VehicleMechanicSystem job, so this works at least.

But doing this does not work neither with Burst nor ScheduleParallel():

Code (CSharp):

1. public class VehicleMechanicSystem : SystemBase
2. {
3.     private BuildPhysicsWorld CreatePhysicsWorldSystem;
4.  
5.     protected override void OnCreate() {
6.         this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
7.     }
8.  
9.     [BurstCompile]
10.     public struct RaycastJob : IJobParallelFor
11.     {
12.         [ReadOnly] public CollisionWorld world;
13.         [ReadOnly] public NativeArray<RaycastInput> inputs;
14.         public NativeArray<RaycastHit> results;
15.  
16.         public void Execute(int index) {
17.             RaycastHit hit;
18.             this.world.CastRay(this.inputs[index], out hit);
19.             this.results[index] = hit;
20.         }
21.     }
22.  
23.     public static JobHandle ScheduleBatchRayCast(
24.         CollisionWorld world,
25.         NativeArray<RaycastInput> inputs,
26.         NativeArray<RaycastHit> results
27.     ) {
28.         return new RaycastJob {
29.             world = world,
30.             inputs = inputs,
31.             results = results
32.         }.Schedule(inputs.Length, 5);
33.     }
34.  
35.     protected override void OnUpdate() {
36.         this.CreatePhysicsWorldSystem.GetOutputDependency().Complete();
37.         PhysicsWorld world = this.CreatePhysicsWorldSystem.PhysicsWorld;
38.  
39.         this.Entities.ForEach((
40.             Entity entity
41.         ) => {
42.             NativeArray<RaycastHit> rayResults = new NativeArray<RaycastHit>(4, Allocator.TempJob);
43.             NativeArray<RaycastInput> rayInputs = new NativeArray<RaycastInput>(4, Allocator.TempJob);
44.  
45.             //  [...]
46.  
47.             JobHandle rayJobHandle = ScheduleBatchRayCast(world.CollisionWorld, rayInputs, rayResults);
48.             rayJobHandle.Complete();
49.  
50.             rayInputs.Dispose();
51.             rayResults.Dispose();
52.         }).WithoutBurst().Run();               // Works fine!
53.         // .WithoutBurst().ScheduleParallel()  // Does not work!
54.         // .Run()                              // Does not work!
55.     }
56. }

One thing is parallism,
calling it without Burst & with ScheduleParallel() it will throw "UnityException: CreateJobReflectionData can only be called from the main thread."

The other thing is Burst,
calling it with Burst & Run() it will throw:
Burst error BC1042: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field `Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<VehicleMechanicSystem.RaycastJob>.jobReflectionData` is not supported

at Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<VehicleMechanicSystem.RaycastJob>.Initialize()
at Unity.Jobs.IJobParallelForExtensions.Schedule(
VehicleMechanicSystem.RaycastJob* jobData,
int arrayLength,
int innerloopBatchCount,
Unity.Jobs.JobHandle dependsOn
)
at VehicleMechanicSystem.ScheduleBatchRayCast(
Unity.Physics.CollisionWorld* world,
Unity.Collections.NativeArray`1<Unity.Physics.RaycastInput>* inputs,
Unity.Collections.NativeArray`1<Unity.Physics.RaycastHit>* results
) (at VehicleMechanicSystem.cs:28)

So how do I call now a job within a parallel-scheduled burst job?
Or which workaround should I use?

 

I am glad you are making progress. We'll get to the bottom of this one too.

Never spawn jobs from jobs, it won't work. Having an Entities.Foreach and then scheduling a job from that won't work for sure. Use Entities.Foreach to collect the data into a NativeArray, and then spawn raycast jobs in parallel that will work on that data. This type of nesting is not supported.

Also, make sure you add similar logic to your Vehicle system as in number 3 (UpdateAfter, UpdateBefore and proper input/output dependencies).

 

Hey petarmHavok, thanks again for your reply!
I followed your advise and it took me a while to transform the VehicleMechanicSystem, but somehow I got it to work.

I have now 3 IJobParallelFor BurstCompiled Jobs, used by the system:

1. The first one collecting the rayInputs NativeArray for the 2nd job.
2. The 2nd one RaycastJob, making the ray casts for each wheel & adding the results to rayResults.
3. The 3rd one is using the rayResults for the main vehicle mechanic part.

For 4 cars in a scene, with each 4 wheels, I would have 4 threads for the 1. job, 16 threads for 2. job and 4 threads for the 3. job.
Since they are depending on each other, I think it makes not much sense to put them each in a own system (& transforming the arrays for the rays to dynamicBuffers).

The hardest part were actual the NativeArrays: rayInputs & rayResults.
I found no way to dissolve them at the right time with the Entity.Foreach() Pattern.
So I read about [DeallocateOnJobCompletion] for properties in a Job, which dissolves the NativeArray when the job is completed, which was working perfect, but I could not use Entity.Foreach() anymore. Maybe there is a workaround existing?

It basically looks now like this (very abstract, not all lines included):

Code (CSharp):

1. [UpdateAfter(typeof(BuildPhysicsWorld)), UpdateBefore(typeof(StepPhysicsWorld))]
2. public class VehicleMechanicSystem : SystemBase
3. {
4.      BuildPhysicsWorld CreatePhysicsWorldSystem;
5.      StepPhysicsWorld stepPhysicsWorld;
6.  
7.      protected override void OnCreate() {
8.          this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
9.          this.stepPhysicsWorld = this.World.GetOrCreateSystem<StepPhysicsWorld>();
10.      }
11.  
12.  
13.     [BurstCompile]
14.     public struct RayParameterJob : IJobParallelFor
15.     {
16.         [NativeDisableParallelForRestriction] // Was needed to fill rayInputs for 16 wheels of the 4 cars -each of the car = 1 thread/index
17.         public NativeArray<RaycastInput> rayInputs;
18.     }
19.  
20.     [BurstCompile]
21.     public struct RaycastJob: IJobParallelFor
22.     {
23.         [ReadOnly][DeallocateOnJobCompletion]
24.         public NativeArray<RaycastInput> rayInputs;
25.  
26.         public NativeArray<RaycastHit> rayResults;
27.     }
28.  
29.     [BurstCompile]
30.     public struct VehicleMechanicJob: IJobParallelFor
31.     {
32.          [ReadOnly][DeallocateOnJobCompletion]
33.          public NativeArray<RaycastHit> rayResults;
34.     }
35.  
36.     protected override void OnUpdate() {
37.         JobHandle buildPhysicsWorldHandle = this.buildPhysicsWorld.GetOutputDependency();
38.         JobHandle jobsBefore = JobHandle.CombineDependencies(buildPhysicsWorldHandle, this.Dependency);
39.  
40.         // All car entities found by query
41.         NativeArray<Entity> carEntities= this.query.ToEntityArray(Allocator.TempJob);
42.  
43.         // rayResults & rayInputs holding data for each wheel of the cars
44.         NativeArray<RaycastHit> rayResults = new NativeArray<RaycastHit>(carEntities.Length * 4, Allocator.TempJob);
45.         NativeArray<RaycastInput> rayInputs = new NativeArray<RaycastInput>(carEntities.Length * 4, Allocator.TempJob);
46.  
47.         // 1. Job
48.         RayParameterJob rayParameterJob = new RayParameterJob {
49.                rayInputs = rayInputs
50.         };
51.         this.Dependency = rayParameterJob.Schedule(carEntities.Length, 32, jobsBefore);
52.  
53.         // 2. Job
54.         RaycastJob raycastJob = new RaycastJob {
55.                rayInputs = rayInputs
56.                rayResults = rayResults,
57.         }
58.         this.Dependency = raycastJob.Schedule(rayInputs.Length, 32, this.Dependency);
59.  
60.         // 3. Job
61.         VehicleMechanicJob vehicleMechanicJob = new VehicleMechanicJob {
62.                rayResults = rayResults,
63.         }
64.         this.Dependency = vehicleMechanicJob.Schedule(carEntities.Length, 32, this.Dependency);
65.  
66.         this.m_EntityCommandBufferSystem.AddJobHandleForProducer(this.Dependency);
67.         this.stepPhysicsWorld.AddInputDependency(this.Dependency);
68.     }
69. }

..But not sure, what this "32" is for, at least its working ^^

 

32 is batch size, basically helps the scheduler divide work into jobs (and threads). If you have a lot of rays 32 is a good choice, but you might want to reduce the number if you have 10 or so raycasts.