Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved Bunch of questions -UnityPhysics with Burst/Parallel

Discussion in 'Physics for ECS' started by Samsle, Jul 22, 2020.

  1. Samsle


    Mar 31, 2020
    Hello :)

    Right now I'm working on a small car game in the dots + physics world. But my profiler showed, that my vehicleMechanic job is taking a lot of time ingame.

    So far I did it in a pure ECS way, related to this approach of a vehicleMechanic job of the UnityPhysicsSamples RaycastCar, just a bit simplified.

    Now I wanted to implement parallel execution & Burst to improve the performance, but then a lot of questions stacked up:
    1. Solved: In the RaycastJob, it does a JobHandle.Schedule() for a IJobParallelFor job. Does it mean that it can be executed in parallel? Because in the Entities.ForEach() you have to write ScheduleParallel() instead of Schedule() to run it parallel, right!?
    2. Open: If I execute the vehicleMechanic job with Burst, it will fail with
      "Burst error: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field is not supported" for
      Code (CSharp):
      1. JobHandle rayJobHandle = ScheduleBatchRayCast(world.CollisionWorld, rayInputs, rayResults);
      2. rayJobHandle.Complete();
      (this specific job is also mentioned here & in the previous linked UnityPhysicsSample job)
      I get the point, that I'm not allowed to get the results via pointer/reference on rayResults with Burst. But which options I have? Since Schedule() does not return something or so.. I have to go the way with a pointer I guess, so how do I fix it? I could create a extra job without Burst, just for the execution.. then save the results in a dynamicBuffer & access it again in the "vehicleMechanic" job.. would this be the right way?
    3. Solved: Another thing is parallism. I have multiple cars in the scene, so it would be perfect if I could run the vehicleMechanic job in parallel. But it will already fail in this line:
      Code (CSharp):
      1. int ceIdx = world.GetRigidBodyIndex(ce);

      "The previously scheduled job VehicleMechanicSystem [...] writes to the NativeArray [...] CollisionWorld.DynamicTree.BranchCount. You are trying to schedule a new job [...] which reads from the same NativeArray. To guarantee safety, you must include Vehicl" (it does not display the full error).
      This ceIdx is from the chassis, which is the physicsEntityBlock of the car, so all calculations of all the wheels are depending on this. Does it mean physics logic in general is not possible with parallism or how can I solve this problem?
    I appreciate your help :)
    Last edited: Jul 23, 2020
  2. Zeffi


    Nov 18, 2017
    1. Schedule() for IJobParallelFor is the same as ScheduleParallel for other kinds of jobs you're used to. Not intuitive! See here:

    So the RaycastJob from ScheduleBatchRayCast is already parallel.

    2. I'm not sure why you're using a ptr here, I admit. You normally should be able to just iterate over the NativeArray<RaycastHit>. I'll leave others to answer this and possibly help you more D:

    3. Putting the whole foreach in parallel has a number of possible traps, because the job may run longer than you expect it will - depending on the dependencies you use, it may run into the next frame. That is what the error you are getting suggests to me.
    This part in particular: The previously scheduled job VehicleMechanicSystem [...] - did you rename the system to VehicleMechanicSystem from the VehicleMechanicsSystem in the sample, by any chance? If so, what's happening is probably that your job is still running, then you're scheduling it again next frame, and it therefore has a clash.

    I'd recommend switching the VehicleMechanicSystem from ComponentSystem to SystemBase if you didn't already. You can then pass "Dependency" as the dependency for the Foreach, like so:

    Code (CSharp):
    1.         Dependency = Entities
    2.             .WithBurst()
    3.             .ForEach((  VehicleMechanics mechanics  ) => {
    4. //code
    5.             }).ScheduleParallel(Dependency);
    [You don't need to declare Dependency, it's from SystemBase itself]

    It might solve your problem already, because it makes sure you're scheduling your job after the dependencies of this system, and ensures that following jobs wait for this one.

    Again, I'*m just assuming you're still using ComponentSystem because the sample uses it. If you don't and already manage dependencies, then sorry for my suggestion D:
    Samsle likes this.
  3. Samsle


    Mar 31, 2020
    Hey Zeffi, thank you so much for your time & effort!

    1. Thanks for this one! :)
    2. Normally it looks like this:

      Code (CSharp):
      1. [BurstCompile]
      2.     public struct RaycastJob : IJobParallelFor
      3.     {
      4.         [ReadOnly] public CollisionWorld world;
      5.         [ReadOnly] public NativeArray<RaycastInput> inputs;
      6.         public NativeArray<RaycastHit> results;
      8.         public unsafe void Execute(int index)
      9.         {
      10.             RaycastHit hit;
      11.             world.CastRay(inputs[index], out hit);
      12.             results[index] = hit;
      13.         }
      14.     }
      16.     public static JobHandle ScheduleBatchRayCast(CollisionWorld world,
      17.         NativeArray<RaycastInput> inputs, NativeArray<RaycastHit> results)
      18.     {
      19.         JobHandle rcj = new RaycastJob
      20.         {
      21.             inputs = inputs,
      22.             results = results,
      23.             world = world
      25.         }.Schedule(inputs.Length, 4);
      26.         return rcj;
      27.     }
      You see? It writes the result of world.CastRay to hit, which we save in results, which is a pointer to the outer world. If I try to call this in my vehicleMechanicSystem job with enabled Burst:

      Code (CSharp):
      2.  var rayCommands = new NativeArray<Physics.RaycastInput>(4, Allocator.TempJob);
      3.  var rayResults = new NativeArray<Physics.RaycastHit>(4, Allocator.TempJob);
      4.  [...]
      5.  var handle = ScheduleBatchRayCast(world, rayCommands, rayResults);
      6.  handle.Complete();
      It will lead to: "Burst error: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field is not supported".
      So I don't know exactly how I should do it. I cannot schedule it via Burst -so what would be the best workaround for this? Like creating a new Burst-less job and save the results via DynamicBuffer for the upcoming VehicleMechanicSystem job?
      Sure, I could run "world.CastRay()" directly in my VehicleMechanicSystem job, but it would not be parallel anymore :(
    3. First, you are right: VehicleMechanicSystem == VehicleMechanicsSystem :oops::)
      And yes, I used SystemBase already :)
      Your idea makes perfectly sense for me, but it did not helped. I get still the same error. I broke it down to:

      Code (CSharp):
      1. [UpdateAfter(typeof(BuildPhysicsWorld)), UpdateBefore(typeof(StepPhysicsWorld))]
      2. public class VehicleMechanicSystem : SystemBase
      3. {
      4.     BuildPhysicsWorld CreatePhysicsWorldSystem;
      6.     protected override void OnCreate() {
      7.         this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
      8.     }
      10.     protected override void OnUpdate() {
      11.         this.CreatePhysicsWorldSystem.GetOutputDependency().Complete();
      12.         PhysicsWorld world = this.CreatePhysicsWorldSystem.PhysicsWorld;
      14.         this.Entities.ForEach((
      15.             int entityInQueryIndex,
      16.             Entity entity,
      17.             in MechanicComponent mechanics
      18.         ) => {
      19.             Entity ce = mechanics.chassisEntity;
      20.             int ceIdx = world.GetRigidBodyIndex(ce); // Not working with Schedule() or ScheduleParallel()
      22.         }).WithoutBurst().ScheduleParallel(this.Dependency);
      23.     }
      24. }
      I renamed the class name to "Z" to see more of the error:
      The previously scheduled job
      writes to the Unity.Collections.NativeArray`1[System.Int32] <>

      You are trying to schedule a new job Broadphase:AllocateDynamicVsStaticNodePairs,
      which reads from the same Unity.Collections.NativeArray`1[System.Int32]
      (via AllocateDynamicVsStaticNodePairs.dynamicBranchCount).
      To guarantee safety, you must include Z:<>c__DisplayClass_OnUpda

      Without the UpdateAfter/UpdateBefore it gives me an additional error:
      The previously scheduled job
      writes to the Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] <>

      You must call JobHandle.Complete() on the job Z:<>c__DisplayClass_OnUpdate_LambdaJob0,
      before you can deallocate the Unity.Collections.NativeArray`1[Unity.Physics.RigidBody] safely.

      So since it wants that I "complete" Z (=VehicleMechanicSystem) I assume parallism is not working with dots physics?
    Last edited: Jul 23, 2020
  4. petarmHavok


    Nov 20, 2018
    Jumping in here. For 1., the answer is correct, not action required from me. :)
    Number 2., your Burst job is fine, the error you linked in the second post is related to the fact you are using a static member somewhere I assume. It looks to me like it's not related to this job in particular.
    Number 3., make sure you add your job as an input dependency for StepPhysicsWorld system (call AddInputDependency() on it and provide the handle). Also, you don't need to complete BuildPhysicsWorld output dependency, just provide it as an input for your ScheduleParallel call (combined with Dependency).
    Samsle likes this.
  5. Samsle


    Mar 31, 2020
    petarmHavok, feel you hugged, it finally worked with Burst & ScheduleParallel! :):) Thank you so much!

    I was actual completely unaware of these dependencies & how to work with them, so I was reading a bit about it first.
    The only thing left is actual number 2. I called now
    directly in a loop within VehicleMechanicSystem job, so this works at least.

    But doing this does not work neither with Burst nor ScheduleParallel():
    Code (CSharp):
    1. public class VehicleMechanicSystem : SystemBase
    2. {
    3.     private BuildPhysicsWorld CreatePhysicsWorldSystem;
    5.     protected override void OnCreate() {
    6.         this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
    7.     }
    9.     [BurstCompile]
    10.     public struct RaycastJob : IJobParallelFor
    11.     {
    12.         [ReadOnly] public CollisionWorld world;
    13.         [ReadOnly] public NativeArray<RaycastInput> inputs;
    14.         public NativeArray<RaycastHit> results;
    16.         public void Execute(int index) {
    17.             RaycastHit hit;
    18.   [index], out hit);
    19.             this.results[index] = hit;
    20.         }
    21.     }
    23.     public static JobHandle ScheduleBatchRayCast(
    24.         CollisionWorld world,
    25.         NativeArray<RaycastInput> inputs,
    26.         NativeArray<RaycastHit> results
    27.     ) {
    28.         return new RaycastJob {
    29.             world = world,
    30.             inputs = inputs,
    31.             results = results
    32.         }.Schedule(inputs.Length, 5);
    33.     }
    35.     protected override void OnUpdate() {
    36.         this.CreatePhysicsWorldSystem.GetOutputDependency().Complete();
    37.         PhysicsWorld world = this.CreatePhysicsWorldSystem.PhysicsWorld;
    39.         this.Entities.ForEach((
    40.             Entity entity
    41.         ) => {
    42.             NativeArray<RaycastHit> rayResults = new NativeArray<RaycastHit>(4, Allocator.TempJob);
    43.             NativeArray<RaycastInput> rayInputs = new NativeArray<RaycastInput>(4, Allocator.TempJob);
    45.             //  [...]
    47.             JobHandle rayJobHandle = ScheduleBatchRayCast(world.CollisionWorld, rayInputs, rayResults);
    48.             rayJobHandle.Complete();
    50.             rayInputs.Dispose();
    51.             rayResults.Dispose();
    52.         }).WithoutBurst().Run();               // Works fine!
    53.         // .WithoutBurst().ScheduleParallel()  // Does not work!
    54.         // .Run()                              // Does not work!
    55.     }
    56. }
    One thing is parallism,
    calling it without Burst & with ScheduleParallel() it will throw "UnityException: CreateJobReflectionData can only be called from the main thread."

    The other thing is Burst,
    calling it with Burst & Run() it will throw:
    Burst error BC1042: The managed class type `System.IntPtr` is not supported. Loading from a non-readonly static field `Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<VehicleMechanicSystem.RaycastJob>.jobReflectionData` is not supported

    at Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<VehicleMechanicSystem.RaycastJob>.Initialize()

    at Unity.Jobs.IJobParallelForExtensions.Schedule(
    VehicleMechanicSystem.RaycastJob* jobData,
    int arrayLength,
    int innerloopBatchCount,
    Unity.Jobs.JobHandle dependsOn
    at VehicleMechanicSystem.ScheduleBatchRayCast(

    Unity.Physics.CollisionWorld* world,
    Unity.Collections.NativeArray`1<Unity.Physics.RaycastInput>* inputs,
    Unity.Collections.NativeArray`1<Unity.Physics.RaycastHit>* results
    ) (at VehicleMechanicSystem.cs:28)

    So how do I call now a job within a parallel-scheduled burst job? o_O
    Or which workaround should I use?
    Last edited: Jul 23, 2020
  6. petarmHavok


    Nov 20, 2018
    I am glad you are making progress. We'll get to the bottom of this one too.

    Never spawn jobs from jobs, it won't work. Having an Entities.Foreach and then scheduling a job from that won't work for sure. Use Entities.Foreach to collect the data into a NativeArray, and then spawn raycast jobs in parallel that will work on that data. This type of nesting is not supported.

    Also, make sure you add similar logic to your Vehicle system as in number 3 (UpdateAfter, UpdateBefore and proper input/output dependencies).
    Samsle likes this.
  7. Samsle


    Mar 31, 2020
    Hey petarmHavok, thanks again for your reply!
    I followed your advise and it took me a while to transform the VehicleMechanicSystem, but somehow I got it to work.

    I have now 3 IJobParallelFor BurstCompiled Jobs, used by the system:
    1. The first one collecting the rayInputs NativeArray for the 2nd job.
    2. The 2nd one RaycastJob, making the ray casts for each wheel & adding the results to rayResults.
    3. The 3rd one is using the rayResults for the main vehicle mechanic part.
    For 4 cars in a scene, with each 4 wheels, I would have 4 threads for the 1. job, 16 threads for 2. job and 4 threads for the 3. job.
    Since they are depending on each other, I think it makes not much sense to put them each in a own system (& transforming the arrays for the rays to dynamicBuffers).

    The hardest part were actual the NativeArrays: rayInputs & rayResults.
    I found no way to dissolve them at the right time with the Entity.Foreach() Pattern.
    So I read about [DeallocateOnJobCompletion] for properties in a Job, which dissolves the NativeArray when the job is completed, which was working perfect, but I could not use Entity.Foreach() anymore. Maybe there is a workaround existing?

    It basically looks now like this (very abstract, not all lines included):
    Code (CSharp):
    1. [UpdateAfter(typeof(BuildPhysicsWorld)), UpdateBefore(typeof(StepPhysicsWorld))]
    2. public class VehicleMechanicSystem : SystemBase
    3. {
    4.      BuildPhysicsWorld CreatePhysicsWorldSystem;
    5.      StepPhysicsWorld stepPhysicsWorld;
    7.      protected override void OnCreate() {
    8.          this.CreatePhysicsWorldSystem = this.World.GetOrCreateSystem<BuildPhysicsWorld>();
    9.          this.stepPhysicsWorld = this.World.GetOrCreateSystem<StepPhysicsWorld>();
    10.      }
    13.     [BurstCompile]
    14.     public struct RayParameterJob : IJobParallelFor
    15.     {
    16.         [NativeDisableParallelForRestriction] // Was needed to fill rayInputs for 16 wheels of the 4 cars -each of the car = 1 thread/index
    17.         public NativeArray<RaycastInput> rayInputs;
    18.     }
    20.     [BurstCompile]
    21.     public struct RaycastJob: IJobParallelFor
    22.     {
    23.         [ReadOnly][DeallocateOnJobCompletion]
    24.         public NativeArray<RaycastInput> rayInputs;
    26.         public NativeArray<RaycastHit> rayResults;
    27.     }
    29.     [BurstCompile]
    30.     public struct VehicleMechanicJob: IJobParallelFor
    31.     {
    32.          [ReadOnly][DeallocateOnJobCompletion]
    33.          public NativeArray<RaycastHit> rayResults;
    34.     }
    36.     protected override void OnUpdate() {
    37.         JobHandle buildPhysicsWorldHandle = this.buildPhysicsWorld.GetOutputDependency();
    38.         JobHandle jobsBefore = JobHandle.CombineDependencies(buildPhysicsWorldHandle, this.Dependency);
    40.         // All car entities found by query
    41.         NativeArray<Entity> carEntities= this.query.ToEntityArray(Allocator.TempJob);
    43.         // rayResults & rayInputs holding data for each wheel of the cars
    44.         NativeArray<RaycastHit> rayResults = new NativeArray<RaycastHit>(carEntities.Length * 4, Allocator.TempJob);
    45.         NativeArray<RaycastInput> rayInputs = new NativeArray<RaycastInput>(carEntities.Length * 4, Allocator.TempJob);
    47.         // 1. Job
    48.         RayParameterJob rayParameterJob = new RayParameterJob {
    49.                rayInputs = rayInputs
    50.         };
    51.         this.Dependency = rayParameterJob.Schedule(carEntities.Length, 32, jobsBefore);
    53.         // 2. Job
    54.         RaycastJob raycastJob = new RaycastJob {
    55.                rayInputs = rayInputs
    56.                rayResults = rayResults,
    57.         }
    58.         this.Dependency = raycastJob.Schedule(rayInputs.Length, 32, this.Dependency);
    60.         // 3. Job
    61.         VehicleMechanicJob vehicleMechanicJob = new VehicleMechanicJob {
    62.                rayResults = rayResults,
    63.         }
    64.         this.Dependency = vehicleMechanicJob.Schedule(carEntities.Length, 32, this.Dependency);
    66.         this.m_EntityCommandBufferSystem.AddJobHandleForProducer(this.Dependency);
    67.         this.stepPhysicsWorld.AddInputDependency(this.Dependency);
    68.     }
    69. }
    ..But not sure, what this "32" is for, at least its working ^^
    Last edited: Jul 31, 2020
  8. petarmHavok


    Nov 20, 2018
    32 is batch size, basically helps the scheduler divide work into jobs (and threads). If you have a lot of rays 32 is a good choice, but you might want to reduce the number if you have 10 or so raycasts.
    Samsle likes this.