Resolved RaycastCommand does not hit anything when in job chain (NativeList issue -> dont use it)

Got a following issue - have a chain of three jobs:
1. Sets up RaycastCommands;
2. RaycastCommand.ScheduleBatch, that should process raycast commands;
3. Applies the results;

First one draws ray correctly via debug, third job debug outputs correct indexes, but ScheduleBatch does not seem to run at all (judging by Profiler);

Here's code:

Code (CSharp):

1. protected override void OnUpdate() {
2.          int totalCount = _boidQuery.CalculateEntityCount();
3.  
4.          NativeList<RaycastCommand> commands = new NativeList<RaycastCommand>(totalCount, Allocator.TempJob);
5.          NativeHashMap<Entity, int> indexLookup = new NativeHashMap<Entity, int>(totalCount, Allocator.TempJob);
6.  
7.          // Setup commands first
8.          Entities.WithName("BoidCollisionSystem_SetupCommands")
9.                  .ForEach((Entity entity,
10.                            in Position posData, in BoidData boidData, in BoidCollisionRule collisionRule) => {
11.             indexLookup[entity] = commands.Length;
12.          
13.             float3 dir = boidData.AccumulatedVelocity;
14.             dir = math.normalizesafe(dir);
15.  
16.             RaycastCommand cmd = new RaycastCommand(posData.Value, dir, collisionRule.DetectionDistance);
17.          
18.             Debug.DrawRay(posData.Value, dir * collisionRule.DetectionDistance, Color.magenta);
19.             commands.AddNoResize(cmd);
20.          }).Schedule();
21.      
22.          // Perform raycasts
23.          NativeArray<RaycastHit> results = new NativeArray<RaycastHit>(totalCount, Allocator.TempJob);
24.  
25.          JobHandle raycastHandle = RaycastCommand.ScheduleBatch(commands.AsDeferredJobArray(), results, 1, Dependency);
26.          Dependency = JobHandle.CombineDependencies(Dependency, raycastHandle);
27.  
28.          // Apply results
29.          Entities.WithName("BoidCollisionSystem_ApplyResults")
30.                  .ForEach((Entity entity,
31.                            ref DynamicBuffer<BoidVelocity> buffer,
32.                            in Position posData,
33.                            in BoidCollisionRule collisionRule) => {
34.                     int index = indexLookup[entity];
35.                     RaycastHit hit = results[index];
36.                  
37.                     if (hit.distance <= 0) return; // Consider as no hits
38.                  
39.                     Debug.Log(hit.point);
40.                     float3 velocity = -hit.normal * collisionRule.Bias;
41.                     buffer.Add(new BoidVelocity(velocity));
42.                  
43.                     Debug.DrawRay(posData.Value, velocity, Color.blue);
44.                  })
45.                  .Schedule();
46.  
47.          // Dispose on completion
48.          Dependency = indexLookup.Dispose(Dependency);
49.          Dependency = commands.Dispose(Dependency);
50.          Dependency = results.Dispose(Dependency);
51.       }

Is RaycastCommand.ScheduleBatch cannot be chained, or does not support DeferredJobArray, or I'm missing something?

 

Why do you think it should? AsDeferredJobArray is for a different purpose, at schedule time it will be 0 here. For schedule job with required length (unknown at schedule time) there is only one job type - IJobParallelForDefer (with NativeList as input argument and AsDeferredJobArray as one of the struct members) and RaycastCommand.ScheduleBatch schedules not this type, and it will fail here at schedule time, as it will be 0.

 

Entities codegen jobs seems to support DeferredJobArray just fine, so I thought everything else should.

Good to know it is RaycastCommand.ScheduleBatch issue, thank you.


How to schedule / chain results in this case? Or is it pretty much .Complete or nothing?

 

DeferredJobArrays does not supported in this case, even if expected length is sent as a parameter, resulting in a hard crash (in case anyone wondered what happens if to hack UnityEngine.PhysicsModule):

Code (CSharp):

1. InvalidOperationException: Result buffer too small, this will be possible once the API uses NativeList
2.   at (wrapper managed-to-native) UnityEngine.RaycastCommand.ScheduleRaycastBatch_Injected(Unity.Jobs.LowLevel.Unsafe.JobsUtility/JobScheduleParameters&,void*,int,void*,int,int,Unity.Jobs.JobHandle&)
3.   at UnityEngine.RaycastCommand.ScheduleRaycastBatch (Unity.Jobs.LowLevel.Unsafe.JobsUtility+JobScheduleParameters& parameters, System.Void* commands, System.Int32 commandLen, System.Void* result, System.Int32 resultLen, System.Int32 minCommandsPerJob) [0x00000] in <7d971898ea0f4307a8eddf491ea76525>:0
4.   at UnityEngine.RaycastCommand.ScheduleBatch (Unity.Collections.NativeArray`1[T] commands, System.Int32 expectedLength, Unity.Collections.NativeArray`1[T] results, System.Int32 minCommandsPerJob, Unity.Jobs.JobHandle dependsOn) [0x00035] in <7d971898ea0f4307a8eddf491ea76525>:0

So I guess no way around it. Unless somehow defer execution of gather commands job, receive results on main thread, and pass it back to the job.

 

Okay, feeling really dumb dumb.
Could've just used plain NativeArray + entityInQueryIndex in ForEach to act as an index.


No need to use NativeList + that solves the issue. Even when in job chain it works correctly.

 

I think you misunderstood the purpose of the deferred array, let me explain.
First of all:

What is your meaning "they support"? There is only 2 codegen gob types -

Entities.ForEach

and

Job.WithCode

, first one don't support numeric arguments in Schedule (as it converts to

IJobChunk

and then schedules based on EntityQuery), the second one too as it only supports

Run

on main thread or

Schedule

in parallel on one worker thread (it haven't parallel schedule).
If you mean construction like this it's of course supported, and this supported by any job because it's a different thing, we not use this for SCHEDULE calculation itself:

Code (CSharp):

1. var nativeList = new NativeList<int>(10, Allocator.TempJob);
2.         var array      = nativeList.AsDeferredJobArray();
3.  
4.         Job.WithCode(() =>
5.         {
6.             for (int i = 0; i < 10; i++)
7.             {
8.                 if (i % 2 == 0)
9.                 {
10.                     nativeList.Add(i);
11.                 }
12.             }
13.         }).Schedule();
14.  
15.         Job.WithCode(() =>
16.         {
17.             Debug.Log($"{array.Length}"); //Will return 5
18.         }).Schedule();
19.  
20.         nativeList.Dispose(Dependency);

Or you can use it like this without deferred array:

Code (CSharp):

1. var nativeList = new NativeList<int>(10, Allocator.TempJob);
2.  
3.         Job.WithCode(() =>
4.         {
5.             for (int i = 0; i < 10; i++)
6.             {
7.                 if (i % 2 == 0)
8.                 {
9.                     nativeList.Add(i);
10.                 }
11.             }
12.         }).Schedule();
13.  
14.         Job.WithCode(() =>
15.         {
16.             Debug.Log($"{nativeList.AsArray().Length}"); //Will return 5
17.         }).Schedule();
18.  
19.         nativeList.Dispose(Dependency);

And third variant is just use

nativeList

as is.
The reason why

AsDeferredJobArray

exists here is you can't use

AsArray

as it will throw you error if you'll try to use it as an member for job IF another job\or anything modifies original list (as it reference to this list and not newly allocated array and any changes in list will invalidate this array):
Will work and return 0:

Code (CSharp):

1. var nativeList = new NativeList<int>(10, Allocator.TempJob);
2.         var array      = nativeList.AsArray();
3.  
4.         Job.WithCode(() =>
5.         {
6.             Debug.Log($"{array.Length}"); //Will return 0
7.         }).Schedule();
8.  
9.         nativeList.Dispose(Dependency);

Wouldn't work as list modified and array invalidated:

Code (CSharp):

1. var nativeList = new NativeList<int>(10, Allocator.TempJob);
2.         var array      = nativeList.AsArray();
3.  
4.        nativeList.Add(1);
5.        //OR
6.        Job.WithCode(() =>
7.        {
8.            nativeList.Add(1);
9.        }).Schedule();
10.  
11.         Job.WithCode(() => //Throw error because row above (no matter inside job it or no - it invalidates AsArray result)
12.         {
13.             Debug.Log($"{array.Length}");
14.         }).Schedule();
15.  
16.         nativeList.Dispose(Dependency);

And this is the reason for

AsDeferredJobArray

when you want to use it initially at

Schedule

time as

NativeArray

field member in a job for the list which we will modify before access array. For example, the latest sample will work if we replace

AsArray

to

AsDeferredJobArray

.

And now we came to the end of the explanation - using this for

Schedule

and why

RaycastCommand.ScheduleBatch

didn't work with that, which is absolutely expected.
The simple reason is

ScheduleBatch

it's the same as

Schedule\ScheduleParallel

method for other jobs, for example, let's get

IJobParallelFor

-

Schedule(10 /*array length*/, 1 /*batch size*/);

where 10 is array length AT SCHEDULE time and 1 is the batch size, we telling to the job (roughly): "Hey run this on 10 worker threads with one

Execute

per thread)" job on main thread prepare everything to process data how we asked him. And it expects you to set some value, if you set - 0 it wouldn't run because of why it should. The same for

RaycastCommand.ScheduleBatch

, on the main thread, while you schedule this job list not populated yet and the deferred array has 0 length (event if you set length to something it also checks pointer to real data and if it's empty will throw you an error (from your latest reply, and as they mentioned they will make it work similar to IJobParallelForDefer) as it will try to schedule a job with length not corresponds to real data in memory.


For this exact purpose exists a specific job type -

IJobParallelForDefer

which

Schedule

gets list and batch size as argument and

AsDeferredJobArray

as member of job. And this allows you not to guess or scheduling a job for the worst case. It will work similar to

IJobParallelFor

, BUT you don't need to specify array length AT SCHEDULE time it will process data of this job and will split it to threads with specified batch size per thread, like "at runtime" after previous job processes list and the job will know result size of the list.

And in the end

RaycastCommand.ScheduleBatch

is not

IJobParallelForDefer

, which is only one unique job type that supports that mechanism.

 

My bad, I meant they allow to receive correctly modified container state after X job in a chain executes (not at schedule time).

Expected only in case if someone poked around sources quite some time.
There's no mention in docs regarding NativeList

Thank you for detailed explanation though, this will definitely be helpful for future readers.