DeferredJobArray with IJobParallelForFilter

In trying to remove sync points from my systems, I've run into a point where I need to use the result of an IJobParallelForFilter as the list for a subsequent job. With other Job types I'm able to use NativeList.AsDeferredJobArray; however, that doesn't seem to be working the same way here. My assumption is this has something to do with ScheduleAppend and ScheduleFilter only taking a NativeList instead of a NativeArray, and that for this to work properly I'd need to be able to pass the AsDeferredJobArray version to the filter job as well.

Code (CSharp):

1. _filteredIndicesList = new NativeList<int>(Allocator.TempJob);
2.             result = new FilterCheckedTreesJob
3.             {
4.                 ToolEvent = toolEvent,
5.                 BrushEvent = brushEvent,
6.                 UncheckedTrees = _uncheckedTreesArray,
7.                 TreeData = GetComponentDataFromEntity<Tree>(true),
8.                 MountainPositionData = GetComponentDataFromEntity<MountainPosition>(true)
9.             }.ScheduleAppend(_filteredIndicesList, _uncheckedTreesArray.Length, 8, result);
10.  
11. result = new ResizeListJob
12.             {
13.                 Size = _filteredIndicesList.AsDeferredJobArray().Length, // returning 0
14.                 List = _treesToDemolish
15.             }.Schedule(result);

Weirdly though, when I Complete the first job manually (for sanity checking that my filter job wasn't just returning 0 results), the subsequent deferred array still returns 0:

Code (CSharp):

1. _filteredIndicesList = new NativeList<int>(Allocator.TempJob);
2.             result = new FilterCheckedTreesJob
3.             {
4.                 ToolEvent = toolEvent,
5.                 BrushEvent = brushEvent,
6.                 UncheckedTrees = _uncheckedTreesArray,
7.                 TreeData = GetComponentDataFromEntity<Tree>(true),
8.                 MountainPositionData = GetComponentDataFromEntity<MountainPosition>(true)
9.             }.ScheduleAppend(_filteredIndicesList, _uncheckedTreesArray.Length, 8, result);
10.             result.Complete();
11.  
12.             UnityEngine.Debug.Log("Hacky complete length: " + _filteredIndicesList.Length); // returns 60
13.  
14. result = new ResizeListJob
15.             {
16.                 Size = _filteredIndicesList.AsDeferredJobArray().Length, // still returns 0
17.                 List = _treesToDemolish
18.             }.Schedule(result);

Any idea what's going on here? Is it simply not currently supported to use deferred arrays with IJobParallelForFilters? If so I suppose I can use the multi-system approach from my other thread, but I'd prefer to avoid that if possible!

 

In this case you no need in deferred list, because you no need length for scheduling job. Just replace Size int to just NativeList and inside your job call Length. Your jobs should be in dependency chain. In addition Deferred array works not like you expect it in this case, at schedule time it will be 0 Length, and you getting length at schedule time just as int and this int variable in your job not change at execution time.

 

Ah thanks for the explanation! To be honest I didn't really understand what a deferred array was doing, I had just copied an example that worked for another case in my project and so assumed I knew enough to use them...

So, I do actually need the length for the NEXT job in the chain. My assumption would be then that I should use the newly resized List as the deferred one, but that requires me to pass it as a NativeArray to ResizeListJob, and NativeArray doesn't have a Resize method. Allocating a new array inside the job doesn't seem to work, I assume due to having to use Allocator.Temp?

Code (CSharp):

1. // [BurstCompile]
2. struct ResizeListJob : IJob
3. {
4.     [ReadOnly] public NativeList<int> SizeList;
5.     public NativeArray<Entity> List;
6.  
7.     public void Execute()
8.     {
9.         List = new NativeArray<Entity>(SizeList.Length, Allocator.Temp, NativeArrayOptions.UninitializedMemory);
10.         // List.Resize(SizeList.Length, NativeArrayOptions.UninitializedMemory);
11.     }
12. }
13.  
14. // [BurstCompile]
15. struct BuildArrayJob : IJobParallelFor
16. {
17.     [ReadOnly] public NativeArray<Entity> UnfilteredEntities;
18.     [ReadOnly] public NativeList<int> FilteredIndices;
19.  
20.     [NativeDisableParallelForRestriction]
21.     public NativeList<Entity> FilteredEntities;
22.  
23.     public void Execute(int index)
24.     {
25.         FilteredEntities[index] = UnfilteredEntities[FilteredIndices[index]];
26.     }
27. }
28.  
29. protected override JobHandle OnUpdate(JobHandle inputDeps)
30. {
31.     // ...
32.  
33.     _filteredIndicesList = new NativeList<int>(Allocator.TempJob);
34.     result = new FilterCheckedTreesJob
35.     {
36.         ToolEvent = toolEvent,
37.         BrushEvent = brushEvent,
38.         UncheckedTrees = _uncheckedTreesArray,
39.         TreeData = GetComponentDataFromEntity<Tree>(true),
40.         MountainPositionData = GetComponentDataFromEntity<MountainPosition>(true)
41.     }.ScheduleAppend(_filteredIndicesList, _uncheckedTreesArray.Length, 8, result);
42.  
43.     // Build list
44.     _treesToDemolish = new NativeList<Entity>(Allocator.TempJob);
45.     result = new ResizeListJob
46.     {
47.         SizeList = _filteredIndicesList,
48.         List = _treesToDemolish.AsDeferredJobArray()
49.     }.Schedule(result);
50.  
51.     // Construct the new filtered list into a usable entity array
52.     result = new BuildArrayJob
53.     {
54.         UnfilteredEntities = _uncheckedTreesArray,
55.         FilteredIndices = _filteredIndicesList,
56.         FilteredEntities = _treesToDemolish
57.     }.Schedule(_treesToDemolish.AsDeferredJobArray().Length, 8, result);
58. }
59.  

If I log SizeList.Length in ResizeListJob then I get the correct value, but if I log FilteredEntities.Length in BuildArrayJob then.. it doesn't log anything, because I assume it's scheduling with a Length value of 0; and if I log the length of it later, outside a job after a sync point, then I get 0.

So I think the question now is "how do I use a list that I need to resize as a deferred array?"

 

Is there documentation on how to use those? This is the only page I see.

 

You should use IJobParallelForDefer instead of IJobParallelFor and pass list to field as deferred array and as argument to Schedule

 

Thanks, you've saved me again! I had no idea IJobParallelForDefer existed, everything works perfectly now.

Documentation on how to use what? Filter jobs? I haven't found anything substantial in the official docs but I did find this set of unit tests to be helpful: https://gist.github.com/tsubaki/ef2387e1b61c36dac351b37fa0527224

 

So you return a bool on execute. What does returning true or false mean? Does it mean to finish the job if false is returned?

 

Returning true means the index is appended to the list of indexes (if using ScheduleAppend) or the item is kept in the list (if using ScheduleFilter). Returning false means the index is not appended (ScheduleAppend), or the item is removed from the list (ScheduleFilter).

 

I see. Thanks. Now I know how to use it.