Examples of ParallelForFilter

I was curious if there were any 'proper' examples as well as perchance documentation for the intended use case of the IParallelForFilter.

 

I played with this some months back (this is still 2018.3, so old API) and leave this here...maybe it helps.

Spoiler: old test

Code (CSharp):

1. namespace ParallelForFilterTest
2. {  
3.     public class JobFilter : MonoBehaviour
4.     {
5.         // Use this for initialization
6.         void Start()
7.         {
8.             NativeList<int> indices = new NativeList<int>(Allocator.TempJob);
9.             NativeArray<int> numbers = new NativeArray<int>(5, Allocator.TempJob);
10.             numbers[0] = 0;
11.             numbers[1] = 1;
12.             numbers[2] = 0;
13.             numbers[3] = 1;
14.             numbers[4] = 0;
15.             var handle = default(JobHandle);
16.            
17.             /*
18.             var condition = new Condition
19.             {
20.                 indices = indices,
21.                 numbers = numbers
22.             };
23.             handle = condition.Schedule(handle);
24.             */
25.            
26.             var append = new Append
27.             {
28.                 numbers = numbers
29.             };
30.             handle = append.ScheduleAppend(indices, numbers.Length, 1, handle);
31.            
32.             var filter = new Filter
33.             {
34.                 numbers = numbers,
35.             };
36.             handle = filter.ScheduleFilter(indices, 1, handle);
37.             handle.Complete();
38.             Debug.Log(indices.Length);
39.             indices.Dispose();
40.             numbers.Dispose();
41.         }
42.        
43.         private struct Condition : IJob
44.         {
45.             public NativeList<int> indices;
46.             public NativeArray<int> numbers;
47.             public void Execute()
48.             {
49.                 for (int i = 0; i < numbers.Length; i++)
50.                 {
51.                     if (numbers[i] == 1)
52.                     {
53.                         indices.Add(i);
54.                     }
55.                 }
56.             }
57.         }
58.        
59.         // the first filter job must be scheduled with ScheduleAppend to create the initial NativeList containing the filtered indices to be iterated through (not required, if the Native List Already Exists)
60.         private struct Append : IJobParallelForFilter
61.         {
62.             public NativeArray<int> numbers;
63.             public bool Execute(int index)
64.             {
65.                 return numbers[index] == 1;
66.             }
67.         }
68.        
69.         // can chain a few of these for further filtering
70.         private struct Filter : IJobParallelForFilter
71.         {
72.             public NativeArray<int> numbers;
73.             public bool Execute(int index)
74.             {
75.                 Debug.Log(index);
76.                 //return true;
77.                 return (index == 1);
78.             }
79.         }
80.     }
81. }

 

This helps partially, but not so much for using it in the job system, as your example has cleanup handled via the start callback.

I'm trying to determine how to dispose/deallocate a native list (the indices) after the job is completed with it. The Attribute doesn't apply to native lists, so the only method I have found is completing the job manually and dispose the list myself.

 

Since you can't dispose of native lists from jobs the simplest solution is to keep a reference to your list in the system and do

Code (CSharp):

1.         if (nativeList_.IsCreated)
2.             nativeList_.Dispose();

At the start of OnUpdate to clean it up next frame and in OnDestroy to clean it up when the system is destroyed.

 

Requires a complete on the next frame, which I don't mind doing, but that makes for a lot of boiler plate. Was just ensuring that there wasn't a better way from within the job. I assume no.

 

I am not entirely sure, what exactly you question is. So the example helped understanding the usage and now you want to know how to deallocate the list?

You could just allocate the list permanently in on create and dispose on destroy.

Then you have to clear it before scheduling the job

 

Use
[DeallocateOnJobCompletion]

 

That doesn't work on NativeList

 

Granted whilst this is true, the example you provided was in non ECS oriented land. Where the job completion is all in your domain. This isn't ideal in some cases. Your example helped, but the issue that arose was the list can not be deallocated via the attribute, which creates a sync point every time you want to use this filter method which is not ideal

What I am after is a filter that would be relatively smooth without forcing main thread to block for more than it has to.

 

Just a heads up you should probably put

Code (CSharp):

1.         if (nativeList_.IsCreated)
2.             nativeList_.Dispose();

in a separate Monobehaviour's FixedUpdate/Update. Use the singleton pattern to assign the NativeContainers of the Monobehaviour from inside a System. The reason for this is that a System may not run next couple of frames if you remove a required component for it to run during the previous frame. With a Monobehaviour the check for disposal always runs, albeit with some overhead. Here's what I did

Code (CSharp):

1. pathfindingDataHandle = GetExposedHelper.ProcessAll(
2.             pathfindingDataHandle,
3.             pathfindingHeightGrid,
4.             pathfindingMapGrid,
5.             pathfindingGridLength,
6.             //FlankDisposer.GetInstance().pathfinderGridIndices,
7.             FlankDisposer.GetInstance().enemiesVantages.AsDeferredJobArray(),
8.             enemiesLength,
9.             collisionWorld,
10.             out var pathfindingExposedGrid,
11.             out var pathfindingExposedGridLength
12.         );
13.  
14.  

The FlankDisposer:

Code (CSharp):

1. public class FlankDisposer : MonoBehaviour
2. {
3.     private static FlankDisposer instance;
4.  
5.     // destination
6.     public NativeList<int> destinationMapMissesIndices;
7.     public NativeList<int> destinationHeightsMissesIndices;
8.     public NativeList<int> coverDisparityMissesIndices;
9.     public NativeList<int3> enemiesVantages;
10.     public NativeList<int> destinationCandidateIndices;
11.     public NativeList<DestinationCandidate> destinationCandidateSuccesses;
12.     public NativeList<int2> destinationCandidateSuccessesLocations;
13.     public NativeList<int> destinationGridIndices;
14.  
15.     // pathfinder
16.     public NativeList<int> pathfinderGridIndices;
17.     public NativeList<int2> pathfindingResults;
18.     public NativeList<int> pathfindingResultsLengths;
19.     public NativeList<byte> pathfindingSuccesses;
20.     public NativeList<int2> path;
21.  
22.  
23.     public static FlankDisposer GetInstance()
24.     {
25.         return instance;
26.     }
27.  
28.     public void Release()
29.     {
30.         if (destinationMapMissesIndices.IsCreated)
31.         {
32.             destinationMapMissesIndices.Dispose();
33.         }
34.         if (destinationHeightsMissesIndices.IsCreated)
35.         {
36.             destinationHeightsMissesIndices.Dispose();
37.         }
38.         if (enemiesVantages.IsCreated)
39.         {
40.             enemiesVantages.Dispose();
41.         }
42.         if (coverDisparityMissesIndices.IsCreated)
43.         {
44.             coverDisparityMissesIndices.Dispose();
45.         }
46.         if (destinationCandidateIndices.IsCreated)
47.         {
48.             destinationCandidateIndices.Dispose();
49.         }
50.         if (destinationCandidateSuccesses.IsCreated)
51.         {
52.             destinationCandidateSuccesses.Dispose();
53.         }
54.         if (destinationCandidateSuccessesLocations.IsCreated)
55.         {
56.             destinationCandidateSuccessesLocations.Dispose();
57.         }
58.         if (destinationGridIndices.IsCreated)
59.         {
60.             destinationGridIndices.Dispose();
61.         }
62.  
63.         if (pathfinderGridIndices.IsCreated)
64.         {
65.             pathfinderGridIndices.Dispose();
66.         }
67.         if (pathfindingResults.IsCreated)
68.         {
69.             pathfindingResults.Dispose();
70.         }
71.         if (pathfindingSuccesses.IsCreated)
72.         {
73.             pathfindingSuccesses.Dispose();
74.         }
75.         if (pathfindingResultsLengths.IsCreated)
76.         {
77.             pathfindingResultsLengths.Dispose();
78.         }
79.         if (path.IsCreated)
80.         {
81.             path.Dispose();
82.         }
83.     }
84.     private void Awake()
85.     {
86.         instance = this;
87.     }
88. }
89.  

Have a x-Disposer for each System, and have a single Monobehaviour Disposer call every x-Disposer's Release method during its FixedUpdate/Update as well as OnDestroy methods.

 

@Soaryn

I do not understand you problem (the sync point), could you paste some code?

If you have a long running job, you can check if it is completed, then call complete & dispose + schedule a new job. I would likely just allocate permanently as I mentioned earlier and .Clear() the list before scheduling the job

 

https://docs.unity3d.com/Packages/com.unity.collections@0.0/changelog/CHANGELOG.html

 

Perfect! Thanks buddy for pointing out that change

 

Finally.