Question How do you setup data for one job using the results of another job

Hello, I have a set of jobs that I want to run sequentially. This of course is easy to do, however I need to use the results from one job to construct the data for the next job. So I am wondering what the best way to do this would be.

Originally I was going to do it in a 'setup' IJob that would run between the 'main' jobs and handle it, however I need to create Unity.Splines and that cannot be done within a job unfortunately as it requires IReadOnlyLists to setup properly.

Any advice would be great!

 

When creating your jobs, use the same NativeCollection (NativeList/NativeArray) as the output of your first job and the input of the second job. Then make sure that the JobHandle of the first job is used as a Dependency on your second job.

Example:

Code (CSharp):

1.  [BurstCompile]
2. public struct Job1 : IJobParallelFor
3. {
4.     [WriteOnly]
5.     NativeArray<int> output;
6.  
7.     public Job1(NativeArray<int> outputArray)
8.     {
9.         output = outputArray;
10.     }
11.  
12.     public void Execute(int index)
13.     {
14.         output[index] = 0;
15.     }
16. }
17.  
18. [BurstCompile]
19. public struct Job2 : IJobParallelFor
20. {
21.     [ReadOnly]
22.     NativeArray<int> input;
23.  
24.     public Job2(NativeArray<int> inputArray)
25.     {
26.         input = inputArray;
27.     }
28.  
29.     public void Execute(int index)
30.     {
31.         int value = input[index];
32.     }
33. }
34.  
35. void Update()
36. {
37.     // Create Array
38.     NativeArray<int> valueArray = new NativeArray<int>(128, Allocator.TempJob);
39.     // Create & Schedule Job1 (using default handle)
40.     Job1 job1 = new Job1(valueArray);
41.     JobHandle intermediateHandle = job1.Schedule(128, 8, default);
42.     // Create & Schedule Job1 (using handle gathered from scheduling Job1)
43.     Job2 job2 = new Job2(valueArray);
44.     JobHandle finalHandle = job2.Schedule(128, 8, intermediateHandle);
45.  
46.     // Complete both Jobs (BLOCKING CALL)
47.     finalHandle.Complete();
48. }

 

Hey, thanks for the reply. But that doesn't solve my problem in this case.
As I said, I know how to run the jobs sequentially using the job handle as a dependency.

However I can't do that for setting up the data because I need to make NativeSplines using the data from the first job. And NativeSplines cannot be created within a job.

 

I haven't personally worked with NativeSplines yet.. But can you not create them in NativeLists, then memcpy the data over to a managed list afterwards?

Code (CSharp):

1. /// <summary>
2. /// Copies from NativeList to Managed List by using MemCpy on the underlying arrays
3. /// </summary>
4. public static unsafe void CopyTo<T>(this NativeList<T> from, List<T> to) where T : struct
5. {
6.     if (to.Capacity < from.Length)
7.         to.Capacity = from.Length;
8.     T[] toArray = NoAllocHelpers.ExtractArrayFromListT<T>(to);
9.     int objSize = UnsafeUtility.SizeOf<T>();
10.     UnsafeUtility.MemCpy(UnsafeUtility.AddressOf(ref toArray[0]), from.GetUnsafeReadOnlyPtr(), from.Length * objSize);
11.     NoAllocHelpers.ResizeList(to, from.Length);
12. }
13.  
14. public static class NoAllocHelpers
15. {
16.     /// <summary>
17.     /// Cached Delegates per type to reduce performance-impact after first call
18.     /// </summary>
19.     private static readonly Dictionary<Type, Delegate> ExtractArrayFromListTDelegates = new Dictionary<Type, Delegate>();
20.     /// <summary>
21.     /// Cached Delegates per type to reduce performance-impact after first call
22.     /// </summary>
23.     private static readonly Dictionary<Type, Delegate> ResizeListDelegates = new Dictionary<Type, Delegate>();
24.  
25.     /// <summary>
26.     /// Extract the internal array from a list.
27.     /// </summary>
28.     /// <typeparam name="T"><see cref="List{T}"/>.</typeparam>
29.     /// <param name="list">The <see cref="List{T}"/> to extract from.</param>
30.     /// <returns>The internal array of the list.</returns>
31.     public static T[] ExtractArrayFromListT<T>(List<T> list)
32.     {
33.         if (!ExtractArrayFromListTDelegates.TryGetValue(typeof(T), out var obj))
34.         {
35.             var ass = Assembly.GetAssembly(typeof(Mesh)); // any class in UnityEngine
36.             var type = ass.GetType("UnityEngine.NoAllocHelpers");
37.             var methodInfo = type.GetMethod("ExtractArrayFromListT", BindingFlags.Static | BindingFlags.Public)
38.                 .MakeGenericMethod(typeof(T));
39.             obj = ExtractArrayFromListTDelegates[typeof(T)] = Delegate.CreateDelegate(typeof(Func<List<T>, T[]>), methodInfo);
40.         }
41.         var func = (Func<List<T>, T[]>)obj;
42.         return func.Invoke(list);
43.     }
44.     /// <summary>
45.     /// Resize a list.
46.     /// </summary>
47.     /// <typeparam name="T"><see cref="List{T}"/>.</typeparam>
48.     /// <param name="list">The <see cref="List{T}"/> to resize.</param>
49.     /// <param name="size">The new length of the <see cref="List{T}"/>.</param>
50.     public static void ResizeList<T>(List<T> list, int size)
51.     {
52.        if (!ResizeListDelegates.TryGetValue(typeof(T), out var obj))
53.        {
54.            var ass = Assembly.GetAssembly(typeof(Mesh)); // any class in UnityEngine
55.            var type = ass.GetType("UnityEngine.NoAllocHelpers");
56.            var methodInfo = type.GetMethod("ResizeList", BindingFlags.Static | BindingFlags.Public)
57.                 .MakeGenericMethod(typeof(T));
58.            obj = ResizeListDelegates[typeof(T)] =
59.                 Delegate.CreateDelegate(typeof(Action<List<T>, int>), methodInfo);
60.        }
61.        var action = (Action<List<T>, int>)obj;
62.        action.Invoke(list, size);
63.    }
64. }

 

Hey, sadly you cannot. As I said in my original post the constructor for the NativeSpline requires a IReadOnlyList as a parameter. Which for some reason NativeArray doesn't implement(!?).

This is the constructor for the NativeSpline

Code (CSharp):

1. public NativeSpline(IReadOnlyList<BezierKnot> knots, IReadOnlyList<int> splits, bool closed, float4x4 transform, Allocator allocator = Allocator.Temp)

And in the body of the constructor it uses a static method in the

CurveUtility

class.

Code (CSharp):

1. public static void CalculateCurveLengths(BezierCurve curve, DistanceToInterpolation[] lookupTable)

So as you can see, it uses arrays which cannot be used in jobs. I was thinking that it might be best to modify the splines code (or a copy of it I guess) to allow for it to work in Jobs since this is the only code at the moment that doesn't work in jobs.

However I would still like to know a good way of handling this sort of scenario in the future of how to modify data before going on to the next job.

 

idk about the other issue with the

CurveUtility

, but this issue can be worked around by making a wrapper type that implements

IReadOnlyList

and stores the

NativeArray

.

Oh, but then you're using interfaces which are boxed and don't work with jobs. Hmm...

 

@MechaWolf99
Did you ever find a nice way to interpret NativeArray as an IReadOnlyList or otherwise construct a Unity spline in Burst?

 

FYI, you can use arrays in jobs, they just can't be burst compiled. The other part is that you can't pass them into a job as they are, since you can only pass in blittable types. You can use

GCHandle

(see here) to pass references to a managed object to the Job. I use that frequently to handle running managed code in between jobs without having to call

Job.Complete()

.

It will look like the following:

Code (CSharp):

1. public static JobHandle RunJob(int[] arr, JobHandle inputDeps)
2. {
3.     var arrHandle = GCHandle.Alloc(arr);
4.  
5.     // schedule the job
6.     inputDeps = new MyJob { ArrayHandle = arrHandle }.Schedule(inputDeps);
7.  
8.     // free the handle after the job finishes
9.     inputDeps = arrHandle.Free(inputDeps);
10.  
11.     return inputDeps;
12. }
13.  
14. // not burst compiled
15. public struct MyJob : IJob
16. {
17.   public GCHandle ArrayHandle
18.  
19.   public void Execute()
20.   {
21.     // get the arr from the handle
22.     var arr = (int[])ArrayHandle.Target;
23.   }
24. }
25.  
26. public static JobHandle Free(this GCHandle handle, JobHandle inputDeps)
27. {
28.   return new FreeHandleJob { Handle = handle }.Schedule(inputDeps);
29. }
30.  
31. public struct FreeHandleJob : IJob
32. {
33.   public GCHandle Handle;
34.  
35.   public void Execute()
36.   {
37.     Handle.Free();
38.   }
39. }
40.  

 

With some tricks you actually can use Lists and Arrays in a job, using somewhat the same trick you do there. I actually have a utility for that here: https://github.com/Per-Morten/UnityUtilities. If you add ViewAsNativeArrayExtensions (and NoAllocHelpers which it currently depends on) you can create a List<>, set it's size to a reasonable (using NoAllocHelpers.ResizeList), view it as a NativeArray, send it to a burst compiled job, and afterwards resize the list to it's actual size. Then send it to the NativeSpline.

Not familiar with splines API, or what transforms you're doing with it. But here's an "arbitrary" example of how to at least deal with using a list in a job, and a way to deal with "variable" length outputs. Currently this doesn't work with IL2CPP (ViewAsNativeArray extensions doesn't work with IL2CPP yet, but that will be updated in the future)

Code (CSharp):

1.  
2. void Update()
3. {
4.     var myList = new List<int>();
5.     myList.Capacity = 30; // Maximum number of elements I'm going to insert
6.     NoAllocHelpers.ResizeList(myList, 30); // Resize so that myListAsNativeArray can write to entire array
7.     using var resultingCount = new NativeReference<int>(0, Allocator.TempJob);
8.     using var _ = myList.ViewAsNativeArray(out NativeArray<int> myListAsNativeArray);
9.     new RandomlyCopyIndexToListJob
10.     {
11.         Result = myListAsNativeArray,
12.         Count = resultingCount,
13.     }
14.     .Schedule()
15.     .Complete()
16.     NoAllocHelpers.ResizeList(myList, resultingCount.Value); // Resize so that List.Count reflets how many values was written to the list.
17.     APITakingIReadOnlyList.DoThing(myList);
18. }
19.  
20. struct RandomlyCopyIndexToListJob
21. {
22.     public NativeReference<int> Count;
23.     public NativeArray<int> Result;
24.     public void Execute()
25.     {
26.         var count = 0;
27.         for (int i = 0; i < Result.Length; i++)
28.            if (i % 2 == 0)
29.               Result[count++] = i;
30.         Count.Value = count;
31.     }
32. }
33.