Question NativeArray get/set_item seems really expensive?

One thing that always feels ugly to me is that getting data into/out of a job seems incredibly expensive, almost to the point of eliminating any significant performance gains of putting the data into a job in the first place. This mostly appears to be due to populating native arrays prior to executing the job, and reading data out of native arrays after the job is complete.

Here I'm populating the array before executing the job:


Here I'm reading the array after executing the job:



This profiling is being done in a dev build with deep profiling enabled.

Is there something I can be doing to greatly reduce this overhead? I'm assuming it's proportional to how much data is going into the array. (I'm currently putting a couple of Vector3s, a couple of bools, and some other floats.) I'm not sure I can really reduce how much data is going in.

I'm on Unity 2019.4. Is there any hope that this sort of thing is faster in later versions of Unity? Faster using il2cpp? Is there just some other approach I should be using to populating a native array for my jobs?

 

I've been working on optimizing my code, following some hunches, and so far I've gotten some really significant improvements in performance by ensuring that the structs in my NativeArrays are as small as they possibly can be. The get/set_item call performance is directly proportional to the size of the struct.

Previously, I was using one fairly large struct as the Input and Output of my job. That seemed efficient to me when I first authored that code. But reading that NativeArray after running the job was very expensive due to all of the "input" data in the struct.

So, I refactored my code to have two Native Arrays, one for input, and one for output. Basically my old struct was split into two structs. I'm finding that populating the input takes less time now (set_item has less work to do), and reading the output takes much less time, since get_item is only accessing a very small struct compared to what it used to be getting.

I'm just writing this up in case it helps anyone. My new "rule" for job performance will be to keep structs as small as possible, even if it means having multiple arrays. This approach of separating input from output has resulted in something like a 40% performance improvement for my code, given that such a large portion of the execution time was just get/set_item.

 

To my surprise, and a bit to my disappointment, I think the large NativeArray copy actually seems to have no impact on performance after all, in a release build. In a Dev build, with Deep Profiling enabled, I was seeing big improvements in both FPS and method call timing, as I reduced the data in my NativeArrays. That was exciting. But when comparing Release builds of the before and after, the FPS was identical. I have to assume that the performance differences I was seeing in the dev build are just optimized away in a release build. Pretty surprising, and makes me wonder how I can effectively profile a dev build if some of the code I'm profiling won't even exist in a release build...

I guess I would try for that, if it weren't that the inputs (and outputs) of my NativeArray have Physics dependencies. for example, I'm populating the NativeArray with Positions and Masses of rigidbodies, and computing forces which end up as the output, to be applied to the rigidbodies. I don't think there's any way to do that kind of thing in bursted code.

 

Yeah I feel like the only way to really profile Jobs/Burst stuff is to compare timings in release with your own instrumentation. The profiler isn't yielding much.

I ended up on this thread after walking into the Editor Profiler and finding a big NativeArray.`1.set_Item() eating up time... but I am going to just say that's likely not an issue in release.

 

Other than NativeList, I don't see this method on other containers. Is it still on the plan or you have changed your thought?

 

You can use extension methods to implement it

Code (CSharp):

1.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
2.         public static unsafe ref T ElementAsRefUnsafe<T>(this NativeList<T> list, int index) where T : unmanaged
3.         {
4.             return ref UnsafeUtility.ArrayElementAsRef<T>(list.GetUnsafePtr(), index);
5.         }
6.  
7.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
8.         public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this NativeList<T> list, int index) where T : unmanaged
9.         {
10.             return ref UnsafeUtility.ArrayElementAsRef<T>(list.GetUnsafeReadOnlyPtr(), index);
11.         }
12.  
13.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
14.         public static unsafe ref T ElementAsRefUnsafe<T>(this NativeArray<T> array, int index) where T : unmanaged
15.         {
16.             return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafePtr(), index);
17.         }
18.  
19.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
20.         public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this NativeArray<T> array, int index) where T : unmanaged
21.         {
22.             return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafeReadOnlyPtr(), index);
23.         }
24.  
25.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
26.         public static unsafe ref T ElementAsRefUnsafe<T>(this UnsafeList<T> list, int index) where T : unmanaged
27.         {
28.             return ref UnsafeUtility.ArrayElementAsRef<T>(list.Ptr, index);
29.         }
30.  
31.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
32.         public static unsafe ref readonly T ElementAsReadOnlyRefUnsafe<T>(this UnsafeList<T> list, int index) where T : unmanaged
33.         {
34.             return ref UnsafeUtility.ArrayElementAsRef<T>(list.Ptr, index);
35.         }
36.  
37.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
38.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this UnsafeList<T> list, int start, int length) where T : unmanaged
39.         {
40.             return new ReadOnlySpan<T>(list.Ptr + start, length);
41.         }
42.  
43.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
44.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeList<T> list) where T : unmanaged
45.         {
46.             return new ReadOnlySpan<T>(list.GetUnsafeReadOnlyPtr(), list.Length);
47.         }
48.  
49.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
50.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeList<T> list, int start, int length) where T : unmanaged
51.         {
52.             return new ReadOnlySpan<T>(list.GetUnsafeReadOnlyPtr() + start, length);
53.         }
54.  
55.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
56.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this UnsafeList<T> list) where T : unmanaged
57.         {
58.             return new ReadOnlySpan<T>(list.Ptr, list.Length);
59.         }
60.  
61.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
62.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeArray<T> array, int start, int length) where T : unmanaged
63.         {
64.             return new ReadOnlySpan<T>((T*)array.GetUnsafeReadOnlyPtr() + start, length);
65.         }
66.  
67.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
68.         public static unsafe ReadOnlySpan<T> AsReadOnlySpan<T>(this NativeArray<T> array) where T : unmanaged
69.         {
70.             return new ReadOnlySpan<T>(array.GetUnsafeReadOnlyPtr(), array.Length);
71.         }
72.  
73.         [MethodImpl(MethodImplOptions.AggressiveInlining)]
74.         public static unsafe ref T ValueAsRefUnsafe<T>(this NativeReference<T> nativeRef) where T : unmanaged
75.         {
76.             return ref UnsafeUtility.AsRef<T>(nativeRef.GetUnsafePtr());
77.         }