Native Arrays approximately an order of magnitude slower than arrays

I made a little (amateurish) bench mark and got this:

Code (CSharp):

1.    
2.  System.Diagnostics.Stopwatch stopWatch = new System.Diagnostics.Stopwatch();
3.  
4.         for (int i = 0; i <= 10; i += 2) {
5.  
6.             int lnght = i * 1000;
7.  
8.             float[] array = new float[lnght];
9.             NativeArray<float> nArray = new NativeArray<float>(lnght, Allocator.Temp);
10.  
11.             for (int a = 0; a < lnght; a++) {
12.                 array[a] = (a % 2 == 0 ? -1 : 1) * a;
13.                 nArray[a] = (a % 2 == 0 ? -1 : 1) * a;
14.             }
15.  
16.             long resultA = 0;
17.             long resultB = 0;
18.  
19.             stopWatch.Restart();
20.  
21.             for (int a = 0; a < lnght; a++)
22.                 for (int b = 0; b < lnght; b++)
23.                     array[a] += array[b];
24.  
25.             resultA = stopWatch.ElapsedMilliseconds;
26.             stopWatch.Restart();
27.  
28.             for (int a = 0; a < lnght; a++)
29.                 for (int b = 0; b < lnght; b++)
30.                     nArray[a] += nArray[b];
31.  
32.             resultB = stopWatch.ElapsedMilliseconds;
33.  
34.             nArray.Dispose();
35.  
36.             Debug.Log("lngth: " + lnght + "  array : " + resultA + "    /     nArray : " + resultB);
37.         }
38.  

lngth: 0 array : 0 / nArray : 0
lngth: 2000 array : 24 / nArray : 337
lngth: 4000 array : 97 / nArray : 1338
lngth: 6000 array : 212 / nArray : 2951
lngth: 8000 array : 389 / nArray : 5241
lngth: 10000 array : 608 / nArray : 8189

I understand that this will be mitigated by the Burst compiler in a Jobs situation, but I'd be interested in hearing people's thoughts on it. Any insight as to why? Is it a beta thing that can be expected to go away?

 

Native array access from c# code outside of jobs is slower because of the extra bounds checks etc. This should not be done in builds.

 

Thanks for the clarification! With "this should not be done", do you mean that it's bad practice and I shouldn't do it, or that the overhead ought to disappear when I build the project? Sorry for being dumb ~

 

No. the compiler should only add the extra checks for the native arrays in the editor. I think this is how it works.

 

Also the point in using NativeArray isn't to have faster random access speed, is to have linear memory layout and not garbage collection at all.

 

Sure, makes sense! But the ultimate point is performance, right ... I'm having trouble actually getting any performance gains out of the jobs system, because I'm bottle-necked by shuffling my data to a nativearray and back again. My other idea was to just keep it in a nativearray to begin with, but that got me here - basically all the logic interacting with that nativearray becomes very slow. So end of the day I'm not really able to get any performance out of the system.

Perhaps my situation is just not what the job system is for.

 

That's because Unity isn't ready yet to use pure ECS effectiveness. We still need to convert from and back to managed objects in order to have some basic functionality, like rendering, sound, camera, etc. This should be changed as the Unity team releases new features compatible with Job System, ECS and Burst compiler.

Also, NativeArrays have a lot of checks to guarantee some safety to us. This slow down the access, but should be removed whe you build for production, outside the Editor, as @LennartJohansen pointed out.

 

Perhaps I should just wait then, and try to solve my problems with old school c# threading in the meantime. There's a lot of potential to the jobs but it might be that it's not really viable outside of specific test scenarios just yet!

As for the safety checks, it's of course good news that it'd be faster in the build, but you still have to have decent performance in the editor to be able to work on the project...

 

Hi, I have run my own test and got a result similar to yours in-editor (10x slower). RW is by ++ operator and the amount of elements is 100,000 ints.

array R 3060
nativeArray R 35090
array W 3070
nativeArray W 29991
array RW 3803
nativeArray RW 59456
nativeArray RW (Job) 59796
nativeArray RW (Job Parallel 1) 64967
nativeArray RW (Job Parallel 2) 54481
nativeArray RW (Job Parallel 4) 43390
nativeArray RW (Job Parallel 8) 51902
nativeArray RW (Job Parallel 16) 49467
nativeArray RW (Job Parallel 32) 47657
nativeArray RW (Job Parallel 64) 47050
nativeArray RW (Job Parallel 128) 47827
nativeArray RW (Job Parallel 256) 47104

However when run this in Android device :

array R 8224
nativeArray R 13705
array W 3639
nativeArray W 9387
array RW 10176
nativeArray RW 17188
nativeArray RW (Job) 8859
nativeArray RW (Job Parallel 1) 61381
nativeArray RW (Job Parallel 2) 26555
nativeArray RW (Job Parallel 4) 14314
nativeArray RW (Job Parallel 8) 10844
nativeArray RW (Job Parallel 16) 7672
nativeArray RW (Job Parallel 32) 6644
nativeArray RW (Job Parallel 64) 6740
nativeArray RW (Job Parallel 128) 6800
nativeArray RW (Job Parallel 256) 7462

Don't pay attention to the difference between read and write, when I rerun the test sometimes read is faster sometimes write is faster. I don't know why. But RW should even everything out.

So
1. Safety checks in editor adds 10 times performance hit.
2. In a job it performs better than out of a job. It still lose to array outside of a job. It looked better in a job probably because locality of data. (regular array has its content on heap even if allocated in a local scope and native array out of the job maybe requires some pointer dereferencing, where in the job I think Unity made it more direct and more local)
3. This is still without Burst

(Blog with the code : https://medium.com/gametorrahod/unity-ecs-native-containers-performance-test-aca8964ba80c)

 

I'm finding a similar slowdown. e.g. see Texture2D.GetRawTextureData<T>() ..... when you deal with it as a generic type and return the native array, accessing the pixel data directly is HUGELY slower than accessing a regular byte array. Like 5seconds verses 35 seconds. This unfortunately seems to render the function practically useless for anything other than convenience and is probably even slower than just using GetPixels()/SetPixels() to make a copy of the pixel data. This seems a bit silly to me. I was hoping to see a much faster way to edit pixel data in the system memory without having to involve a copy operation on the whole buffer. But the performance is really bad.

 

I am interested in this new functionality, too (I.e. get pixel data as native array) - when you say no speed gains, did you follow what was said by Joachim in the post before yours?

I will only get to it in 10 days when i return. If you find something out earlier it would be great if you could share

 

Why do you have to add an attribute to enable burst compilation? I might have missed something, but I can't see any reason to not turn on burst if it's possible to do so. If there exists corner cases where it should be disabled, I think it makes more sense for there to be a [DisableComputeOptimization] attribute for those cases rather than the other way around.

Or is this simply something you have planned, but not gotten around to yet?

 

If someone uses something like nativearrayutility.getunsafeptr() and uses unsafe (pointers-are-ok) code.... would this present a lower-level access to the internal "real" memory buffer (like struct data) of the native array, so that when you access elements like in an array, there is none of the "overhead" of native arrays, and it performs at maximum access speeds (like it would when using pointers on regular memory buffers?) or will there still be some kind of behind the scenes middle-man kind of activity interpreting all the accesses? ie can you do away with all the bounds checking and interfaces and so on and just get "full speed" access to the memory this way, even in the editor and without burst-stuff or trying to write jobs?

And if so, how would you pin or 'fix' the memory so that the garbage collector doesn't try to move it?

 

If I recall about jobs there's not really a 'safe' way to have multiple jobs/threads literally share access to the same memory buffer, ie, potential of reading and/or writing to the same byte in the same buffer 'unsafely'... which I actually want to be able to do in my case.

Performance tests so far report that the UnsafePtr approach is waaay faster and pretty much the same performance as a regular int[] array access. Typically the native array is coming in around 7 times slower (in editor), while the unsafe ptr version is about the same "full" speed as a normal array access.

I was able to pin the native array using:

ulong handle; //gc handle
int* pinned = (int*)UnsafeUtility.PinGCObjectAndGetAddress(native,out handle);
int* myints = (int*)NativeArrayUnsafeUtility.GetUnsafePtr(native);

These two return different addresses. I presume the first is the address of "object" (which contains data and a pointer to some memory), while the seconds returns the address of the actual memory? Do you think this guarantees that the pointer to the actual memory is also fixed because the object itself is fixed? Or does the UnsafePtr internally fix the memory behind the scenes?

I'm also not sure if what I'm pinning here is the actual native array of just my reference to it?

(also should I use .SetAtomicSafetyHandle() for some reason to lock it down further or is that just to get some kind of ownership over it so that other methods etc can ask whether it's okay to also access the data at the same time, or?)

(note to self, objects larger than 85000 bytes are put on the large object heap and won't be moved by the garbage collector).

 

Hey @Joachim_Ante while I'm here.... is there any reason why Unity still doesn't let you Apply() a texture with a rect? In my case I am having to split up a large texture into many smaller textures in order to avoid having to push the entire large texture to graphics memory each frame. If Texture2D.Apply() would simply take a rect like Apply(new rect(0,0,64,64)); to apply only a small portion of the texture over the graphics bus, I would be able to just use one large texture and deal with which parts of it need uploading myself. This would open up a world of better performance for me, plus vastly less draw calls because splitting up a big texture into small ones ramps up the draw calls pretty fast to get the upload sizes to be smaller.

e.g in OpenGL v1 there was the instruction glCopyTexSubImage2D() which does exactly this. Any chance we can get a version of Apply() that'll take a rect and only upload the part of the texture within the rectangle?

 

@Joachim_Ante I see.

Are you saying then that if I use the job system, I should be able to access the native array without any hacking around and read/write from/to it for massively intensive 'pixel' operations, and it will run at the same performance level as the GetUnsafePtr() method ...... but with that perofrmance level in the build, not in the editor?

And if I switch over to using jobs then, how would I mark the Unity-created native array from a Texture2D, as having a disabled safety restriction so I can deal with the race conditions "manually"?

You're winning me over to the job system but I need clarification that I'm not going to see LESS performance as a result of working with the Texture's native array this way, and seeing it run so much slower in the editor is a bit offputting.

 

Here's some results I'm seeing in the build using IL2CPP backend, and not using the job system.



note the two main groups are still about 6.9x difference in performance, so I'm assuming anything using pointers is going to be 6.9x faster than 'safe' code.

here I'm using unsafe code and use of pointers with the first 3 tests. The native array which was 7x slower in the editor is now almost as fast as a direct int pointer accessing the ptr of the memory from the native array.

interestingly the int array was way slower, and this is probably because it's using a managed array with bounds checks and all that stuff every time it's accessed - which mainly shows you that using pointers can be a massive performance boost, cus using the same array with an int pointer was much faster. Similarly setpixels32 has a major overhead from being managed (i'm guessing).

interesting also that the native array runs almost as fast as using pointer memory access. is that what I should expect to be seeing? bearing in mind I'm still not using jobs and this is still single-threaded.

the only other difference is that using a pointer on the native array works much faster in the editor than treating it as a regular array.

@Joachim_Ante if I switch to a job with burst compiler etc.... should I possibly see any further improvements to the native array speed in the build (with one thread)?

 

@imaginaryhuman if you search for a thread with scratch pixel collision you can find some benchmarks I did with threads / jobs - last time i updated it with burst results, but this was still editor only at the time (burst not avail for builds). I intend to update this with build benchmarks and also want to update unity to make use of the native arrays for get Pixels ... you will see that I had massive performance gains from burst in the editor

 

having trouble finding it.. do you have a link or the title of the thread or something?

maybe this? https://forum.unity.com/threads/tho...-do-scratch-style-collision-detection.517400/

 

Still wondering how to disable "safety checks" on a shared native array, using the attributes, when the native array was created as part of Texture2D... not something I created with my own native array? Is it even possible?

[EDIT] I found that in the job itself, when you define the native array 'field' in the struct, which is assigned by the main thread, if you put the 'disable container checks' attribute above that field, even though I did not have access to define the nature of the buffer when it was created by Texture2D, assigning the native array to that field tells the job system to 'use it' in that way, allowing multiple threads to share concurrent read/write access to the array (which of course produces per-pixel race conditions).

 

I set up a c# job. It runs provided Burst compiler is not activated. A little faster than running in the main thread.

But I can't get burst to compile on a mac. It fails to build. And the burst inspector window doesn't show any code output for the row that shows up representing my job (even when clicked). I'm using the latest preview package 0.2.4 preview 20. I do understand its in preview and may not work perfectly yet. I have it on IL2CPP and .net4. The errors upon building are:

Failed running ... /Library/Unity/cache/packages/packages.unity.com/com.unity.burst@0.2.4-preview.20/.Runtime/bcl.exe @/var/folders/j3/qggdf4vx76scb9ng7r_gbftw0000gn/T/tmp76c2ae67.tmp

stdout:
No method compiled. Missing options [--assembly and --type] or [--assembly-folder and --method]
stderr:

UnityEngine.Debug:LogError(Object)
UnityEditorInternal.Runner:RunProgram(Program, String, String, String, CompilerOutputParserBase) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:128)
UnityEditorInternal.Runner:RunManagedProgram(String, String, String, CompilerOutputParserBase, Action`1) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:73)
UnityEditorInternal.Runner:RunManagedProgram(String, String) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:45)
Unity.Burst.Editor.BurstAotCompiler:OnPostBuildPlayerScriptDLLs(BuildReport) (at /Users/ImaginaryHuman 1/Library/Unity/cache/packages/packages.unity.com/com.unity.burst@0.2.4-preview.20/Editor/BurstAotCompiler.cs:189)
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

Exception: ... /Library/Unity/cache/packages/packages.unity.com/com.unity.burst@0.2.4-preview.20/.Runtime/bcl.exe did not run properly!
UnityEditorInternal.Runner.RunProgram (UnityEditor.Utils.Program p, System.String exe, System.String args, System.String workingDirectory, UnityEditor.Scripting.Compilers.CompilerOutputParserBase parser) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:130)
UnityEditorInternal.Runner.RunManagedProgram (System.String exe, System.String args, System.String workingDirectory, UnityEditor.Scripting.Compilers.CompilerOutputParserBase parser, System.Action`1[T] setupStartInfo) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:73)
UnityEditorInternal.Runner.RunManagedProgram (System.String exe, System.String args) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildUtils.cs:45)
Unity.Burst.Editor.BurstAotCompiler.OnPostBuildPlayerScriptDLLs (UnityEditor.Build.Reporting.BuildReport report) (at /Users/ImaginaryHuman 1/Library/Unity/cache/packages/packages.unity.com/com.unity.burst@0.2.4-preview.20/Editor/BurstAotCompiler.cs:189)
UnityEditor.Build.BuildPipelineInterfaces.OnPostBuildPlayerScriptDLLs (UnityEditor.Build.Reporting.BuildReport report) (at /Users/builduser/buildslave/unity/build/Editor/Mono/BuildPipeline/BuildPipelineInterfaces.cs:452)
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

 

ok so once I found some extra documentation where it says I need the [burstcompile] attribute on the job, and then finding a couple of lines that are not burst-compatible and fixing those, I no longer got any errors and everything works.

 

Here's some numbers. Going to multithreaded through the burst-compiled job provided about a 3x performance increase compared to single-threaded. All the job is doing is writing 1024x1024 4-byte pixels, over and over.

Now I have to think about how to go "data first" to layout the data for optimal processing.

 

Glad you got it working - it also had good results with burst. Similar use case - i was only reading pixe data though. The thread you linked above sounds right - although I could not open it - I have a lousy connection.

Are you using the native pointer to the textures - this is something that wa snot available in the unity version I used last...

 

Question... if the main thread sets a field of the job struct, to store a native array that was created in the main thread, is it going to put a reference to the native array into the struct field or is it going to try to copy the entire native array into the struct? Like if the native array is a 1024x1024 RGBA texture's data, and you assign the job's field to that native array, will the job have to COPY every single pixel into the job's struct before it can work on it, or is the native array simply "shared" by both systems? Now that I ask this I guess the purpose of the native array is to be shareable right, so it's contents are not automatically copied?

 

You can get the 'unsafe ptr' to the memory buffer of the native array used by the texture2D and do pointer access in unsafe code with that, yes. You can also just use the native array directly like a normal array it's pretty much the same performance in the build, though not in the editor.

 

Ok - great. I will try this out when I am back in a few days. I don’t need the pixel collision for anything - it just came out of this forum post, but it is a little nice side project to see the performance gains from the new features...

 

@Joachim_Ante So... in order to get the most out of the burst compiler and jobs... what are the basics of how memory should be organized (e.g. arrays of data etc), to make it the most optimal output?

I see unity.mathematics has a bunch of nifty SIMD optimized instructions similar to shader languages, which if used could produce a big boost.

I also saw in a video, mention of putting each piece of data into its own array, like an array for x position, an array for y position?

Read/write accesses to data suggests, where possible, to always define if a native container is read-only, write-only or both, so that multiple things can read from the same data where possible. Or I presume that disabling this check with an attribute lets any thread access the data simultaneously.

Ideal to have one thread work on one block of data without other threads interfering, e.g. dividing up a pixel operation into blocks.

There's a preference to use 'temp' allocated memory where possible (e.g in scope of a function), then short-term, then permanent.

Try to avoid jumping around in memory, compact data into arrays and use structs.

Split off 'cold' infrequently accessed data from 'hot' frequently accessed data.

I'm not sure how to handle the case where you have multiple blocks of data (e.g. grid of textures) and you want some job to possibly read/write across many many native arrays, and you want all jobs to be able to do that, and to let them share access... I guess a native array of native arrays given to each job?

Should we be attempting to interleave data at all, like combing x and y coordinates into x, y, x, y kind of pattern?

How can you have for example a single native array that's acting as some kind of index or something, that all of the job instances need to access and modify, like they need to read it and write it and move stuff around in it, but you don't want race conditions... does each job have to get a 'safety handle', like does that work like a mutex to lock the thread until access is available ie safety handle is surrendered by some other thread that was using it?

 

I hope you get an answer - but I think you should try to dig yourself a little in the ECS documentation and examples - they also cover / refer to the jobs documentation. The docs are still very basic and the examples take some time to understand/ explore, because the api is not cast in stone and you can do the same thing in multiple ways.

But it contains a few answers to your questions and there is still some experimenting to be done, depending on your specific use case.

I am myself not very experienced with ecs or threaded code, but I am happy to connect when I am back and share experiences- we seem to do some similar things with accessing / manipulating pixel data. I had optimized pretty much everything in mono, before il2cpp Mac, ecs, burst, math came and that opened up many new opportunities. It’s exciting to experiment with it and see the improvements.

 

I'm a bit stuck. I have a grid of native arrays. For example this represents several Texture2D objects and each of their memory buffers needs to be stored in some way that a single Job can get efficient access to all of them (e.g. particles which scatter to random locations). So the easy way would be putting these into a Native Array. However, this now means putting the native array inside of another native array. A native array is apparently not a blittable type, so Unity won't allow me to do this. So you can't have a native array containing native arrays. I also tried making a custom struct which contains a native array, then having a native array of that custom struct. This also wasn't allowed.

So I'm not sure how I can get access to multiple native arrays (the number of which must be flexible) from within a Job? The only thing I can think to do is to get all the memory buffer pointers to those arrays into an array which seems a bit hacky.

 

Hard to say what the best solution is without knowing what your grid looks like, and why it's a grid. One general approach is map logical arrays to a single concrete array. But a grid sort of implies you need to lookup stuff by coordinates, which implies at least some scale, which implies if you have textures in each cell, a lot of memory. So I'm having a difficult time visualizing this.

 

I basically need to pass a fairly large number of native arrays to a job somehow. Apparently I can't do an array of native arrays. I also can't do a native array of int pointers. So it's probably going to come down to allocating some memory manually and stuffing it with pointers then passing a single int ptr pointing to the memory.

The grid of arrays can be just one dimensional. The hard bit seems to be passing multiple arrays into a job, not having a fixed number of them.

 

can't you start a job for each native array you have, passing only that?
jobs should be pretty lightweight. also those jobs will execute in parallel unless they have dependencies

Code (CSharp):

1. struct Job : IJob {
2.     public NativeArray<Color32> texture;
3.     public void Execute() {...whatever; }
4. }
5.  
6. // main thread
7. NativeArray<Color32>[] allTextures = ...;
8. foreach(var tex in allTextures)
9.     new Job {texture = tex}.Schedule(); // save the handles, you will need them later to complete the jobs
10. JobHandle.ScheduleBatchedJobs(); // <- this does the heavy scheduling work. call once

 

I would be interested in the findings...I have done something similar before (where the work in the job was small, but there were many of those) and it turned out that the scheduling & completing took longer than simply running the loop on the main thread. Maybe I did something wrong.

I guess it depends how much work each job has to do, it would be interesting to get a rough break even point.

 

Can you pass an array of IntPtr and just convert to a pointer within the job? (I was looking at trying this tomorrow for something)

 

unfortunately not a single job needs to access all of them.

 

Yes that's what I ended up doing, making a custom memory buffer with pointers in it and passing the pointer to the buffer into the job.

 

I figured out why an array of native arrays doesn't work, it's because this creates a reference type array as the main array, then that array holds non-reference-type arrays. So e.g. NativeArray<int>[] is actually a reference type array containing native arrays. So then when you try to pass it to a job it won't accept it. So it's not truly a native array of native arrays, it's a managed array of native arrays. Oh well.

 

what deal with those zilion memcpys? are they safety checks too?