[Solved] Access readonly NativeArray in c# task thread

The nativearray is set to readonly in the declaration and in the job.

When the job is not running, I can access the nativearray from another thread without error messages.

When the job is running, I get the message its forbidden.

What's the sense here? The nativearray is completely readonly, so there can't be any thread safety issues.

I need this for my setup as I have a background worker queue system which is doining non-frame critical work which really works well for me.

EDIT: Title updated for more clarity. The question is about c# task threads accessing readonly nativearrays.

 

As long as you only read from the array in all threads it should be fine and be allowed. While anyone is reading, writing is forbidden as it is not necessarily atomic and surely not deterministic.

 

But unity tells me otherwise:
You are not allowed to access a NativeArray outside the main thread and job threads.
at (wrapper managed-to-native)

The regarding nativearray is set to readonly...

 

Ah, your own thread is the problem. I'm pretty sure it's a bug that the exception is not thrown when no job accessed the array yet. I don't know the exact reasons, probably Unity is doing some per thread intializations to make the safety system work correctly which is not executed for your thread. Would be nice to have a way to integrate your own threads, yes.

Workaround, pass an aliased NativeArray to your worker:

Code (CSharp):

1. threadArray = NativeArrayUnsafeUtility.ConvertExistingDataToNativeArray<int>(array.GetUnsafeReadOnlyPtr(), array.Length, Allocator.Invalid);
2. #if ENABLE_UNITY_COLLECTIONS_CHECKS
3.         NativeArrayUnsafeUtility.SetAtomicSafetyHandle(ref threadArray, AtomicSafetyHandle.Create());
4. #endif

Of course this makes the safety system unaware of invalid access patterns.

 

Ok I think my description was a bit lazy: I want to read from a readonly nativearray via a c# task thread along with a unity job.

It works when I don't have any unity jobs reading from that nativearray. But if a unity job is reading from it too in parallel to a c# task thread, I get that error message.

So I suppose accessing a nativearray from a c# job is forbidden, even if its readonly ? Just want to clarify that.

For my explanation I'm using persistent c# background tasks for loading big data chunks in the background as its not frame critical.

Spoiler

System.InvalidOperationException: You are not allowed to access a NativeArray outside the main thread and job threads.
at (wrapper managed-to-native) Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckReadAndThrowNoEarlyOut_Injected(Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle&)
at Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle.CheckReadAndThrowNoEarlyOut (Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle handle) <0x42625be0 + 0x00052> in <2f14d49edc704940aee557642343f344>:0
at Unity.Collections.NativeArray`1[T].CheckElementReadAccess (System.Int32 index) [0x0005e] in C:\buildslave\unity\build\Runtime\Export\NativeArray\NativeArray.cs:113
at Unity.Collections.NativeArray`1[T].get_Item (System.Int32 index) [0x00003] in C:\buildslave\unity\build\Runtime\Export\NativeArray\NativeArray.cs:134

 

thank you I will try that out !

 

Well, I can say that read and write NativeArray on multi thread is possible.
I have bunch of native arrays, which are accessed by entities in different threads.
Some are accessed write and read from different threads at same time.

Spoiler

With Burst

 

But do you access nativearrays both on unity jobs and normal c# threads at once ?

 

What you mean exactly by "normal c# threads"?

I access them in

Code (CSharp):

1. struct MyJob: IJobProcessComponentDataWithEntity

For example read write

Code (CSharp):

1. [NativeDisableParallelForRestriction]
2. public NativeArray <int> naReadWrite ;
3.  
4. [ReadOnly]
5. public NativeArray <int> naJustRead ;

 

I am talking of System.Threading.Tasks

Before adapting my system to Unity.Jobs / ECS etc. I was using a persistent c# background thread for heavy work loads, which will be sent to mainthread when its done via concurrent queue. You can't do that with unity jobs, as you are forced to complete them each frame, or am I wrong ?

I understand that unity's nativecontainer collection (despite nativehashmap) is not allowed to be multithreaded except mainthread + unity job threads.

But as I didn't get any error message (please read above) I thought maybe it works, but as it seems the error handling is buggy, maybe, when using c# threads (non unity job threads).

 

Ah. then I am pass
I used them in past very briefly. But my knowledge is near null.

 

OK solved! I just realized I CAN create persistent IJobs. I didn't know that!

I thought that you are forced waiting for completion each frame but thats wrong...

 

yes, you can check with isComplete each frame and when your data is ready then you need to call Complete, this way it will not use the mainthread at all.

 

Im glad you found a way how to solve your problem.

I would like to ask, however, again about the original question: How can I read from NativeArrays in a C# task thread?

Lets say I have a code where I use persistent NativeArrays in my application instead of managed C# arrays (a heavy math simulation on voxels where I swap pointers to read and write arrays to avoid copying data between managed C# arrays and NativeArrays after each step of simulation).

Now during this simulation I have a "maintenance" window where i want to run some code in parallel but i dont know upfront which of the NativeArrays will i need to access (therefore i cant use Unity jobs) but i know i will only need to read from them so the C# tasks seem like an ideal way.

However, once i read from the NativeArray in a task i get:

Code (CSharp):

1. System.InvalidOperationException: You are not allowed to access a NativeArray outside the main thread and job threads.

Will this be supported in the future or is there any way how to disable this check?
Thanks

 

As julian-moschuering already pointed out, it could and most probably just is a bug / missing feature that unity is only capturing you accessing the nativearray from another thread (aside main- and job-thead), when the mainthread or a job thread is accessing it too.

As the error message states you are not allowed to do that, so I would not recommend doing it at all.

A work around for you could be the NativeHashMap, which is an exception and may be multithreaded.

But I do not understand, why can't you just use a unity job, as I do, too ? You can treat is as a normal c# task.

 

I cant use a unity job because each time i need to access a variable amount of native arrays in the job.

Something similar to this question https://forum.unity.com/threads/native-array-of-native-arrays.541648/, the guy it working in an extremely hacky way .

I was just curious if there is a nicer way since I know I will be only reading from those arrays. Or a way to disable the checks on nativearray so i can read from it in a C# task thread.

 

I had this problem too, and sovled it by storing data in one large linear nativearray rather than in multiple nativearray chunks, and providing an index offset instead of an array ID. Maybe thats applicable for you too.

 

I feel bad bumping an old thread, but is it solved though?

@jmedveckyh1 did you happen to solve this nicely? Maybe with some attribute?

 

Nativearrays are really meant to be used together with jobs and burst, that's where you get the most out of them, performance and security wise.

Afaik nativearrays are even slower than managed arrays if not used in jobs with burst: https://forum.unity.com/threads/job-system-not-as-fast-as-mine-why-not.518374/#post-3397587

(Best usecase for nativearrays is IL2CPP + jobs + burst)

I guess if you have to use classic c# threads for whatever reason, you should stick to managed arrays anyways. I can imagine it's annoying if you are in the midst of it, but in my experience you shouldn't and even can't mix it up (I think nativehashmaps are an exception, but not sure if this is still the case)

What is your usecase in general, if you want to explain ?

 

Is that still the case? If that is, how is that possible?

 

Yes. I do remember unity once said in a forum thread that they indeed want to support async jobs, meaning they run over multiple frames. They also removed the warning (when a job runs longer then 3 frames), so it's fine.

Doing it is simple, you just don't call .Complete() each frame, instead you just call it when the job is done (e.g. kinda job queue count == 0).

Keep in mind that it can lead to error messages if you write to the same nativearray from another job, because you didn't call Complete() on the async job, which is important due to thread saftey.

 

Thank you very much!

 

sorry mistype, "meaning they run over multiple threads", I meant to say "meaning they run over multiple frames.". edited.