Unmanaged variants of native collections needed (because of Burst, ref not allowed)

Hi! I've just hit the wall with native collections.

I want to (sort-of) mimic virtual function calls (because of divide-et-impera splitting the code into smaller number of classes) however, those function calls need to operate over big structs of mine. Naturally, I do not want to copy the whole struct when invoking my functions.

Usually, one would use "ref" keyword, i.e., passing that struct to the function via reference. However, that's the point where Burst steps in and fails to compile such a function.

Ok, we can switch to unsafe and pointers, should work right? Well yes, as long as you work with "unmanaged" types only. However, if your struct contains, e.g., NativeArray<int>, it throws your struct into the "managed" park.

You can create your custom native array to circumvent this, but obviously, that's not a wise path to take.

Here is the code that illustrates the problem. If you try to compile this, it will fail with CS0208 because TestStruct is not an unmanaged type because NativeArray<int> is not an unmanaged type.

Code (CSharp):

1.    
2.     public struct TestStruct
3.     {
4.         public NativeArray<int> nativeArrayInt; // if you comment this out, the code will compile  
5.     }
6.  
7.     public unsafe static class TestManipulation
8.     {
9.         public static void DoSomething(void* data)
10.         {
11.  
12.         }
13.  
14.     }
15.  
16.     public unsafe struct TestUser
17.     {
18.         public TestStruct data;
19.  
20.         public void DoIt()
21.         {
22.             fixed (TestStruct* ptr = &data) // COMPILER ERROR CS0208 because TestStruct is managed
23.             {
24.                 TestManipulation.DoSomething(ptr);
25.             }
26.         }
27.     }
28.  

 

Have you tried using unsafe pointer for the native array struct field instead?

Just in case, NativeArray is already a wrapper for the unmanaged buffer of memory.
Which is pretty much looks like this:

Code (CSharp):

1. array.m_Buffer = UnsafeUtility.Malloc(size, UnsafeUtility.AlignOf<T>(), allocator);

So you can directly access m_Buffer by fetching pointer with GetUnsafePtr / GetUnsafeReadOnlyPtr.

Also, there's UnsafeList available, if you need array-alike unsafe container.

 

You can get a pointer to the buffer of a NativeArray without requiring pinning. Look at the GetUnsafePtr and GetUnsafeReadOnlyPtr methods.

 

Thanks for replies! Are you suggesting to stop using NativeArray completely?

 

If your intention to pass it via unsafe struct in an unsafe context - yes.
Its extremely useful in any other case.

By the way, burst should compile just fine if you store NativeArray in TestUser instead, and skip unsafe / pointer part.
Native container's data pointers are copied, so they do always act as a "ref".

 

I'm aware of pointer to m_buffer, but that feels like circumventing the Unity collection design. Wrt. to the design, my rant is that Burst and jobs are about unmanaged stuff, yet NativeArray (counterintuitively) isn't an unmanaged struct.

Not using NativeArray at all would require quite a refactoring in my case, already having a lot of code using NativeArrays while also completely abandoning Unity collections. Whereas simple regexp over my files switching NativeArray with hypothetical NativeArrayUnmanaged would do the trick.

About passing structs around, there is always a performance penalty hit in contrast to just passing pointers, even more as I have multiple NativeArray instances within my structs.

Finally, NativeArray inside TestStruct cannot be moved into TestUser as TestManipulation is meant there to provide virtual call dispatch delegating "DoSomething" to someone else allowing to organize the code base in more managable way. I mean, whole point of this is not to have all TestStruct manipulation code (which can be plenty) within TestUser.

 

Burst and Jobs are designed to work with safe context. Whether that is MonoBehaviour approach, or Entities.
So it makes zero sense to require unsafe context on assemblies that should not have it.

If you want, you can plain copy & paste code from NativeArray, and mark it with unmanaged
Alternatively, there's UnsafeList:
https://docs.unity3d.com/Packages/c...y.Collections.LowLevel.Unsafe.UnsafeList.html

Or, you can create your own native container (link contains how to):
https://docs.unity3d.com/ScriptRefe...LowLevel.Unsafe.NativeContainerAttribute.html

I think real issue here that you're trying to "hack" around DOD instead of using it.
There's probably a better way to avoid all of this. Why do you need virtuals in the first place?

 

Oh, I missed that UnsafeList, that might do the trick.

Yes, I'm hacking around DOD because I just need to harness job threads and Burst. I'm working on high-fidelity AI and DOD style proved too difficult for that plus I need NavMesh at hand (reimplemented that one) and A* over NavMesh violates DOD anyway.

 

Uhhhh, not really?
https://forum.unity.com/threads/open-source-planar-navmesh-built-on-dots.1060805/

There's also an AI Planner package available for DOD AI:
https://docs.unity3d.com/Packages/com.unity.ai.planner@0.3/manual/index.html

 

Thanks for sharing! Missed those! My solution is 2yr. old now integrated deep in the code, but glad that someone picked this up + making this open source, impressive! I will have to check that as A* over NavMesh always means jumping around the memory (you just cannot map 2D into 1D so you always get sequential reads during searching).

Interesting as well! Though I do not require full planning.