ref returns in native collections?

Is there a reason that native collections (array and list) don't return refrences to their elements instead of copies?
Also I wonder the same about APIs like GetComponent as well? Is it because of safety guarantees? I don't see exactly how

 

I believe there are issues with ref returns on structs as it would be possible for the member reference to live longer than the struct itself which is bad from a type safety perspective. If you try this yourself, you get the following error:

Struct members cannot return 'this' or other instance members by reference [Assembly-CSharp]csharp(CS8170)

If NativeArray was a class then this would work today.

Here's an example of problematic code if ref returns on structs was allowed:

Code (CSharp):

1. struct Container  {
2.     public Container(int value) {
3.         this.value = value;
4.     }
5.     private int value;
6.     public ref int Value { get {return ref value; } }
7. }
8.  
9. class TestClass {
10.     public ref int getValueRef() {
11.         Container test = new Container(10);
12.        
13.         // here we are returning a pointer to memory inside of the local variable 'test', but test will no longer exist after returning from the function so we are left with a dangling pointer.
14.         return ref test.Value;
15.     }
16.  
17.     public void Foo() {
18.         getValueRef() = 20;
19.     }
20. }

More info: https://github.com/dotnet/csharplang/blob/master/meetings/2015/LDM-2015-09-01.md#this-in-structs

 

If you really do like the syntax, you can always extend it yourself. Unity exposes some unsafe helper functions that makes it possible to quite easily fetch elements as ref. You can always do it like this:

Code (CSharp):

1.  
2. public static class NativeArrayRefExtensions
3. {
4.     public static ref T GetRef<T>(this NativeArray<T> array, int index)
5.         where T : struct
6.     {
7.         // You might want to validate the index first, as the unsafe method won't do that.
8.         if (index < 0 || index >= array.Length)
9.             throw new ArgumentOutOfRangeException(nameof(index));
10.         unsafe
11.         {
12.             return ref UnsafeUtility.ArrayElementAsRef<T>(array.GetUnsafePtr(), index);
13.         }
14.     }
15. }

And then you can use the array like this:

Code (CSharp):

1. NativeArray<Translation> translations = new NativeArray<Translation>(1, Allocator.Temp);
2. ref Translation translation = ref translations.GetRef(0);

Disclaimer: I just threw this together without actually testing it, but I'm fairly certain it should work. If anyone has any comments on potential problems with such a solution, I'm happy to hear them.

 

@Zec_ Ah thanks I did not know about the unsafe method there. Highly appreciated

Are you aware of any method to get component by reference other than using chunks/foreach? for example some sort of GetComponent which gets it by reference?

 

Sorry, haven't investigated that deeply into the subject. I think they aren't exposing any such functions as the data (components) for entities move around in memory when destroying entities, potentially invalidating existing memory references. I'm quite certain they use the SwapBack-technique. This means that whenever an entity in the middle of an archetypechunk is destroyed, they take the data for the last entity in the chunk and move it into the now vacant memory slot.

Take this as an hypothetical example, if such a method that you propose would exist.

Code (CSharp):

1. protected override void OnUpdate()
2. {
3.     // Imagine that these two entities exist in the same ArchetypeChunk
4.     Entity lastEntityInArchetypeChunk;
5.     Entity anotherEntityInArchetypeChunk;
6.  
7.     // This function doesn't exist, but lets play with the thought that it did. It would return a memory reference to the location in memory where the entity component exists.
8.     ref var translation = EntityManager.GetComponentAsRef<Translation>(lastEntityInArchetype);
9.  
10.     // This following destroy-call will swap back the data for the final entity to the now empty entity data location.
11.     // After this call, the translation reference fetched above will still point to the memory location where the lastEntityInArchetypeChunk used to exist. A memory location that is now an unused entity memory slot.
12.     EntityManager.DestroyEntity(anotherEntityInArchetypeChunk);
13. }

 

@Zec_ Yes I understand but theoratically at least you could get the ref and set component fields before executing your commandbuffer which might introduce structural changes.

@Joachim_Ante Thanks, Yes I see what you are saying and most cases will be solved with the current stuff and without copies but for example in my AI system , entities have a list of potential entities which can be considered as targets and I need to write to their components once in a while and pass by reference could introduce some speedups in the process. I'm sure you are aware of the usecases anyways

 

Hi,
I am really strugling to understand the Unsafelists. I crated some mesh data in different jobs with the same type but different sizes. Assigning these mesh data to mesh in a for loop is very cpu expensive.
So my intention is to combine these arrays in a parallel or seperated jobs into one big unsafe list. There is no error and the referance of the biglist in jobs seems filled with the data. But when i try to get the data from main thread it seems empty. Is there any reason for this???

Here is my code sample (simplified):

Code (CSharp):

1.  
2. public struct SegData
3. {
4.     public float CreationTime;
5.     public bool ConnectedToNextSeg;
6.      public Vector3 v1, v2, v3, v4;
7.     public Vector3 uv1, uv2, uv3, uv4;
8.     public Vector3 n1, n2, n3, n4;
9.     public Color c1, c2, c3, c4;
10. }
11. public class SkidJobsystemV2_With_Nativearrays_Base : MonoBehaviour
12. {
13.     UnsafeList<SegData> CombinedSdList = new UnsafeList<SegData>();
14.     List<NativeArray<SegData>> SegDataList = new List<NativeArray<SegData>>();
15.  
16.     WheelCollider[] wcs;
17.     WheelSkidTag[] TaggedWheels;
18.  
19.     void Start()
20.     {
21.         for (int i = 0; i < TaggedWheels.Length; i++)
22.         {
23.             NativeArray<SegData> SegData = new NativeArray<SegData>(MaxSegmentCount, Allocator.Persistent);
24.             SegDataList.Add(SegData);
25.         }
26.         CombinedSdList = new UnsafeList<SegData>(1000, Allocator.Persistent);
27.     }
28.  
29.     private void OnDestroy()
30.     {
31.         for (int i = 0; i < wcs.Length; i++)
32.         {
33.             SegDataList[i].Dispose();
34.         }
35.         CombinedSdList.Dispose();
36.     }
37.  
38.     [BurstCompile(CompileSynchronously = true)]
39.     void Update()
40.     {
41.  
42.         /* In this section a job list fills the SegDataList. */
43.  
44.         var CombinedDependencies = JobHandle.CombineDependencies(jhs);
45.  
46.         //Here is combining Job Section
47.         CombinedSdList.Clear();
48.         NativeList<JobHandle> CombineArraysJobHandles = new NativeList<JobHandle>(wcs.Length, Allocator.TempJob);
49.         for (int i = 0; i < wcs.Length; i++)
50.         {
51.             CombineArrays ca = new CombineArrays
52.             {
53.                 SegDatas = SegDataList[i],
54.                 wcIndex = i,
55.                 CurrentIndex = CurrentIndexList[i],
56.                 CombinedSdList = CombinedSdList
57.             };
58.             CombineArraysJobHandles.Add(ca.Schedule(CombinedDependencies));
59.         }
60.         JobHandle.CompleteAll(CombineArraysJobHandles);
61.         CombineArraysJobHandles.Dispose();
62.         //CombinedSdList is empty even in job it seems filled.
63.     }
64.  
65.     [BurstCompile(CompileSynchronously = true)]
66.     public struct CombineArrays : IJob
67.     {
68.         public NativeArray<SegData> SegDatas;
69.         public int wcIndex;
70.         public NativeArray<int> CurrentIndex;
71.         public UnsafeList<SegData> CombinedSdList;
72.         public void Execute()
73.         {
74.             for (int i = 0; i < CurrentIndex[0]; i++)
75.             {
76.                 SegData sd = new SegData();
77.                 sd.v1 = new Vector3(1, 2, 3);
78.                 CombinedSdList.Add(sd);
79.             }
80.  
81.             //When debugged here ı see that CombinedSdList is filled. But when i try to get data in main thread it is empty
82.         }
83.     }
84. }
85.  

 

UnsafeList is a value type which stores the buffer pointer and length directly as fields, meaning that changes made to those fields in a job don't get copied out. In constrast, NativeList has a pointer to an UnsafeList so it is merely copying a reference to the UnsafeList into a job.

 

local ref variable working only in methods and are short lived You also can't use multithreading with ref variables (ref/in/out arguments with C# async/await). In fact, it should rarely happen that a locale reference shows a deallocated memory.
As long as the deallocated of the entity/component is always between the frames, there should be no problem.
A scenario would be a reference from a native array, other other sources, that is used in a long-living thread in an infinite loop and the source is deallocated.