How to set/get NativeMultiHashMap<int, NativeArray<Vector3>> ?

Hello,

I am struggling with a way to convert a Dictionary<int, Vector3[]> into its ECS equivalent.
I am been looking for hours (a whole day actually) on forums and found those ones:

1. I tried to follow Raph_Wa code as seen further below : Unity job system and multi dimensional arrays
2. Iterating NativeMultiHashMap
3. Hybrid native container : NativeHashMapList <TKey, TValue>
4. NativeHashMap

But none seem to work for me, there are no clear example about how to use NativeMultiHashMap (Get & Set). I am currently using Unity 2020.1.11f1 with

• Entities 0.16.0-preview.21
• Jobs 0.7.0-preview.17

Here is the use case:
I define a planar mesh by xColumns and zRows at y=0.
For each xColumn I store controlPoints (used for modifying the rows afterwards).
So I thought about using Dictionary<int, Vector3[]> controlPoints then use xColumns for the keys to retrieve the control points I need at xColumn.

Moving to ECS, I tried this:

Code (CSharp):

1. public class TestNativeMultiHashMap: MonoBehaviour
2. NativeMultiHashMap<int, NativeArray<Vector3>> controlPoints;
3. int xColumns = 10;
4. int zRows = 100;
5.  
6. void Start()
7. {
8.   controlPoints = new NativeMultiHashMap<int, NativeArray<Vector3>>(xColumns, Allocator.Persistent);
9.  
10.   for (int x = 0; x < xColumns; p++)
11.   {
12.     NativeArray<Vector3> ctrlPts = new NativeArray<Vector3>(4, Allocation.Persistant);
13.     ctrlPts[0] = new Vector(0,0,-1)
14.     ctrlPts[1] = new Vector(0,0,-.2)
15.     ctrlPts[2] = new Vector(0,0,.2)
16.     ctrlPts[3] = new Vector(0,0,1)
17.  
18.     controlPoints.SetValue(ctrlPts, new NativeMultiHashMapIterator<int>());
19. }
20.  
21. //some code lines later to retrieve control points value
22. void SomeMethod()
23. {
24.     NativeArray<Vector3> ctrlPts;
25.     controlPoints.TryGetFirstValue(p, out ctrlPts, out _);
26.     //doing stuff with ctrlPts...
27.   }
28. }

But the line 8 above Editor throws me the following error at start:

Obviously I can't use controlPoints in a job...
Thank you very much for your help

 

Do you always have 4 control points? If so why do you need an array to store them? Just use a struct with 4 values

 

It is only for example, I have currently 7 points in my code, but I can add more control points for a better accuracy of the mesh modification.

 

Would a 2D array be sufficient? If so, there are indexing algorithms that let you treat a 1D array as a 2D array.

 

You could use a NativeMultiHashMap<int, int> where each value is an index into a NativeArray<Vector3> (or NativeList if you don't know the maximal amount of values ahead of time).

Code (CSharp):

1. points  = new NativeArray<Vector3>(xColumns * 4, Allocator.Persistent);
2. indices = new NativeMultiHashMap<int, int>(xColumns, Allocator.Persistent);
3.  
4. for (int x = 0, i = 0; x < xColumns; x++)
5. {
6.     indices.Add(x, i);
7.     points[i++] = new Vector3(0, 0, -1);
8.     indices.Add(x, i);
9.     points[i++] = new Vector3(0, 0, -.2f);
10.     indices.Add(x, i);
11.     points[i++] = new Vector3(0, 0, .2f);
12.     indices.Add(x, i);
13.     points[i++] = new Vector3(0, 0, 1);
14. }

 

Is it some sort of a curve?
https://forum.unity.com/threads/a-fast-blobcurve.985941/#post-6415941

 

The reason you're getting this error is you're trying to put a native collection (in this case your NativeArray<Vector3>) inside of a native collection (in this case your NativeMultiHashMap). As the error message states, you can't put a native collection inside another native collection. This is documented somewhere, but I looked around quickly and couldn't find where. It's definitely written down somewhere though.

The good news is, you don't need to put the NativeArray inside the NativeMultiHashMap to store multiple Vector3s for each int index. The NativeMultiHashMap already provides this functionality. I've rewritten your example code to take advantage of this:

Code (CSharp):

1. public class TestNativeMultiHashMap: MonoBehaviour
2. {
3.     NativeMultiHashMap<int, Vector3> controlPoints;
4.     int xColumns = 10;
5.     int zRows = 100;
6.     void Start()
7.     {
8.         // just declare the NativeMultiHashMap as an <int, Vector3>
9.         controlPoints = new NativeMultiHashMap<int, Vector3>(xColumns, Allocator.Persistent);
10.    
11.         for (int x = 0; x < xColumns; p++)
12.         {
13.             // just add values for the key, calling add multiple times inserts additional values for that key
14.             controlPoints.Add (x, new Vector(0,0,-1.0f));
15.             controlPoints.Add (x, new Vector(0,0,-0.2f));
16.             controlPoints.Add (x, new Vector(0,0, 0.2f));
17.             controlPoints.Add (x, new Vector(0,0, 1.0f));
18.         }
19.     }
20.     //some code lines later to retrieve control points value
21.     void SomeMethod(int x)
22.     {
23.         // you can iterate over all of the values stored for a given key by using
24.         // the GetValuesForKey() method, which returns an Enumerator for your values
25.         // stored for the given key
26.         foreach (var controlPoint in controlPoints.GetValuesForKey(x))
27.         {
28.             // do something to the control point
29.         }
30.     }
31. }

I haven't tested the code written above (since I was just modifying the code you posted), but I have tested that declaring a NativeMultiHashMap<int, Vector3>, adding multiple values for a key, and then retrieving them using GetValuesForKey() to retrieve them works. Here's an excerpt from that test code:

Code (CSharp):

1.            
2.             var ctlPoints = new NativeMultiHashMap<int, Vector3> (8, Allocator.Temp);
3.             ctlPoints.Add (0, new Vector3 (0, 0, -1));
4.             ctlPoints.Add (0, new Vector3 (0, 0, -2));
5.             ctlPoints.Add (0, new Vector3 (0, 0, +2));
6.             ctlPoints.Add (0, new Vector3 (0, 0, +1));
7.             foreach (var value in ctlPoints.GetValuesForKey(0)) {
8.                         UnityEngine.Debug.LogFormat ("{0}", value);
9.             }
10.             ctlPoints.Dispose ();
11.  

One quick note about using a NativeMultiHashMap, I'm not sure if the values stored for each key are stored as an ordered collection of values or an unordered collection of values. What I mean by this is I don't know whether you are always guaranteed to get the values out in the same order as you put them in. The documentation states that NativeMultiHashMap is an "Unordered associative array, a collection of keys and values. This container can store multiple values for every key". The "unordered associative array" bit is a description of a HashMap, but it is unclear to me whether the collection of values for a given key is ordererd or not.

Perhaps someone from Unity can clarify?

 

No they are not. They may look like, or be in reverse orther, but dont get fooled.
Don't try to relay on the order from dictionary or hash maps.

Consider situation, if you write to hasmaps in multithreaded jobs.
You won't be able to specify an order, how things are written into.
And the general nature of hash collection.

 

To elaborate on this for my own use case:
I have a single massive NativeArray<int> of every path pre-calculated for my pathfinder from node to node. Since this also gets updated as more paths are added, I use a separate NativeArray<int2> which helps to index the larger one, of which the length of the array is the amount of nodes, the first value is the index of the node to connect to, and the last value the length of the path in steps. The reason it's all Arrays is that the amount of nodes remain the same.

 

I wish I could keep the order of the values in the NativeMultiHashMap, so it won't work as expected this specific case as my control points are ordered along the xColumn setting the zRows. DK_A5B 's solution looks great but not the best, at least this helps me to understand that NativeContainer.

TheZombieKiller 's solution is basically a 1D array converted from 4D one if I understand well. I think I tried already but I am not sure it was performant enough, I will try again. So for 10 xColumns with 7 ControlPoints each, I have to pass the NativeArray of 70 elements just to find 7 out of them...I don't know what way is the best solution, performance wise...

 

StarChick971, what you're looking for is to have a 2D array of 3D points (i.e. Vector3s), is that correct? That is, you want something like this:

Code (CSharp):

1. [  /// COLUMN_0          COLUMN_1             COLUMN_2
2.    [new Vector3 (),     new Vector3 (),     new Vector3 ()],  // ROW_0
3.    [new Vector3 (),     new Vector3 (),     new Vector3 ()],  // ROW_1
4.    [new Vector3 (),     new Vector3 (),     new Vector3 ()],  // ROW_2
5.    [new Vector3 (),     new Vector3 (),     new Vector3 ()]   // ROW_3
6. ];

That would be a 3 column x 4 row 2D array.

Assuming that my understand is correct, my next question is - do all of your rows have the same number of columns? Or do your rows have an arbitrary number of columns?

 

That's exact!
Actually, my target goal is to have lots of rows and fewer variable columns set as startup, like 10 columns with 100 rows for each column...

 

I'm not sure if it fits your problem exactly but you should take a look at https://docs.unity3d.com/Packages/com.unity.collections@0.14/api/Unity.Collections.NativeStream.html

It's fast and deterministic but not suitable if you need lots of random access

 

If there is always going to be four Vector elements per key, you could try creating a struct which contains four Vector fields and store that instead.

 

So, if you have a 2D array where you all of the rows have the same number of columns, this is pretty straightforward. You can do this:

Code (CSharp):

1. // a method demonstrating how to access an
2. public static float3 GetControlPoint (NativeArray<float3> controlPoints, int columnIndex, int rowIndex, int numOfCols) {
3.     var controlPoint = controlPoints[columnIndex + rowIndex * numOfCols];
4.     return controlPoint;
5. }
6.  
7. // setup your array
8. int NUM_OF_COLS = 10;
9. int NUM_OF_ROWS = 100;
10. var controlPoints = new NativeArray<float3> (NUM_OF_COLS * NUM_OF_ROWS, Allocator.TempJob);        // using float3 instead of Vector3, because as long as you're using ECS, you might as well use the com.unity.mathematics package.
11.  
12. // populate your array
13. for (int y = 0; y < NUM_OF_ROWS; y++) {
14.     for (int x = 0; x < NUM_OF_COLS; x++) {
15.         controlPoints[x + y * NUM_OF_COLS] = new float3 (0, 1, 2);
16.     }
17. }
18.  
19. // get the control point stored in the 7th column of the 42nd row
20. var controlPoint = GetControlPoint (controlPoints, 6, 41, NUM_OF_COLS);

 

Actually, I have tried this right before your anwser, and I am glad you have confirmed that logic. I think I will stay with this.

Very interesting too, I take a note about NativeStream. It could be useful, thanks!

Of course not, there could be much more elements per column at the end, it was just an example.

Yeah, chained Bezier curves indeed.