Proper way of handling shared data between systems

At first glance, this is going to sound like a question asked time and time again, but I've got a few things that I just need cleared up that I haven't found clarified in previous threads

I am trying to make a hexed based grid with an arbitrary (and potentially "infinite") number of points. What I'm struggling with is how to store the data and access it. In my earliest tests before I was going all in with the ECS, I made a data map dictionary that was accessed by cubic coordinate. However, this didn't seem the proper way of doing things and I was getting data access errors.

I went out to find a more ECS friendly way of doing it. The closest I've found to how to do this in a reasonable manner is here. I think the native array shared values is the way to go maybe, but I am a little concerned about resizing it later on. If, for example, I move off in the negative direction along the x/y/z axis, I'm going to need to resize things which sounds not very ideal.

I could also create 6 arrays, one for each hextant and index it based on that as it cones out on one of the 6 directions. This also seems like an ok solution for once I need to store this in a file for save data later. It would be something like this:


I know I'm not supposed to worry about optimization until it becomes a problem, but since I'm treating this as much as a learning exercise as anything, I really am trying to figure out the best way to handle the ECS instead of just hacking things together and calling it a wrap.

I guess at the end of the day, I'm struggling with finding good information for what I'm doing. It seems like a lot of examples are out of date, or only half of what I need. I used the github example and it got me this far, but I'm sort of in need of a nudge to get over this next step.

 

If you're not looking for deeper LOD you could try something like this (untested):

Spoiler

Code (CSharp):

1. public struct Hex
2. {
3.     public float3 A, B, C, D, E, F;
4.  
5.     public float3 this[ int index ]
6.     {
7.         get {
8.             switch ( index )
9.             {
10.                 case 0: return A;
11.                 case 1: return B;
12.                 case 2: return C;
13.                 case 3: return D;
14.                 case 4: return E;
15.                 case 5: return F;
16.                 default: throw new IndexOutOfRangeException();
17.             }
18.         }
19.         set
20.         {
21.             switch ( index)
22.             {
23.                 case 0: A = value; break;
24.                 case 1: B = value; break;
25.                 case 2: C = value; break;
26.                 case 3: D = value; break;
27.                 case 4: E = value; break;
28.                 case 5: F = value; break;
29.                 default: throw new IndexOutOfRangeException();
30.             }
31.         }
32.     }
33. }
34.  
35. public struct HexTriangle
36. {
37.     public Hex A, B, C, D, E, F, G, H, I, J, K, L, M, N, O;
38.     public bool pointIsUp;
39.     public Hex this[ int index ]
40.     {
41.         get {
42.             switch ( index )
43.             {
44.                 case 0: return A;
45.                 case 1: return B;
46.                 case 2: return C;
47.                 case 3: return D;
48.                 case 4: return E;
49.                 case 5: return F;
50.                 case 6: return G;
51.                 case 7: return H;
52.                 case 8: return I;
53.                 case 9: return J;
54.                 case 10: return K;
55.                 case 11: return L;
56.                 case 12: return M;
57.                 case 13: return N;
58.                 case 14: return O;
59.                 default: throw new IndexOutOfRangeException();
60.             }
61.         }
62.         set
63.         {
64.             switch ( index)
65.             {
66.                 case 0:  A = value; break;
67.                 case 1:  B = value; break;
68.                 case 2:  C = value; break;
69.                 case 3:  D = value; break;
70.                 case 4:  E = value; break;
71.                 case 5:  F = value; break;
72.                 case 6:  G = value; break;
73.                 case 7:  H = value; break;
74.                 case 8:  I = value; break;
75.                 case 9:  J = value; break;
76.                 case 10:  K = value; break;
77.                 case 11:  L = value; break;
78.                 case 12:  M = value; break;
79.                 case 13:  N = value; break;
80.                 case 14:  O = value; break;
81.                 default: throw new IndexOutOfRangeException();
82.             }
83.         }
84.     }
85. }
86.  
87. // add to world Entity or Chunk Entity
88. public struct SuperHex : IBufferElementdata
89. {
90.     public Hex center;
91.     public HexTriangle A, B, C, D, E, F;
92.  
93.     public HexTriangle this[ int index ]
94.     {
95.         get {
96.             switch ( index )
97.             {
98.                 case 0: return A;
99.                 case 1: return B;
100.                 case 2: return C;
101.                 case 3: return D;
102.                 case 4: return E;
103.                 case 5: return F;
104.                 default: throw new IndexOutOfRangeException();
105.             }
106.         }
107.         set
108.         {
109.             switch ( index)
110.             {
111.                 case 0: A = value; break;
112.                 case 1: B = value; break;
113.                 case 2: C = value; break;
114.                 case 3: D = value; break;
115.                 case 4: E = value; break;
116.                 case 5: F = value; break;
117.                 default: throw new IndexOutOfRangeException();
118.             }
119.         }
120.     }
121. }

 

If you don't need Hex point data often, reduce the memory size like so and generate point data when needed:

Code (CSharp):

1. public struct Hex
2. {
3.     public float3 center;
4.     public float radius;
5. }

Presuming you'd like spatial testing, you could use a quadtree or check the distance for each SuperHex, or Hex. Also could add a bounding triangle for your HexTriangle (point inside triangle). Since this is a blittable structure, you should be able to use the job system with burst and get some reasonable performance.

Don't take my word on any of this, I'm just a hobbyist.

 

Long term, a lot of this is likely the direction I'll need to head in, though my super hexes will likely be bigger. The example picture I have is a small sample from the eventual size of everything. I already have a lot of the math in place for detecting grid coordinates/centers/etc so that should come together cleanly with what you have suggested.

I'm still scratching my head though, as to how to do the Systems side of things (accessing data across systems). I'm messing around with Native Arrays and Native Array Shared Values, hopeful that I'll actually get my code working sometime in the next couple hours.

Do you know if nested Native arrays are a possibility?

 

We can't nest native collections unfortunately.

If you already have a lot of the math done, might be able to get away with dynamic buffer of hex structs per chunk. Systems could then query the chunks entities. Increasing or decreasing LOD level should be pretty simple then, just compute the hex values and replace the existing buffer (careful with chunk capacity). If you need to save your LOD levels instead of procedural generating them, you could look into BlobAsset. I haven't worked with BlobAsset to/from disc yet.

 

Ok, I think this is making a little more sense.

This was the route that I was going with, but it seems that I can't get the native array shared values to work with anything but int arrays. I was trying to make it so that I could convert my HexData to an int so it would understand the mapping, but I guess that isn't a thing?

Spoiler

Code (CSharp):

1.  
2.     public struct HexData : IComparable
3.     {
4.         public CubicHex Location;
5.         //Some other information about hex walls and stuff
6.  
7.         public static implicit operator int (HexData d) => HexUtils.Index (d.Location);
8.         public int CompareTo (object obj)
9.         {
10.             return ((int) this).CompareTo ((int) obj);
11.         }
12.     }
13.  
14.     [WorldSystemFilter (WorldSystemFilterFlags.Default | WorldSystemFilterFlags.EntitySceneOptimizations)]
15.     public class HexGrid : JobComponentSystem
16.     {
17.         NativeArray<HexData> data;
18.         NativeArraySharedValues<int> keys;
19.  
20.         protected override void OnCreate ()
21.         {
22.             Entities.ForEach ((Entity entity, ref HexGridGenerator gridGenerator) =>
23.             {
24.                 int bufferSize = gridGenerator.ColRange * gridGenerator.RowRange;
25.                 //int[] indexArr = new int[bufferSize];
26.  
27.                 data = new NativeArray<HexData> (bufferSize, Allocator.TempJob);
28.  
29.                 int position = 0;
30.                 //TODO: this is where we'll read data from a file or generator.
31.                 for (int col = gridGenerator.minCol; col <= gridGenerator.maxCol; col++)
32.                 {
33.                     for (int row = gridGenerator.minRow; row <= gridGenerator.maxRow; row++)
34.                     {
35.                         OffsetHexCoord coord = OffsetHexCoord.Of (col, row);
36.                         CubicHex cubic = CubicHex.Of (coord);
37.  
38.                         data[position] = new HexData ()
39.                         {
40.                             Location = cubic
41.                         };
42.                         position++;
43.                     }
44.                 }
45.  
46.                 keys = new NativeArraySharedValues<HexData> (data, Allocator.Temp);
47.             });
48.         }
49.     }
50.  
51.    public struct HexGridGenerator : IComponentData
52.     {
53.         //TODO: here we can define parameters for map generation.
54.         //This is placeholder test stuff.
55.         public int minCol, maxCol, minRow, maxRow;
56.  
57.         public int ColRange => Math.Abs (maxCol - minCol) + 1;
58.         public int RowRange => Math.Abs (maxRow - minRow) + 1;
59.     }
60.  

Regardless, if I committed to your data structure suggestion, I would be able to have a queryable system without using the NativeArraySharedValues at all, instead just an array of super hexes. At that point, it's not too bad to loop through each super hex to decide if it contains the point I'm looking for. Is that how this would work then?

 

My original example shows how to get around the component collection / nested collection restrictions if the size and layers are pre-determined. But for your use case this option might be a dev trap down the road.

If you only want one LOD loaded at a time, just use a single buffer on the chunk component itself. A system can be responsible for procedurally generating or loading from file the next LOD values when the time is right.

Alternatively, you could spread your data across multiple entities. If you want all LODs loaded at the same time, your chunk entity could have a buffer where each item is a LOD entity reference. The reference points to an entity with a LOD specific hex, triangle buffers / components.

If you want all LODs in a 'nested' LOD format, use the same entity ref buffer concept as previously mentioned like a linked list. The higher LOD has a buffer of respective lower LOD entities. Might be useful for a LOD to keep a reference to it's 'parent' LOD as well.

If you want all LOD and you never need to modify your hex data ever, mash all LODs into a single blob asset, and your chunk component keeps a reference.

 

A hex grid is just a square grid with odd lines shifted half a distance
https://imgur.com/a/WECwATe

You can store data in a hex grid the very same way you would in a square grid

 

Ok, this looks like the best route to take for my use case. It seems like, if performance becomes an issue, moving it into the nested design should be relatively trivial, but I'll resist the urge to overcomplicate from the outset.

Thank you for the suggestions.

 

Probably don't need to store triangle data since you could compute a hex's triangle by the distance and angle to the superhex origin.

 

Oh, another question. Right now I have my native array being put upon a system that I access in other components. Should that actually be added to a IBufferElementdata and then queried instead?

 

I think accessing a system array from other systems should be okay technically -- I've seen other people do that with native queues as a simple event system. It might not be best-practice, but with DOTS most of us don't know what best-practice should look like yet. I suppose you might come across race conditions or limitations using jobs if you're not careful. I think just use a collection with the concurrent option if you want parallel read / write.

https://docs.unity3d.com/Packages/c....Collections.NativeList-1.ParallelWriter.html

 

Thanks, I think that's what I needed to hear. I've just been trying to find the *right* way of doing things, but I guess we don't really know yet.

 

I typically have a few possible solutions in mind while unsure what will work best, so I create a quick prototype of each solution and then stick with the one that performs the best or is easiest for me if the performance differences aren't too harsh. This makes development slow, but I expect this since Unity DOTS is pretty new to me.

Rapidly adding and removing components isn't the most performant option at scale (this may change), especially when an entity has a lot of components. But it's much easier for me to code and understand, so I stick with it. To cut down on component count I sometimes split an entity into multiple and relate them by reference.
.

 

You could check out the Shared Statics from burst documentation, it basically provides you a "global" variable that can be read/write from a bursted job, You could put NativeMultiHashMap in there and create your hex grid from mono, and then do read/write operations from your job. So i am guessing your NativeMultiHash could be like NativeMultiHashMap<int, Tile> for grouping your tiles.

 

Shared Statics looks very useful, but also a little confusing at first glance. Thank you for sharing.