Allocating NativeHashMap is painfully slow (obviously). Any Recommendations for my situation?

Okay so I have been trying to solve this issue for a very long time. Basically, I have a task that I want perform, but it requires that one job knows the length of a NativeHashMap before it is scheduled.

Here is a synopsis of what I am doing:

Basically, I have an entire world of sections of voxel terrain. Each section is called a "chunk" (confusing in ECS context, but that is what it is called in voxel context, so imma still use this terminology). For each chunk, I do the following to create a mesh for it:

1. Generate voxel data for the "chunk" using a compute shader inside of a component system, and populate the dynamic buffer attached to the entity with this data

After this everything falls to shambles. Ideally, this is how it would work after the data generation:

1. Find all active edges and voxels within the chunk, and add them to a nativehashmap with their own unique codes

However, the next step requires that the length of this nativehashmap is known so that a vertexBuffer for a mesh can be filled. Therefore, it creates a super annoying sync point. So I honestly have no idea what to do for this situation.

1. Find all active edges and voxels within the chunk, and add them to a nativehashmap with their own unique codes
   
2. Visit each active voxel, and create a vertex that will inserted into the vertex buffer <--- huge annoying sync point required for this one
3. Create a triangle order, this also required that the length of the hashmap is known

On top of all of these issues, I have no way of deallocating the nativehashmaps without adding anther sync point since they cant be deallocated with the tag.

AND, allocating these hashmaps is painfully slow. At the most, a chunk can contain 3 times the amount of voxels it contains. That means that the length of the hashmap must be SizeX * SizeY * SizeZ * 3. Allocating this takes a painful amount of time in the editor.

So in total, here are my 3 issues (along with some possible solution that also aren't very ideal):


1. Solving the issue of needing to know the size of a nativehashmap before running the next step (job)

• Perhaps attempt to add the vertices that comprise the mesh to another hashmap?

2. Somehow saving hashmaps for another job system, or deallocating on the fly somehow

• I guess you can save hashmaps for another job system by adding a SharedComponent that contains these hashmaps to your entity. However, this just adds more possible sync points (since the first job would have to finish before reading from the hashmap in the next step)

3. Super slow allocation time

• TBH I have no idea what the hell I am doing at this point LOL. Plz help.

Here is my JobComponentSystem code so far for the first step. I still don't really know if what I am doing is correct. Please help me correct any mistakes.

Thanks guys!!

Code (CSharp):

1. using Unity.Entities;
2. using Unity.Jobs;
3. using Unity.Collections;
4. using UnityEngine;
5.  
6. using static Unity.Mathematics.math;
7.  
8. using Unity.Mathematics;
9. public class ChunkMeshingSystem : JobComponentSystem
10. {
11.     EntityQuery chunksQuery; //and entity query that will grab all the chunks currently in the world
12.     NativeArray<Entity> chunks;
13.     Entity chunk;
14.  
15.     NativeHashMap<uint, Edge> activeEdges;
16.     NativeHashMap<uint, uint> activeVoxels;
17.  
18.     NativeArray<int4> AXIS_OFFSET;
19.     NativeArray<int4> EDGE_NODE_OFFSETS;
20.  
21.     static int4[] AXIS_OFFSET_ARRAY =
22.     {
23.         int4(1, 0, 0, 0),
24.         int4(0, 1, 0, 0),
25.         int4(0, 0, 1, 0)
26.     };
27.  
28.     static int4[] EDGE_NODE_OFFSETS_ARRAY =
29.     {
30.         int4(0), int4(0, 0, 1, 0), int4(0, 1, 0, 0), int4(0, 1, 1, 0),
31.         int4(0), int4(1, 0, 0, 0), int4(0, 0, 1, 0), int4(1, 0, 1, 0),
32.         int4(0), int4(0, 1, 0, 0), int4(1, 0, 0, 0), int4(1, 1, 0, 0),
33.     };
34.  
35.  
36.     protected override void OnCreate()
37.     {
38.         chunksQuery = World.Active.EntityManager.CreateEntityQuery(typeof(NeedsMeshing)); //initialize the chunk query to grab everything with "ChunkInfo" on it
39.     }
40.  
41.     protected override JobHandle OnUpdate(JobHandle inputDeps)
42.     {
43.  
44.         chunks = chunksQuery.ToEntityArray(Allocator.TempJob);
45.  
46.         if(chunks.Length == 0)
47.         {
48.             chunks.Dispose();
49.             return inputDeps;
50.         }
51.  
52.         chunk = chunks[0];
53.         chunks.Dispose();
54.  
55.         ChunkInfo chunkInfo = EntityManager.GetComponentData<ChunkInfo>(chunk);
56.  
57.         ChunkMeshingInfo chunkMeshingInfo = new ChunkMeshingInfo
58.         {
59.             activeEdges = new NativeHashMap<uint, Edge>((chunkInfo.size.x) * (chunkInfo.size.y) * (chunkInfo.size.z) * 3, Allocator.TempJob),
60.             activeVoxels = new NativeHashMap<uint, uint>((chunkInfo.size.x) * (chunkInfo.size.y) * (chunkInfo.size.z), Allocator.TempJob),
61.         };
62.  
63.  
64.         EntityManager.AddSharedComponentData(chunk, chunkMeshingInfo);
65.  
66.  
67.         AXIS_OFFSET = new NativeArray<int4>(3, Allocator.TempJob);
68.         AXIS_OFFSET.CopyFrom(AXIS_OFFSET_ARRAY);
69.  
70.         EDGE_NODE_OFFSETS = new NativeArray<int4>(12, Allocator.TempJob);
71.         EDGE_NODE_OFFSETS.CopyFrom(EDGE_NODE_OFFSETS_ARRAY);
72.  
73.  
74.         var FindActiveVoxels = new FindActiveVoxelsAndEdgesJobs() {
75.             activeEdges = EntityManager.GetSharedComponentData<ChunkMeshingInfo>(chunk).activeEdges,
76.             activeVoxels = EntityManager.GetSharedComponentData<ChunkMeshingInfo>(chunk).activeVoxels,
77.             AXIS_OFFSET = AXIS_OFFSET,
78.             EDGE_NODE_OFFSETS = EDGE_NODE_OFFSETS,
79.             voxelData = EntityManager.GetBuffer<VoxelData>(chunk).Reinterpret<float>().ToNativeArray(Allocator.TempJob),
80.             SizeX = chunkInfo.size.x,
81.             SizeY = chunkInfo.size.y,
82.         };
83.  
84.         EntityManager.RemoveComponent<NeedsMeshing>(chunk);
85.  
86.         return FindActiveVoxels.Schedule(chunkInfo.size.x * chunkInfo.size.y * chunkInfo.size.z,1, inputDeps);
87.  
88.  
89.     }
90.  
91. }
92.  

 

Only allocate the hashmap once in create and dispose of it in destroy.
Update its capacity when needed.
Clear it in a job.

 

how stupid of me to not even realize that I could have done that. Thank you so much bro for curing my absolute brain fart!