Question NativeList in ForParallelJob

Hello,

currently iam new at Dots and now come to a point, i can't find a solution with my knowledge at this point, so i need help from higher level knowledge?

OnEnable:

Code (CSharp):

1.  NativeArray<float3x4> matrices = new NativeArray<float3x4>(voxelCount * 6, Allocator.TempJob);
2.  

The Job:

Code (CSharp):

1.     [BurstCompile(FloatPrecision.Low, FloatMode.Fast, CompileSynchronously = true)]
2.     struct SetMatrices : IJobParallelFor {
3.  
4.         [ReadOnly]
5.         public NativeArray<Object> objects;
6.  
7.         [WriteOnly]
8.         public NativeArray<float3x4> matrices;
9.  
10.         public void Execute (int i) {
11.             Object o = objects[i / 6];
12.  
13.             NativeList<float3x4> tempList = new NativeList<float3x4>(Allocator.Temp);
14.  
15.             if (o.solid) {
16.                 float3x3 r = float3x3(new quaternion(0.7071068f, 0, 0, 0.7071068f));
17.                 tempList.Add(float3x4(r.c0, r.c1, r.c2, v.position));
18.             }
19.             matrices = tempList;
20.         }
21.     }

But at the Point "matrices = tempList" does nothing.

In short words:
"i need a NativeArray with custom length depending on the if solid statement or a NativeList as output for the ParallelJob"

I'am dont want to waste more time, iam currently at Day 2 of wasting time.

Hope someone can explain how to solve this?

If someone find some things that can be written better for performance reasons or shorter code, let me know iam interested on all knowledge.
Thank you

EDIT:

if i write directly to the NativeArray it works, because the fixed length OnEnable.

Code (CSharp):

1. //Like this:
2.  
3. matrices[i] = ...;
4.  
5. and remove all List-Things from the code.

but i need to check if solid and remove all non-solid Objects.

 

#bump/edit

Updated Code:

Code (CSharp):

1. [BurstCompile(FloatPrecision.Low, FloatMode.Fast, CompileSynchronously = true)]
2.     struct SetMatrices : IJobParallelFor {
3.  
4.         [ReadOnly]
5.         public NativeArray<Object> objects;
6.  
7.         [WriteOnly]
8.         public NativeList<float3x4> elements;
9.  
10.         public void Execute (int i) {
11.             Object o = objects[i];
12.  
13.             if (o.solid) {
14.                 float3x3 r = float3x3(dir]);
15.                 elements.Add(float3x4(r.c0, r.c1, r.c2, v.position));
16.             }
17.         }
18.     }

Code (CSharp):

1. JobHandle handle = new SetMatrices { objects = objects, elements = elements }.Schedule(count, 1);
2.             handle.Complete();

Its possible to run this code parallel? because unity get me the error:

Code (CSharp):

1. InvalidOperationException: SetMatrices.elements is not declared [ReadOnly] in a IJobParallelFor job. The container does not support parallel writing. Please use a more suitable container type.

 

When the job is scheduled parallel, you have to use NativeList<T>.ParallelWriter and use AsParallelWriter() when setting the job parameter. Be careful though, when used in parallel there's no resizing so the List has to have the proper size before the job is scheduled. Set the size with .Capacity beforehand.
I'd advise against a job in parallel with a NativeList because of thread locks, it usually performs slower than single-threaded.

 

Thank you for the answer.

But resizing is the thing that i need. Because it's depending on the data if some object are solid or not, then i need to calculate the size of the list/array with that. It will be cool if writing to a list as parallel job will be possible, because iam only use the Add Function and if i use them 20 times or 1 times the list should have the objects.

The order does not matter for the list, only new objects should be added and the length of the list automatically changed. If it were possible I could change the length by hand +1?

 

This is the equivalent of adding without resizing (reallocating might be a better word here). A NativeList is actually a pointer to an array where you add elements to until it fills up to capacity. Once it is full, it needs to allocate a new larger array and copy all the elements to it. Unity does not support this process happening in parallel. I you set your capacity to a high value, then you don't have to worry about this. Otherwise you will need to write to a different container and then copy that to a NativeList in a separate job.

 

Okay thats interesting, so if i create a large List at OnEnable and then add parallel all elements, its work in parallel job?
Which arguments i need for the job to work without errors?

After all jobs are done, i'am convert the list to an nativeArray because iam need an array for computebuffer, if the list has a size of 100 but only 12 things are added during the jobs, how can i only add the 12 things to the array with best performance?

 

If you set the capacity to a large enough value on OnEnable (the constructor might take the initial capacity as an argument), then you just need a NativeList.ParallelWriter for your parallel job. But be careful. NativeList.ParallelWriter does not work well with lots of writes.

If you convert the list of 100 capacity but only contains 12 items (length of 12), then the array will be of length 12. However, using ComputeBuffer.SetData can be relatively slow compared to using SubUpdate buffers. So if performance absolutely matters here, you will want to switch to that. I recently did that for a custom skinning solution, so if that's a path you want to pursue, I can give you some insight as to how to do it right. There's some buffer cycling you have to do to make it stable.

 

Okay so ParallelWriter for amount of 10000 times is no good solution right?
Iam currently create the ComputerBuffer every time, the job updates the size of the objects

Code (CSharp):

1.             matricesBuffer = new ComputeBuffer(matrices.Length, stride);
2.  
3.             matricesBuffer.SetData(matrices);

i know thats not so good, but iam new at it and it workes by 1 millions cubes with 80 fps now, but i want to make the project available for lower pcs.

 

Out of how many possible input elements that you are iterating over? And how much other logic is in your inner loop? If you have lots of filtered out elements and a large inner loop, you might be fine. But otherwise there are other containers and strategies available.

The biggest issue is that you are generating GC every frame. It is much worse if you aren't calling Dispose() on the ComputeBuffer. All of this will result in "spike frames" where a single frame takes way longer than the others and results in a stutter. Turning on Incremental GC might be sufficient to solve this for you.

But the idea with SubUpdate buffers is that you can write to them directly from jobs. However, you have to keep a cycle of buffers because the GPU keeps a couple of frames in flight. For devices with async readback, you can request a readback for a tiny buffer, and when the readback finishes, that means all buffers used for that frame are free to use again. If the device does not support async readback, then you just wait for 3 or 4 frames.

 

I got 2 jobs, first one creates a fixed array with length of renderdistance³, then get noise on all objects, Output the array and use it on job 2

Job 2 looks at all objects if solid and some other if statements like if the object is on border, then fills a list with only the objects i want. Object is struct with some data.

Job 2 outputs a list these are converted to array and added to the buffer.

Its possible to create a buffer at onenable with a maximum possible length if all objects are solid and so on? But only puts in with Computebuffet.adddata the amount of objects i need? So its possible to create the buffer once and dispose them at OnDisable?

Or second option/question the buffer is used in DrawMeshInstanced evers frame in update, so i need them always or onl once?

 

Yeah. You can create a buffer with the max size and only write what you need. And you only need to create a buffer once and reuse it. However, if you are using SubUpdates, you need a few buffers in rotation.

 

Okay i got it worked with a global Buffer, what do the SubUpdates in my case if i implement them?

Other little question i got the error

Code (CSharp):

1.  Burst error BC1051: Invalid managed type found for the field `neighborIndexes` of the struct `Object`.: the type `int[]` is a managed type and  is not supported

how can i use a int[] in struct with burst?

 

You can't as far I am aware.
In that case either use class as a contaimer, or use native collection as an array.

 

I find a solution for the int[] case.
you need to add this to the struct:

Code (CSharp):

1.     int a1, a2, a3, a4, a5, a6;
2.  
3.     public int GetNeighborIndex (int element) {
4.         switch (element) {
5.             case 1:
6.                 return a1;
7.             case 2:
8.                 return a2;
9.             case 3:
10.                 return a3;
11.             case 4:
12.                 return a4;
13.             case 5:
14.                 return a5;
15.             case 6:
16.                 return a6;
17.             default:
18.                 return -1;
19.         }
20.     }
21.  
22.     public void SetNeighborIndex (int element, int value) {
23.         switch (element) {
24.             case 1:
25.                 a1 = value;
26.                 break;
27.             case 2:
28.                 a2 = value;
29.                 break;
30.             case 3:
31.                 a3 = value;
32.                 break;
33.             case 4:
34.                 a4 = value;
35.                 break;
36.             case 5:
37.                 a5 = value;
38.                 break;
39.             case 6:
40.                 a6 = value;
41.                 break;
42.         }
43.     }

I think a wrapper like a custom class or struct for this is better if the code is used many times.

 

So iam at the point to test NativeList<float3x4>.ParallelWriter but i dont know how can i get the length of the List or copy all content to a nativeArray?

 

You can take length before you assign parallelWriter.
And you can also get content of once written NativeList after the job. Just don't use ParallelWriter.

var myNativeList = ....
var myNativeListAsParallelWriter = myNativeList.AsParallelWriter();

Pass myNativeListAsParallelWriter to the parallel job and assign data.
Use another job, to read data from myNativeList.

 

Okay and on the end i need to fill my buffer with that data from the list, is there a way to convert the NativeList.ParallelWriter to NativeArray?

And Dispose a NativeList.ParallelWriter doent work? Do i need for save memory such a function?

 

A ten parallare writing, just use myNativeList directly (without parallel writer) , that where date is stored. You can read from it and dispose it.
Consider parallel writer as a function and reference to the actual list. Hence it is not a separate collection.