Search Unity

[SOLVED] Access job-index from IJobParallelFor to write array not overlapping&avoid concurrency.

Discussion in 'Data Oriented Technology Stack' started by iam2bam, Dec 16, 2018.

  1. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Hello, I've been messing around with ECS/Jobs, but I stomped a couple times with an inconvenience.

    1) Want to initialize Mathematics.Random with a different seed per job.
    2) I want to write collision data to predefined NativeSlices (to iterate afterwards)
    3) I want to do it in parallel

    Right now I'm using a non-parallel job for (1) and (2), but would like to know if there's a way to know the native job index from inside the IJobParallelFor interface (does that makes sense?).
    Knowing that and the maximum native job count (don't know how either) I could at least avoid time penalties from concurrency locks.
    These systems are barriers for other systems and having just one job may free the main thread but keep the other cores idle.

    I was also wondering how well batching my own IJobs and combining all dependencies would work, too.

    Have you experienced this and solved it, or tested performance for alternatives?
    Any tips are greatly appreciated, thanks.
     
  2. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    What I want to do is, say

    Code (CSharp):
    1.  
    2.  
    3. rngs = new NativeArray<Unity.Mathematics.Random>(Jobs.MaxJobs, ...);
    4. for(int i = 0; i<Jobs.MaxJobs; i++)
    5.     rngs[i] = new Unity.Mathematics.Random(UnityEngine.Random.Range(int.MinValue, int.MaxValue));
    6.  
    7. //...
    8. new ParallelJob {
    9.      rngs = rngs;
    10. }.Schedule()
    11. //...
    12.  
    13. struct ParallelJob : IJobParallelFor {
    14.      NativeArray<Unity.Mathematics.Random> rngs;
    15.      void Execute(int index, int jobIndex) {
    16.         var value = rngs[jobIndex].NextFloat();
    17.         //...
    18.      }
    19. }
    Or

    Code (CSharp):
    1.  
    2. const int MaxCollisions = 100000;
    3. collisions = new NativeArray<CollisionData>(MaxCollisions, ...)
    4. counts = new NativeArray<int>(Jobs.MaxJobs, ...)
    5.  
    6. //...
    7.  
    8. new ParallelJob {
    9.     stuff = stuff
    10.     , collisions = collisions
    11.     , counts = counts
    12.     , size = MaxCollisions / Jobs.MaxJobs
    13. }.Schedule().Complete();
    14.  
    15. CompactCollisions(Jobs.MaxJobs, counts, collisions);  //This could compact slices using UnsafeUtil.MemCpy backwards and such
    16.  
    17. //...
    18.  
    19. struct ParallelJob : IJobParallelFor {
    20.     public NativeArray<Stuff> stuff;
    21.     public NativeArray<CollisionData> collisions;
    22.     public NativeArray<int> counts;
    23.     public int size;
    24.  
    25.      void Execute(int index, int jobIndex) {
    26.         var count = counts[jobIndex];
    27.         var offset = jobIndex * size;
    28.  
    29.         for(int j = index+1; j < stuff.Length; j++) {
    30.             if(Collides(stuff[index], stuff[j]))
    31.                 collisions[offset + count++] = new CollisionData(index, j);
    32.  
    33.             if(count == size) {
    34.                     counts[jobIndex] = count;
    35.                     return;
    36.              }
    37.         }
    38.  
    39.         counts[jobIndex] = count;
    40.      }
    41. }
     
  3. Guerro323

    Guerro323

    Joined:
    Sep 2, 2014
    Posts:
    25
    Hello, you can know which thread is currently executed in a ParallelFor job with the [NativeSetThreadIndex] attribute

    Code (CSharp):
    1. struct ParallelJob : IJobParallelFor
    2. {
    3.     [NativeSetThreadIndex]
    4.     private int m_ThreadIndex;
    5.     ...
    6.  
    7.     public Execute(int index)
    8.     {
    9.         ...
    10.     }
    11. }
    https://github.com/Unity-Technologi...erence/custom_job_types.md#better-cache-usage
     
    florianhanke, iam2bam and Attatekjir like this.
  4. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Wow, I totally missed that (a lot to soak in). Thanks!!!
     
  5. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    It works!

    Test code:
    Code (CSharp):
    1.     [BurstCompile]
    2. using Unity.Collections.LowLevel.Unsafe;
    3.  
    4. //...
    5.  
    6.     struct TestJob : IJobParallelFor {
    7.         [NativeDisableContainerSafetyRestriction]
    8.         [WriteOnly]
    9.         public NativeArray<int> vs;
    10.  
    11.         [NativeSetThreadIndex] int threadId;
    12.  
    13.         public void Execute(int index) {
    14.             vs[threadId]++;
    15.         }
    16.     }
    17.  
    18. //...
    19.  
    20.         var vs = new NativeArray<int>(JobsUtility.MaxJobThreadCount, Allocator.TempJob, NativeArrayOptions.ClearMemory);
    21.         var tjob = new TestJob { vs = vs };
    22.         tjob.Schedule(1000, 1).Complete();
    23.         for(int i = 0; i < vs.Length; i++) {
    24.             if(vs[i] > 0)
    25.                 Debug.Log("VAL " + i + " = " + vs[i]);
    26.         }
    27.         vs.Dispose();
     
    Last edited: Dec 17, 2018
    Guerro323 likes this.
  6. elcionap

    elcionap

    Joined:
    Jan 11, 2016
    Posts:
    87
    iam2bam likes this.
  7. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Thanks for this! That was the missing part.

    That's a nice idea, I will probably implement it that way if I need to use it in multiple places, but for the time being I'm happy with passing a NativeArray, performance-wise I think they're both kindof just copying a pointer, a length and an allocator enum.
     
  8. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    Now that I tried it, I remembered you need to "update" the NativeArrays elements because NativeArray does not return (yet) a reference to it's element.
    i.e.
    Code (CSharp):
    1. var rng = rngs[threadId];
    2. props.color = new float4(rng.NextFloat3(), 1);
    3. rngs[threadId] = rng;        //This is necessary to update the state of the element inside the array.
    4.  
    So, rolling my own custom NativeContainer for the RNG would probably be the least error-prone approach, for just a little more effort.