Search Unity

n:1 (input : output) not possible using IJobParallelFor, correct?

Discussion in 'Data Oriented Technology Stack' started by uani, Jun 28, 2018.

  1. uani

    uani

    Joined:
    Sep 6, 2013
    Posts:
    64
    NativeArray<int> of length n as input and NativeArray<int> of length 1 as output in an IJobParallelFor.

    Calculating min/max.

    Error "ReadWriteBuffers are restricted to only read & write the element at the job index".

    Thus I cannot have 1 output value for multiple input values, correct?

    If not, how can I ? :)
     
  2. Goldseeker

    Goldseeker

    Joined:
    Apr 2, 2013
    Posts:
    37
    That would require you to sync access to that variable from multiple job threads, and avoiding manual synchronization is one the corner stones of Job System
     
  3. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,265
    Will [NativeDisableParallelForRestriction] on the output NA help in this case?

    But that would be a chaos when 2 job read current max variable at the same time, both of them find that lesser than what it is working on and wants to modify but the lesser max of the 2 value got written later and you don't get the real maximum.
     
  4. uani

    uani

    Joined:
    Sep 6, 2013
    Posts:
    64
    hi, thank you for the hint! I don't know I have removed the job approach for the part in question already.

    You appear to be right regarding parallel writes: this being executed in parallel (like the name suggests :| ) (not simply "more efficient") the simple min/max approach likely fails :( .
     
  5. sSaltyDog

    sSaltyDog

    Joined:
    Mar 4, 2017
    Posts:
    33
    Old thread I know but hoping I can get some more info on this.

    I have the following code, which converts a heightmap into 3D terrain for an nxnxn cube and also returns the composition of that cube (all ground, all air or mixed).

    Code (CSharp):
    1. [BurstCompile]
    2. struct BlocksJob : IJobParallelFor
    3. {
    4.     //  Block data being generated in this loop
    5.     [NativeDisableParallelForRestriction] public NativeArray<Block> blocks;
    6.  
    7.     //  2 items long where 0 is air and 1 is ground
    8.     [NativeDisableParallelForRestriction] public NativeArray<int> hasAir_hasGround;
    9.  
    10.     [ReadOnly] public int cubeStart;
    11.     [ReadOnly] public int cubePosY;
    12.  
    13.     [ReadOnly] public NativeArray<Height> heightMap;
    14.     [ReadOnly] public int cubeSize;
    15.     [ReadOnly] public JobUtil util;
    16.  
    17.  
    18.     public void Execute(int i)
    19.     {
    20.         float3 pos = util.Unflatten(i, cubeSize);
    21.         float3 position = new float3(pos.x, pos.y+cubePosY, pos.z);
    22.  
    23.         int hMapIndex = util.Flatten2D((int)pos.x, (int)pos.z, cubeSize);
    24.         int type = 0;
    25.  
    26.         //  Check height map
    27.         if(position.y <= heightMap[hMapIndex].height)
    28.         {
    29.             //  Ground
    30.             type = 1;
    31.             hasAir_hasGround[1] = 1;
    32.         }
    33.         else
    34.         {
    35.             //  Air
    36.             //  type = 0;
    37.             hasAir_hasGround[0] = 1;
    38.         }
    39.  
    40.         blocks[i + cubeStart] = new Block
    41.         {
    42.             index = i,
    43.             type = type,
    44.             squareLocalPosition = position,
    45.         };
    46.     }
    47. }
    This seems to be a valid use case and certainly appears to work. But it raises questions:

    - Could using NativeDisableParallelForRestriction in this way cause any problems?
    - Does this use case mean there should be a feature in the API to do this? Is there?
    - Is there a better way to go about this?

    Thanks in advance :)
     
  6. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    You have an alternative for min/max: Using an output array, with an element for each thread:

    Code (CSharp):
    1.  
    2. using Unity.Collections.LowLevel.Unsafe;
    3. using Unity.Jobs.LowLevel.Unsafe;
    4.  
    5.  
    6. //...
    7.         var output = new NativeArray<int>(JobsUtility.MaxJobThreadCount, ...);
    8. //...
    9.  
    10.     struct Job : IJobParallelFor
    11.     {
    12.         [NativeDisableParallelForRestriction]
    13.         public NativeArray<int> output;
    14.  
    15.         [NativeSetThreadIndex]
    16.         int thread;
    17.  
    18.         public void Execute(int arrayIndex) {
    19.             //Code here
    20.             output[thread]  = math.max(output[thread], arrayIndex);
    21.         }
    22.     }
    [NativeDisableParallelForRestriction] will disable concurrency warnings, but you must be certain no two jobs (threads) will be writing or reading the same position at the same time. This case is one.

    JobsUtility.MaxJobThreadCount is self explanatory.

    And afterwards you do a final run (Main thread or serial IJob) over the output array, that waits/depends on the previous parallel job handle, and gets you your single output.
     
    Last edited: Dec 19, 2018
  7. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,668
    While a good rule of thumb, it's not always necessarily true. Primary in the specific case of never reading the value and all your threads only set the same value then it doesn't matter if multiple threads are writing to the same position.

    A good case for this is doing some type of 'any' logic over multiple threads. If any of these threads is true, set array[0] = 1. It doesn't matter the order of the operations only the final value.

    The code above using

    [NativeDisableParallelForRestriction] public NativeArray<int> hasAir_hasGround;


    from a quick glance actually looks safe to me, because while multiple threads might write to hasAir_hasGround[0] and hasAir_hasGround[1], they only ever set the value to 1 and no thread will ever read from it. The result should always be as expected.
     
    sSaltyDog likes this.
  8. iam2bam

    iam2bam

    Joined:
    Mar 9, 2016
    Posts:
    36
    I get where you're going, but I think that only works on very specific cases. Also depending on packing, alignment and CPU architecture writing one of two consecutive int32, could be actually "written" as a read-patch-write of e.g. an int64.
    That's all the craze of packing and alignment. There could be some atomicity according to the actual machine instruction but I don't think we'll ever get that kind of control from an engine.
     
    sSaltyDog likes this.