Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Voting for the Unity Awards are OPEN! We’re looking to celebrate creators across games, industry, film, and many more categories. Cast your vote now for all categories
    Dismiss Notice
  3. Dismiss Notice

How to write to another index with the jobsystem?

Discussion in 'Entity Component System' started by Desoxi, Mar 2, 2018.

  1. Desoxi

    Desoxi

    Joined:
    Apr 12, 2015
    Posts:
    195
    Hey there,

    i know its still in beta etc. etc., but maybe someone has a solution for this.
    Im trying to access another index of a specific native array in my job system code, but the limit is always what i set when scheduling the job.

    So for example when scheduling a job like this:
    Code (CSharp):
    1. forcesJobHandle = forcesJob.Schedule(objectCount, 64, positionsJobHandle);
    It can access other indices, but only in the range of [0..64], which is ok. But is there a workaround to access index 65 for example? I tried using the same array twice (one is readonly the other is writeonly) and in between the frames i copy the write only one into the read only array.
    I thought that would be a solution because the error mentions "ReadWriteBuffers are restricted to only read & write the element at the job index. You can use double buffering strategies to avoid race conditions due to reading & writing in parallel to the same elements from a job.", so i thought if it is write only it may be possible. Unfortunately i was wrong :D

    Greetings,
    Desoxi
     
  2. Krajca

    Krajca

    Joined:
    May 6, 2014
    Posts:
    347
    I think it's a bit unsafe to access other indices within a job because you schedule 64 jobs and you can't be sure in what order they will be done, so i.e. you can access unprocessed indices.
     
  3. Desoxi

    Desoxi

    Joined:
    Apr 12, 2015
    Posts:
    195
    Yep exactly. In computeshaders you have the possibility to use protected writing through the "InterlockedAdd" method which guarantees that every threads data change is applied correctly.

    I dont know if the job system has this kind of functionality, too. But it would be pretty handy.
     
  4. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    558
    you can read from multiple threads at a time.

    if you declare one field as [ReadOnly] you should be able to read from it at any index.
    then you set the result at your index of another [WriteOnly] array.

    instead of copying the arrays over you can simply swap the references in the main thread.

    e.g.:
    Code (CSharp):
    1. struct Job : IJobParallelFor {
    2.   [ReadOnly] NativeArray<int> input;
    3.   [WriteOnly] NativeArray<int> output;
    4.   void Execute(int index) {
    5.     output[index] = input[whatever...];
    6.   }
    7. }
    8.  
    9. // main thread
    10. NativeArray<int>  array1, array2 = ...; // Allocator.Persistent
    11. // init array1
    12. var job1 = new Job {input = array1, output = array2}.Schedule(len, 64);
    13. var job2 = new Job {input = array2, output = array1}.Schedule(len, 64, job1); //...chain as many as you like
    14. job2.Complete();
    15. // final output is in array1 for even number of jobs, array2 for odd number of jobs
     
    eterlan likes this.
  5. Desoxi

    Desoxi

    Joined:
    Apr 12, 2015
    Posts:
    195
    I tried something similar in doing the following:


    Code (CSharp):
    1.     struct ForcesJob : IJobParallelFor
    2.     {
    3.         [WriteOnly]
    4.         public NativeArray<Vector3> force;
    5.         [ReadOnly]
    6.         public NativeArray<Vector3> readOnlyForce;
    7.  
    8.         public void Execute(int i)
    9.         {
    10.  
    11.             force[i] = readOnlyForce[i] + xzy;
    12.             force[i+1] = readOnlyForce[i] - xyz;
    13.         }
    14.     }
    Thats what i meant with the access to another index. The increment is not a problem as long as im in the range of [0..63]
     
  6. Krajca

    Krajca

    Joined:
    May 6, 2014
    Posts:
    347
    That should be an error because for i==last_index there is no force[i+1]

    I think in this case I would rather do one job with for in it.
     
  7. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    558
    you are writing twice on the same element, possibly from multiple threads and with undefined ordering.
    e.g. element n+1 is accessed from Execute(i = n) as force[i+1] and from Execute(i = n+1) as force

    you can do something like this instead
    Code (CSharp):
    1.     struct ForcesJob : IJobParallelFor
    2.     {
    3.         [WriteOnly]
    4.         public NativeArray<Vector3> force;
    5.         [ReadOnly]
    6.         public NativeArray<Vector3> readOnlyForce;
    7.         public void Execute(int i)
    8.         {
    9.            if (i == 0) force[i] = readOnlyForce[i] - xyz;
    10.            else force[i] = readOnlyForce[i-1] + xzy + readOnlyForce[i] - xyz;
    11.         }
    12.     }
    notice I only write at force and read from multiple indices.
     
  8. Desoxi

    Desoxi

    Joined:
    Apr 12, 2015
    Posts:
    195
    Very nice, i was trying the same approach atm! Thank you @M_R
     
  9. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    That is indeed the correct solution to this.

    As a general rule aiming for determinism is a good idea when writing multithreaded code. And in this case even if you use atomic ops it would still result in non-deterministic behaviour, since it depends on which jobs runs where. Double buffering as M_R explains is a solid solution to that.


    Secondly using atomic ops or generally reading / writing to the same array that another worker thread might be reading/writing to will most likely result in significantly worse performance.

    Specifically when reading / writeing to the same cache line in parallel the CPU has to perform very expensive cache line synchronization stalling the whole pipeline.

    These are two big reasons why by default we allow only writing to the parallel for index.
     
    Last edited: Mar 4, 2018