Search Unity

How to run an iterative algorithm in JobComponentSystem?

Discussion in 'Data Oriented Technology Stack' started by TMPxyz, Jul 11, 2019.

  1. TMPxyz

    TMPxyz

    Joined:
    Jul 20, 2012
    Posts:
    765
    Hi, I've an algorithm that needs to iterate multiple times in a frame until converge. I want to utilize JobComponentSystem to parallelize it, but I don't quite know how to make it.

    Currently I'm using a ComponentSystem with a while-loop to do that, here is the pseudo code:

    Code (CSharp):
    1. void OnUpdate(...)
    2. {
    3.      do {
    4.          // ***PARALLEL***
    5.          //loop through [readonly] a long array RECORDS with multiple jobs,
    6.          //generate some pending modifications, write them into another array MODS
    7.          _Read_Records_and_Generate_Modifications_Parallel();
    8.  
    9.          // ***SINGLE THREAD***
    10.          // wait for all jobs, then apply the modifications in the MODS back to RECORDS
    11.          _Write_Modifications_into_Records();
    12.  
    13.      } while( ! _Is_Data_Converged() );
    14. }
    As I'm using ComponentSystem instead of JobCompnentSystem, I've to use IJobParallelFor to spread the work on jobs in _Read_Records...() function, then wait for its completion to run the _Write_Modification...().

    How can I use the dependency system to make the jobs loop and check convergence by themselves? So I can use a JobComponentSystem to get better core utilizations.
     
  2. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,650
    I don't think you really can in a single frame without sync points.

    Either need to find a quick method of precomputing the number of loops you require, rethink your logic, spread the work over multiple frames or settle on the fact you'd need sync points.

    I hope someone else has better news for you.
     
  3. TMPxyz

    TMPxyz

    Joined:
    Jul 20, 2012
    Posts:
    765
    Assume I could get an appoximate upper bound for the loops needed (eg: N):
    Can I pre-schedule N loops of jobs at start, and cancel the remaining scheduled jobs when the data converges in the loop X?
     
  4. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    459
    you can't cancel the jobs, but you can early out of them from
    Execute
     
  5. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,355
    Is there a reason why you need to do the work in small partitions like that? Vs a single job that creates the modifications, chained into another job that does something with them?

    There are use cases that do require partitioning the work like your example has of course, it's not not very usual which is why I ask. But anyways in that case you can simply create/schedule the jobs in a loop and chain their dependencies. Between JobHandle.CombineDependencies and the ability to store JobHandle's in a NativeArray, you can setup all sorts of complex dependency chains.
     
  6. TMPxyz

    TMPxyz

    Joined:
    Jul 20, 2012
    Posts:
    765
    I need to repeat the "Create_Modification -> Write_Back_Dataset" loop until the dataset finally converges, I cannot tell how many loops it needs beforehand. And I prefer it done in single frame to avoid interference from/to other modules.

    Is it possible to let the job to re-schedule itself?
     
  7. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,650
    You should consider jobs as completely isolated, they have no knowledge of the outside world and can not interact with it.

    Without knowing the details on the work your algorithm is doing, can't really provide suggestions on alternatives.
     
    Last edited: Jul 12, 2019