Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

What's the proper way to use the job system with jobs that take longer than a couple frames?

Discussion in 'C# Job System' started by Snubber, Apr 12, 2022.

  1. Snubber

    Snubber

    Joined:
    Jan 20, 2020
    Posts:
    65
    Hello! I am making a game with procedural terrain generation. Terrain is loaded in as chunks. Loading the chunks can take longer than a couple frames. For the most part I am using C# multithreading but I am trying to use the Unity Job System more.
    So if a job takes longer than 4 frames you're supposed to use
    Allocator.Persistent
    but it sounds like that's pretty inefficient as the documentation says "Don’t use Persistent where performance is essential" also other blog articles / research provide evidence to support this.

    But the main thing I am confused about is what should I do if I don't really need the results of a job until it is done? (i.e. I am generating a mesh for the grass on the terrain chunks)
    With C# multithreading I can make a callback to run when the thread is finished. But with the job system it seems like you need to either check if the job has been completed every frame and then when it is you can run your "callback" OR you just need to call
    .Complete()
    and hope that the job was close enough to finishing that you won't lose frames. Neither of those solutions seem ideal to me. What do y'all do in this situation?
     
    Last edited: Apr 12, 2022
  2. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,222
    You don't want to make to make lots of small allocations with Persistent. But once you have the allocated memory, it is as fast as any other memory. Also, while Persistent may not be the fastest option, it isn't necessarily slow either.
     
  3. Snubber

    Snubber

    Joined:
    Jan 20, 2020
    Posts:
    65
    Ok sure. But I am really wondering about how you should wait for a job to be finished if it can take a while.
     
  4. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,926
    Never done it in practice, but

    If you want to, you could create a system that has a List<DoOnJobComplete>, where DoOnJobComplete is a class that holds the JobHandle and a System.Action of what to do when the job completes. Internally the system would check jobHandle.IsCompleted for all elements in that list in its OnUpdate(), and call the Action when appropriate

    But otherwise, simply checking .IsCompleted every frame will be pretty harmless. Compared to a theoretical OnComplete() callback, it gives you better control over when exactly that logic is executed, with the systems ordering and all. Sometimes that order will be very important
     
    Last edited: Apr 12, 2022
    CodeSmile likes this.
  5. inSight01

    inSight01

    Joined:
    Apr 18, 2017
    Posts:
    90
    Curious to know how this works if you are working on data that needs to be accessed by another system (e.g. Spatial partition system where another system may want to access data from it etc). The whole job system is a bit out of my league but I'm still learning. But from what I understand you can't access a container that is currently being written to in a job. So, how would you work around this?

    Would something like this work?

    Code (CSharp):
    1.  
    2. public partial class SomeSystem : SystemBase
    3. {
    4.     public static SomeSystem Instance { get; private set; }
    5.     private NativeArray<int> _someNativeArray;
    6.     private NativeArray<int> _someNativeArrayForJob;
    7.     private JobHandle _someJobHandle;
    8.  
    9.     protected override void OnCreate()
    10.     {
    11.         if (Instance == null) Instance = this;
    12.     }
    13.  
    14.     protected override void OnDestroy()
    15.     {
    16.         _someNativeArray.Dispose();
    17.         _someNativeArrayForJob.Dispose();
    18.     }
    19.  
    20.     protected override void OnStartRunning()
    21.     {
    22.         _someNativeArray = new NativeArray<int>(100000, Allocator.Persistent);
    23.         _someNativeArrayForJob = new NativeArray<int>(100000, Allocator.Persistent);
    24.     }
    25.  
    26.     protected override void OnUpdate()
    27.     {
    28.         if (!_someJobHandle.IsCompleted) return;
    29.         _someJobHandle.Complete();
    30.         _someNativeArray = _someNativeArrayForJob;
    31.         _someJobHandle = new SomeJob()
    32.         {
    33.             SomeNativeArray= _someNativeArrayForJob
    34.         }.Schedule(_someNativeArrayForJob.Length, 1);
    35.     }
    36.  
    37.     // Can be instanced by another system using SomeSystem.Instance.GetArray().
    38.     public NativeArray<int> GetArray()
    39.     {
    40.         return _someNativeArray;
    41.     }
    42.  
    43.     private struct SomeJob : IJobParallelFor
    44.     {
    45.         public NativeArray<int> SomeNativeArray;
    46.         public void Execute(int index)
    47.         {
    48.             // Some code here
    49.         }
    50.     }
    51. }
    52.  
     
    Last edited: Jan 5, 2023
  6. SF_FrankvHoof

    SF_FrankvHoof

    Joined:
    Apr 1, 2022
    Posts:
    780
    While the job is running, you shouldn't be reading from the container. This is because you don't know if any of the data is valid.
    If you're constantly updating, and just want to get the 'last valid data', you can swap between 2 NativeContainers, and allow the one not currently being used by a job to be read by other objects.
     
    Snubber likes this.