Search Unity

Best Way to Implement This Workload Using Unity's Job System?

Discussion in 'C# Job System' started by Slight0, Mar 14, 2019.

  1. Slight0

    Slight0

    Joined:
    Dec 6, 2013
    Posts:
    28
    I've been looking into all the threading improvements made to unity over the years and wanted to bring some of the huge performance gains it boasts to some of my own projects. I have an existing Unity project that is a sort of voxel engine that uses regular C# threading to accomplish fast world generation and chunk mesh building from the voxel data.

    It's not too complex to imagine. Like you would expect, the map is a 2D array of Chunk structs which itself contains a 3D array of Voxel structs (among other metadata) to form the geometry of the world. Imagine each Voxel is basically an int.

    My current implementation uses two worker threads; a world builder thread and the chunk builder thread which are created at the start of the game and put to sleep when not being used. The world builder thread pulls from a queue of Chunks to be generated and does so one by one using a procedural algorithm to fill in the voxels. It then passes the generated Chunk to the chunk builder thread's queue and wakes that thread if it is not already running. The chunk builder thread pulls from its queue of Chunks and generates vertex and UV data for the Chunk and places the finished vertex/UV data in an output queue. Finally, the main thread during LateUpdate will check this output queue and based on some quick maths pulls from the queue the number of meshes it thinks it can turn into GameObjects within the remaining frame time. It uses locking and all that fun stuff to maintain thread safety.

    When it comes to refactoring it to use Jobs, the thing I'm struggling to wrap my head around is how I'm supposed to do large multi-frame workloads (possibly seconds of work) that periodically spit out results (Chunk meshes) to the main thread as they're worked. I get that I'd need to have at least two IParallelJob structs to do each Chunk at a time. I think I would need to assign dependency somehow so that the world builder job could pass it's NativeArray output to the mesh builder job when it finishes each Chunk, but I'm not sure if that's possible. If not, I could combine them together into one IParallelJob instead of two as I don't think that'd really affect much. My next problem is when exactly to call ScheduleBatchedJobs (start of frame? end of frame?). My biggest problem is how the main thread is supposed to know when to call Complete on the mesh builder job. The job itself might be in the middle of it's workload still, but may have a few Chunk meshes finished for the main thread to process. How do I access that "incomplete" resultset? Normally I would just get a lock for the output queue object, take take what I needed, then release the lock and instantiate the meshes. However, the IParallelJob can't push things out "piecewise" to my knowledge; it's all or nothing whereas I'm looking for more of a stream of output.

    Any ideas as to how I should pull Chunk meshes into the main thread in a piecewise fashion from a Job?