Suggestion: Job Priority System

I love the job system and what Unity has done with it.
Something that is missing from the job system is a priority system for long-running background jobs.
And I'm certainly not the only one

Before you say that I have to use .Complete(), no I don't want to use complete on a long-running high priority job, because it blocks the main thread.

Before you say that that's not what the job system is for, making your own threads and mixing that with the job system doesn't match very well, because the job system makes sure the main thread is unhindered as much as possible.

 

The reason this isn't supported is because what happens if all your worker threads start executing long-running jobs and then you need to schedule a high priority job? Only your main thread will be able to handle it. Some people want that behavior. Others will want to limit the long-running job to 1 or 2 threads.

It might be what you want is the ability to define your own thread pool and allow Unity to schedule Burst jobs against that thread pool instead of the default one.

I encourage you to propose a real performance problem you are running into and to post numbers. Understanding why a particular problem cannot be solved with what currently exists will help Unity find a solution to that problem.

 

If all are long-running low prio, the ones with highest prio should take over.
Is that not how a priority system works?

There are so many real problems that would benefit from that.
An example: generating far away chunks in a voxel game vs generating chunks close-by (high prio), while constantly loading and storing files in another long-running low priority job.

 

Once a job starts, it can't be stopped, paused, or interrupted without another thread. Introducing another thread causes context switching overhead. That's not a Unity limitation. That's a general computing limitation that some software frameworks sacrifice performance to abstract away from you.

Most things that would benefit from a priority system can also benefit from a round-robin system. Round-robin comes with the added benefit of being deterministic-capable, which is nice for multiplayer. It's also relatively easy to implement in DOTS.

So again, I encourage you to present a real performance problem you are running into and to post numbers. DOTS is all about using the right tool for the right task, and Unity wants to keep the number of tools down for learning and development reasons. So if you want a new tool to solve the problem, you best bet is to show that no existing tool can solve that problem.

 

How is it hard to imagine you want one job to have a higher priority? I don't get that you can't see how useful that would be in many many use-cases (I already gave an example).

Round-robin scheduling would be a nice step in the good direction.
But as far as I'm aware, there is none of that in the job system.
The job system just runs a job and doesn't stop.

Best solution would just default as is and if you want to choose another scheduler for example round-robin or priority queues, etc.
Context switches will only occur if you choose one of those schedulers, it's still worth the overhead!

If it's a hard limitation that you can't interrupt a thread for another one, then fine I get it.

You keep on telling me to post numbers, but it's not about numbers. It's about a ton of use-cases.

 

@DreamingImLatios

You certainly know this better than I do, but I was wondering also if it would not be possibly to have a priority system (let's forget job dependencies for a moment and say 2 independent jobs). See here: https://forum.unity.com/threads/ijo...-not-use-the-main-thread.682078/#post-4874603

My understanding of how the scheduler works is based on this: https://docs.unity3d.com/Manual/JobSystemParallelForJobs.html

So basically, it creates batches of native jobs (i.e. self defined, chunks or could even split an ijob - then requires dependency) and put's them in a queue, could we not have a high priority queue and a low priority queue and whenever the high priority is empty, the scheduler pulls from the low priority queue. This would not require to stop / pause / interrupt a job in my understanding.

Could be all wrong, but happy to understand this better...

 

Everything you have stated has been hypothetical, not a real problem you are struggling with right now that is causing you pain with no currently available viable solution. As such, I am able to propose hypothetical alternative solutions that do not require additional work by Unity. If I were Unity, I wouldn't want to spend development effort solving a problem that could potentially already be solvable by work already done.

Round-Robin scheduling is game-specific. Its something you have to do. You can schedule by system, spatially, a combination of both, and even with a prioritization scheme that puts focus on entities in view. You just have some component that keeps track of the round robin state and run systems and jobs based on that each frame.

You can interrupt a thread with another thread. That's handled at the OS level. It just isn't performant and on some OSes there's no thread priority system so the high and low priority threads would be treated equally.

What happens if you have a break in high priority jobs and then all the threads start working on long low-priority jobs and then a new wave of high priority jobs get scheduled? That's the real problem here with a priority system.

 

I would very much also love a priority system, but I'll survive if that's not doable.

However, I would settle for an alternate solution. Would it be possible for Unity to add a functionality where you can specify which thread a specific system or job can run on? For example, I would very much like to:

1. Request the number of threads the machine the game is running on has.
2. Identify which one is the main thread.
3. Schedule certain jobs, say simulation jobs, on X number of threads, excluding the main thread, specifying a maximum allotted amount of time to complete the jobs.
4. The main thread is now free to deal with main threaded code only, such as drawing. At the end of the allotted amount of time, sync data with the main thread where necessary.

I think this solution would avoid context switching as well as the pitfalls of a traditional priority system as mentioned by DreamingImLatios. Feel free to point out any absurdities with my proposal.

 

Are you a forum troll?
Did you even read my example?
Again, how is it hard to imagine you want one job to have a higher priority?
I feel there is no point in replying to you anymore, you're just stuck in your own beliefs.

 

Yes please! That would be very helpful for many of us!
Splitting up the background job and slowly scheduling jobs for a low prio job feels like a workaround.
I don't think workarounds is what you stand for.

 

I believe that is actually not the case.
The high prio jobs are often on the critical path and determine the efficiency.
But when low prio jobs are taking up the spots, the high prio will have to give up space for them at the current state.

Let me give an example:
- Some jobs are just loading and saving data for persistence of the game, but they can be done later, whatever, it's low prio.
- While a saving job takes up a spot, suddenly the player moves and a tree needs to be generated asap, because it's quite close.

 

So what happens when a heavy game is trying to push the hardware as fast as possible (trying to hit 144 Hz on max settings) and all the worker threads are too busy handling high priority jobs that they never get around to the low priority jobs and all of a sudden the game never gets saved or the in-game market never updates its prices?

I think you underestimate the number of edge cases you introduce when you let a job not be required to finish for an indefinite number of frames.

 

Which is why I proposed a time limit earlier. If not, then at least a certain number of cycles per high priority jobs before it decides to check for a lower priority one.

Or a manual division of threads like I posted earlier. I’m not too familiar with the terms, but I believe that’s what Joachim meant by dedicated job queues. You’d have one for low prio and one for high prio. This of course introduces a few issues such as computation waste (one queue might not have enough work, wasting time doing nothing while the other is busy. But in that case, who really cares if work is wasted on the background threads as long as the main thread is free to run wild at 144hz and the task is completed in 0.3 seconds), but I would be EXTREMELY curious to see results with a tick system. I’m very tempted to believe that properly designed code would work even better with the priority method than with the current ‘force every job to complete in one frame because the main thread needs that data periodically’ method.

 

Yep!

It should be in our hands to decide what happens when workers threads are too busy handling high prio jobs.
I think you underestimate programmers.
A solution could be to let us implement the scheduler behavior of our needs and just expose the necessary handles.
Maybe put in the docs that these handles are for advanced users.
Or even better, have a prio scheduler built in that does retain some fairness.
Remember that the default behavior doesn't have to be affected by that.

 

I have to echo some of this, it feels essential to have Some form of priority queue. Attached image shows my current problem.


This is a built in Unity method iterating over the objects in the scene. Its now taking a significant amount of my frames time because its waiting on other jobs finishing. This is horrible!

I'm only just starting to research this problem and its feeling like overal if my jobs are running for any significant amount of time then it can be extremely unreliable to maintain a steady frame since a Unity method will randomly wait on a thread (probably cause I have a long job queue).

It feels like any solutions I have are working around, frankly hacking around, the problem. I don't want any of my jobs to block any Unity based jobs which are syncing on the main thread, and I don't know how to tell them to do that :/