Best Way to Split a Long-Running Function So it's Non-Blocking?

I have a very expensive function that manipulates game objects that can take 1-2 seconds to finish running. At the moment, if I run it the game will simply hang until it finishes.

I've put it into a Coroutine and had it

yield return null

after each of its loops, this works, and prevents my game from hanging, however, now the process is taking WAY longer than it needs to to complete, more like 5-6 seconds, because it only does one loop per frame and then stops, even though there's plenty of time left in the frame to do more processing (assuming a goal of 60fps).

So I added an

if (loopIndex % 5 == 0)

around my yield, to have it process 5 loops before waiting for the next frame.

This works, but doesn't adjust per device the way I'd like, and on slower devices, this leads to freezing yet again.

Is there a way to basically run code as long as your script execution still has time?

 

Technically, yeah, there's always a way. But in practice, in the general sense it can get really hairy.

Also, in the particular case of adding lots of GameObjects, I know they're not linear: last I checked, adding 1000 GameObjects is WAY more than 10x slower than adding 100 Gameobjects, I presume due to inefficiencies in newly-added stuff, not sure/ No way to tell with a closed-source engine like Unity.

Trying to tie into system timers and judge how long things have taken so far will get you a certain distance, but it might break down on certain platforms and not be reliable. plus the timing calculations can get really difficult to judge and tune.

Another dynamic way to do it is to be happy accepting an occasional frame drop, then start by doing 1 object, yield, 2 objects, yield, 3 objects, etc and steadily building that loop count number up. All the while you do that you are monitoring framerate (in code) and the moment it drops below your acceptable limit you knock a few numbers off that loop count value and don't increment it again for a short while.

Obviously you'd never let the count go below 1 otherwise your subtask would stall out. And yet it COULD go as low as 1 on a crummy system, so you have to be willing to accept the worst possible loading time.

It's not pretty. There's a reason we hide behind loading screens.

NOTE: you can always do stuff on another thread... UNLESS it needs to transact with the Unity engine, which obviously you do in this case, and in almost any non-trivial case of Unity engineering.

 

We just talked about this the other day, the best way you can do is (since the engine doesn't have time budget, you have, so it doesn't have infrastructure for you to find out how much time you have until your own deadline) that you measure time from the top of your update method until the WaitForEndOfFrame.

https://docs.unity3d.com/Manual/ExecutionOrder.html

And then you estimate how much time you still have and do some work. Depending on what device you're on. But you will have to test it on actual target devices and obviously go with your gut too.

 

If you can avoid budgeting according to time passed, I really recommend you do. Requiring such precise time measurements is risky; different devices and platforms give different time measurements, sometimes even different OS versions behave differently, and applications can change their time measurement capabilities depending on lots of things in a single runtime.

An easy alternative to using precise timings could be to treat it as something similar to adaptive resolution: Start with an estimated number of iterations per frame; increase it if fps is high, decrease it if it's low. It still can cause some frame drops, and it won't always use all the resources available immediately, but it should be enough for a lot of cases.

That said, if you really want that precision budgeting. You could replace your

if (loopIndex % 5 == 0)

with

if (lastTime - currentTime >= timeBudget)

. You could use Time.realTimeSinceStartupAsDouble to obtain lastTime and currentTime; the docs suggest it can be too imprecise in some devices, although I haven't used enough to be sure of it. Another way of obtaining frame independent time could be to use AudioSettings.dspTime; it's worked very well for me in some cases, although I've not used it for this kind of stuff. Also, I seem to remember an audio has to be running for it to work in some devices; I might be wrong, though. Finally, you could use a native API from IOS or Android; I've never done that, so I can't advise you there, but I know it's possible.

Again, if you can avoid going through all that trouble, I really recommend it.

EDIT
Sorry, I forgot the obvious way of measuring time: the Stopwatch. I've actually seen it in production in .net applications for Windows. I don't know how it works in production in mobile, though; I've only used for development in Unity.

EDIT 2
Sorry! It seems I forgot the entire .net API exists. You may also be able use System.DateTime.Now.Milliseconds to get currentTime and lastTime. That said, if a Stopwatch doesn't work, DateTime, probably won't work either.

 

Thanks for your replies everyone, these are all very interesting options, I think I'm gonna go with this option:

It sounds dynamic enough to work across all platforms, and I don't mind a single frame dropped, probably won't even be noticed by players. I think I'll sort of do an inverse exponential backoff, so that on faster systems it more quickly builds up to a faster pace.

Thanks a lot guys, just wanted to confirm there wasn't some surefire way of handling that everyone uses and I was overlooking!

 

Here's my fairly naive generalized implementation, of course this can be tweaked for your needs, and potentially a smarter algorithm, rather than multiplying the number of iterations processed by 1.25 each attempt, would potentially be preferable:

Code (CSharp):

1.   IEnumerator LagFreeProcessor(Action<int> process, int iterations)
2.   {
3.     var lagFreeIterations = 1;
4.     for (var currentIteration = 0; currentIteration < iterations; currentIteration++)
5.     {
6.       process(currentIteration);
7.       if (currentIteration % lagFreeIterations != 0) continue;
8.       if (1 / Time.smoothDeltaTime < 60.0f)
9.       {
10.         lagFreeIterations = (int) Math.Max(1, lagFreeIterations / 1.25f);
11.       }
12.       else
13.       {
14.         lagFreeIterations = (int) Math.Ceiling(lagFreeIterations * 1.25f);
15.       }
16.       yield return null;
17.     }
18.   }

I also implemented this solution as well:

Code (CSharp):

1.   IEnumerator LagFreeProcessor(Action<int> process, int iterations)
2.   {
3.     for (var currentIteration = 0; currentIteration < iterations; currentIteration++)
4.     {
5.       var startTime = Time.realtimeSinceStartup;
6.       process(currentIteration);
7.       if (Time.realtimeSinceStartup - startTime > 0.001f)
8.       {
9.         yield return null;
10.       }
11.     }
12.   }

I found this one a lot easier to tweak, and it just seemed to work better, although I suspect that constant for how much time you're willing to allocate would have to vary somewhat from platform to platform, depending on how much extra time you have per frame, so that could be problematic.

Used like this:

Code (CSharp):

1. IEnumerator levelLoader = LagFreeProcessor((int indexToLoad) => LoadByIndex(indexToLoad), loadingSpaces.Length);
2. StartCoroutine(levelLoader);

(Yes, in that example I could have just passed in the LoadByIndex function without the surrounding fat arrow function, but I wanted to show the parameter type)

This definitely isn't perfect, it basically accepts that you're gonna drop a decent number of frames, but it prevents you from hanging entirely and lets your loop get up to near full speed without hanging.