Invoke almost 5x faster than Coroutine (Performance Benchmarking)

Hey guys,

I had always considered Unity Invoke method to be slow because it was based on reflection. Have read many forum posts recomending the use of StartCoroutine + yield return new WaitForSeconds(delay) for the same matter. It made sense to me, and I've been using coroutines for a long time. Today I was doing some benchmarcking and decided to see exactly how faster coroutines were. For my surprise coroutines performed almost 5 times slower than Invoke.

First Test:

• Used System.Diagnostics.Stopwatch to get accurate measurements.
• Called Invoke("Method", 0.5f) a million times, stopped the watch exactly when all invokes were executed.
• Then called StartCoroutine(Method()); a million times. Inside the method put a yield return new WaitForSeconds(0.5f). Stopped the watch after all coroutines finished.

Results:

• Invoke: 1058ms (total) - 500ms (delay) = 558ms (overhead)
• Coroutine: 3031ms (total) - 500ms (delay) = 2531ms (overhead)
  • 2531ms / 558ms = 4.5 times longer

Second Test:

• Called 10k Invoke("Method", 0.5f) per frame.
• Called 10k StartCoroutine(Method()); per frame. Again with the yield return new WaitForSeconds(0.5f) inside.

Results:

• Invoke: running at 60 FPS, with few GC spikes.
• Coroutine: running at ~30 FPS, with frequent GC spikes.

Profiling 10K Invokes called every Frame Update:


Profiling 10K Coroutines called every Frame Update:


Conclusion:
I'm not a professional benchmarker, but in my tests Invoke performed almost 5x faster than starting coroutines, and also seems to generate less garbage. Later I thought it could be some issue with Unity Editor, but I got similar results when building for Windows, Android and Web. I even tried to feed random methods for the Invoke, instead of using a string literal for the method name, and again got similar results.

Final Thoughts:
I would like to understand better how both methods works inside the Unity internals. Because it seems to me that calling a funcition by reference should always be faster than calling it thought its name written on a string (reflection).

Can anyone give me some light here?

 

Sharing code could help optimise both approaches, but personally I never equate coroutines to performance. If anyone is saying that, they're pretty bad developers. What coroutines/invoke actually do is provide convenience, so they shouldn't be used for performance - almost every other pattern you could take will be faster, especially when scaled.

Is the goal to optimise these because of the convenience or to get best performance? because you don't want either for best perf.

 

Thank you for replying! I will organize my code and post it here.

I try to avoid using both actually. I develop mostly for mobile, so performance is always a matter.

The case I use them the most is when I need something to happen X seconds after. The alternative I could think of, would be to have something like this on the Update: if (waitingForEvent && Time.time > eventTargetTime). It would need to be checked every frame.

 

Code (CSharp):

1. using System.Collections;
2. using System.Diagnostics;
3. using UnityEngine;
4.  
5. public class Experiment1 : MonoBehaviour {
6.     public int repeatNumber = 1000000;
7.     public float delaySeconds = 0.5f;
8.     int invokeCounter, coroutineCounter;
9.     long invokeCallsElapsed, invokeExecutionElapsed, coroutineCallsElapsed, coroutineExecutionElapsed;
10.     Stopwatch stopWatch;
11.  
12.  
13.     // Method to be invoked
14.     void InvokeMethod() {
15.         // Counter
16.         invokeCounter++;
17.         if (invokeCounter >= repeatNumber) {
18.             InvokeFinished();
19.         }
20.     }
21.  
22.     // Testing Invoke
23.     void InvokeTest() {
24.         invokeCounter = 0;
25.  
26.         // Start watch
27.         stopWatch = new Stopwatch();
28.         stopWatch.Start();
29.  
30.         // Invoke InvokeMethod repeatNumber times
31.         for (int i = 0; i < repeatNumber; i++) {
32.             Invoke("InvokeMethod", delaySeconds);
33.         }
34.  
35.         // Finished calls
36.         invokeCallsElapsed = stopWatch.ElapsedMilliseconds;
37.     }
38.  
39.     // Finished executing
40.     void InvokeFinished() {
41.         stopWatch.Stop();
42.         invokeExecutionElapsed = stopWatch.ElapsedMilliseconds;
43.  
44.         // Report
45.         UnityEngine.Debug.Log(string.Format("Invoke took {0}ms for calls and {1}ms to complete execution.", invokeCallsElapsed, invokeExecutionElapsed));
46.     }
47.  
48.  
49.     // Method to be started
50.     IEnumerator CoroutineMethod() {
51.         yield return new WaitForSeconds(delaySeconds);
52.  
53.         // Counter
54.         coroutineCounter++;
55.         if (coroutineCounter >= repeatNumber) {
56.             CoroutineFinished();
57.         }
58.     }
59.  
60.     // Testing Invoke
61.     void CoroutineTest() {
62.         coroutineCounter = 0;
63.  
64.         // Start watch
65.         stopWatch = new Stopwatch();
66.         stopWatch.Start();
67.  
68.         // Start CoroutineMethod repeatNumber times
69.         for (int i = 0; i < repeatNumber; i++) {
70.             StartCoroutine(CoroutineMethod());
71.         }
72.  
73.         // Finished calls
74.         coroutineCallsElapsed = stopWatch.ElapsedMilliseconds;
75.     }
76.  
77.     // Finished executing
78.     void CoroutineFinished() {
79.         stopWatch.Stop();
80.         coroutineExecutionElapsed = stopWatch.ElapsedMilliseconds;
81.  
82.         // Report
83.         UnityEngine.Debug.Log(string.Format("Coroutines took {0}ms for calls and {1}ms to complete execution.", coroutineCallsElapsed, coroutineExecutionElapsed));
84.     }
85.  
86.     void Update() {
87.         // Run Invoke test
88.         if (Input.GetKeyDown(KeyCode.Alpha1)) {
89.             InvokeTest();
90.         }
91.  
92.         // Run Coroutine test
93.         if (Input.GetKeyDown(KeyCode.Alpha2)) {
94.             CoroutineTest();
95.         }
96.     }
97. }

 

This is the first test I described, you can press the "1" key to run the Invoke Test and the "2" to run the Couroutine Test.

 

Yes, Invoke is bad for performance (generally speaking), but Coroutines are bad as well. Neither of these are for performance as @hippocoder said.
And these have two different goal as well. The Invoke is to provide "variable" function calls, which means you can decide on run-time.
Co-routines are for distributing (moderately) lengthy calculations among subsequent frames.

But both of them have tremendous overhead if we compare them to a simple static function call.
Keep in mind that we are talking in general, and we are talking about performance and optimization.

Never ever optimize prematurely. Optimize when the problem arises and the thing you have __found__ problematic.
Don't _think_ something is wrong, look for it, find it, _know_ that it is wrong or causes problems.

I myself the only thing I prematurely optimize are:
- I do not use properties only if there is a need to do other functions when I add/get a value (function calls)
- I do not use delegates/events/actions in the hot path also do not use any virtual calls (no overloaded function calls on the hot path)
- I try to avoid using 3rd-party script assets in the update loop
- I have my own update loop (one MonoBehaviour calls a scriptable object's Tick function and that calls everything needed)

Everything else is on the 'I will fix when it becomes a problem'.

 

Code (CSharp):

1. using System.Collections;
2. using UnityEngine;
3.  
4. /* Invoke vs Coroutine Performance Test
5. * Makes repeatNumber calls every frame
6. * Press space bar to alternate testing Invokes/Coroutines
7. */
8. public class Experiment2 : MonoBehaviour {
9.     public int repeatNumber = 10000;
10.     public float delaySeconds = 0.5f;
11.     int invokeCounter, coroutineCounter;
12.     bool testingInvoke;
13.  
14.     // Method to be invoked
15.     void InvokeMethod() {
16.         // Just to make sure it's being executed
17.         invokeCounter++;
18.     }
19.  
20.     // Testing Invoke
21.     void InvokeTest() {
22.         // Invoke InvokeMethod repeatNumber times
23.         for (int i = 0; i < repeatNumber; i++) {
24.             Invoke("InvokeMethod", delaySeconds);
25.         }
26.     }
27.  
28.  
29.     // Method to be started
30.     IEnumerator CoroutineMethod() {
31.         yield return new WaitForSeconds(delaySeconds);
32.  
33.         // Just to make sure it's being executed
34.         coroutineCounter++;
35.     }
36.  
37.     // Testing Coroutine
38.     void CoroutineTest() {
39.         // Start CoroutineMethod repeatNumber times
40.         for (int i = 0; i < repeatNumber; i++) {
41.             StartCoroutine(CoroutineMethod());
42.         }
43.     }
44.  
45.  
46.     void Update() {
47.         // Alternate Invoke and Coroutine testing
48.         if (Input.GetKeyDown(KeyCode.Space)) {
49.             testingInvoke = !testingInvoke;
50.         }
51.  
52.         // Run tests every frame
53.         if (testingInvoke) {
54.             InvokeTest();
55.         } else {
56.             CoroutineTest();
57.         }
58.  
59.         // Report
60.         float fps = 1f / Time.smoothDeltaTime;
61.         UnityEngine.Debug.Log(string.Format("{0} -> FPS: {1}", testingInvoke ? "Invoke" : "Coroutine", fps));
62.     }
63. }

 

LurkingNinjaDev: Thanks for the response!

I agree with you, both should be avoided for performance. I never use them for stuff that happens every frame. My goal was trying to figure out which is less heavy for using when you need something to happen X seconds later.

Thinking about convenience, it's a lot easier to just call a Invoke or Coroutine then needing to write a whole loop to manage this delayed calls.

I'm also not in favor of optimizing early, but for this specific usage (delayed calls) I can use both. I know both are heavy calls, but if one performs significantly better than the other I will give preference to this one.

 

Are virtual calls that slow too? I know they have some overhead, but having inherited classes can be very useful for organizing behaviors.

 

They have overhead. On the update, every tick counts if you are writing an action game. Of course if I make a tic-tac-toe or some board game or some lightweight one, it isn't a problem.
But I found these problems very long and hard to refactor if problems arise, so I just developed the habit to avoid these as much as possible.

 

Yes, that's what coroutines do under the hood. They check every frame and the perf is not as good due to the allocations.

Then you could look on asset store, github or online for improved coroutines. Random example: https://www.assetstore.unity3d.com/en/#!/content/54975.

You can also just manage the memory of your coroutines. This will at least remove the allocations:

Code (csharp):

1.  
2. WaitForSeconds delay = new WaitForSeconds(1f);
3.  
4. while (!isComplete)
5. {
6.    yield return delay;
7. }
8.  

Read Unity's own advice: https://docs.unity3d.com/Manual/BestPracticeUnderstandingPerformanceInUnity3.html

 

I just made a little experiment with checking if (Time.time > eventTargetTime). I used to think it would have a big impact on performance because the need to run every frame. But it still turns out to be a lot more performance wise than Invoke or Coroutine. I was able to run 500k if (Time.time > eventTargetTime) every frame while keeping at steady 60 FPS, without a single GC peak.

Yes, this way you only allocate the WaitForSeconds a sigle time.

Makes sense, it was never meant to be used for delayed calls.

Sometimes though, it's very practical to use Invoke(method, time). I guess Invoke is more suitable than Couroutine for this usage, when you need to call a method after a delay. But I dislike having to call a method by a string, seems to me like a very bad programming practice. You cannot refactor, for instance, and it seems to make debugging harder too.

 

Do you get any better performance if you cache the WaitForSeconds?

YieldInstructions are a class, so they will be subject to the normal GC issues. Creating a million instances of even the most light weight class is going to thrash your garbage collector.

 

That's true!

Tried caching it, performance was improved by 258ms (for 1 million calls)... It's almost a 10% improvement. But it is still much behind Invoke.

 

Kinda off topic, but is there any point in using wait for end of frame when returning null is basically the same but cheaper?

 

Null waits to the next frame. WaitForEndOfFrame waits to the end of the frame. They produce different results.

 

Makes sense, so use null if you don't care where in the frame it happens?

 

Yup.

 

My guess is it's calling startcoroutine every time that is causing you problems.
If you know you need to do it x times. Just start the coroutine once.
Cache the waitfor and loop x times. Yielding each time through the loop.

But if you really need to do something like this I think there are much better designs than using either method.

To me Coroutines are really just used to make it easier to write code that needs to be executed over multiple frames.
This has nothing to do with performance. Other than I can spread the work across multiple frames.

 

Pretty sure he was doing it every frame as its too hard to measure the performance of a single call, not because that's his actual use case

 

This is an interesting read!

So what is best practise now for and actual Unity Version 2017.x and above for starting a method, which acutally does something? Like creating chunks for a landscape or populating Monsters (gameobjects) in the Scene with some calculations?

And is it the same for Unity 2018, too or do we habe better alternatives then?

Thanks a lot! Very interesting stuff in here.

 

ECS > ALL

 

this is true to a point, but with coroutine when you stop it, it stops, with invoke it runs the function one more time after you stop it

 

Why? As far as I remember those things don't have noticeable performance overhead.

This is a nice practice, but it's also a rare case where I think singletons are a better option.
Do you want to explain how do you benefit from using a scriptable object?

 

Good to know I always preferred Invoke as it was just so much simpler and easier to use, the only downside being potential Typos for the function name.

If Unity added an Invoke(methedName, delay, repeatdelay) without the strings would it be even faster and easier to use than co-routines.

And if it used delegates it could still be changed dynamically but in a type safe manner.

 

This would be really nice to have, but for now we have to use custom implementations. At least this is trivial to implement.

If you want to use Unity's Invoke function with function references instead of strings you can create an overload so that it takes a delegate/action as a parameter and uses reflection to pass the function name to Unity's Invoke. But this is a lot of work for little gain.

 

How would you do that? Would that solve the reflection overhead in calling Invoke ("MethodeStringName")
Can this be solved in a clever way? Maybe then it would really make sense to use Invoke in some cases instead of using StartCoroutine...

I am using StartCoroutine in my chunk based voxel engine a lot, each Performance increase would be great, I did not have the idea yet Invoke could be better than StartCoroutine.

 

If you look for efficiency don't use coroutine or invoke. Period.
Therefore it is illogical to pursue a fallacy.

When ECS ships, I recommend you look at it + jobs + burst. There is your efficiency!

 

@Firlefanz73 The most cpu efficient way of handling this would probably be to have a MonoBehaviour class with a list of pending actions (paired with timeouts), where you would check timeouts in the update function.
Instead of adding your actions to Unity's Invoke list, you'll just use a custom object.
It'll work pretty much the same way, expect this way you avoid reflection, which should make it more efficient.

 

This is not that correct, their main usage is to execute async code in a synchronous manner.

 

always thought it was to make your codebase nearly impossible to maintain and throw away any chance of refactoring and being able to relie on debugging tools.

 

Is this the best way to call performance heavy methods ? Like spherecast all

Code (CSharp):

1.  
2. time = 0;
3.  
4. //call this in update
5. time += Time.deltaTime;
6.         while (time > 0.2)
7.         {
8.             time -= 0.2f;
9.             InvokeAllMethods();
10.         }

 

No, avoid invoke. Reasons to avoid have been mentioned in this thread.

Both coroutines and invoke are not good for efficiency. Call the method normally.

You shouldnt need to use coroutines as c# in unity has had proper async for a long time now. Just use an async method instead if you want to do something across a number of frames. Otherwise, just call the method.

 

No. Profile your code if you care about performance. Do not guess.

One of the (most?) useful application of coroutines is not to provide asynchronous execution, but to spread linear task over multiple frames while knowing exactly when then next frame happens. While they can be used for asynchronous execution, their primary useful application is not that, and in practice one could say they implement green threads.

Async tasks do not seem to be able to provide the same functionality:
http://www.stevevermeulen.com/index.php/2017/09/using-async-await-in-unity3d-2017/

 

This is key, never expend your most essential commodity, time on something that might not be a performance problem.

Remember the 80/20 rule, 80% of your code runs fine, only 20% needs to be optimized and the only way you can detect the real 20% is by profiling your code.

 

I am sorry, "InvokeAllMethods();" mentioned in my code is just a simple function, that contains all functions of the script that I want to call. It has nothing to do with Invoking. Sorry for confusing function name.