Unity Test Runner is much slower if Unity is minimized or the screen is locked

3 minutes in focus VS 30 min with locked screen on Windows 10
Unity 2017.4.16

 

I wonder what are your results if you run your tests from the command line.

 

also around 3-4 minutes

 

Well, there is a workaround... dont run tests and then minimize

 

This is very visible in 2018.4.2f1. I've got a two-monitor setup, and if I tab to the other monitor, the speed of the progress bar immediately slows down massively.

I don't quite have the time to test 2018.4.3f1 right now, but unless you think the fix hit in that exact patch, I'd say it still there in 2018.

 

It's a bit hard, it doesn't seem consistent.

I've only been able to repro it in our full game, not in a repro project, and only sometimes - it seems that it happens when Windows Defender is doing a scan.

Anyways, that means that it's probably an OS issue? If the system is low on resources, it makes sense that it prioritizes the window that has focus over windows that doesn't.

The difference is pretty staggering - 10 seconds when Unity has focus, 60 when it doesn't. But, as I said, only reproducible when there's a virus scan going on, and only in one, large project.

I'm not going to spend any more time trying to create a repro, as this doesn't really affect me that much. But it does for sure happen, it's just hard to tell why and when.

Here's a gif of it in action. The cancel button has a blue outline when Unity has focus:

 

That's already done. Rider has started giving an option to automatically configure the repo and related files to be excluded by Defender.

I have tried to exclude Unity, but I can't quite figure out if you're supposed to exclude the app name (Unity) or the executable name (Unity.exe), or the path to the executable or something else. Any insight?

 

I am seeing this issue in Unity 2022.2.17f1 and Rider 2023.1 (April 4th).

When I run all tests, I get about 5-7 times slower test runs in Rider vs TestRunner:

TestRunner: 3-4s
Rider: ~20s

When I start to run all tests in Rider and immediately tab into Unit Editor, the tests finish in 4s as expected.

I've opened an issue here: https://youtrack.jetbrains.com/issu...slower-in-Rider-vs-TestRunner-in-Unity-Editor

I have tried several Unity and Windows settings (see issue details) but none made a difference.

 

UPDATE: I checked the reverse situation: running TestRunner in the background.

Indeed TestRunner runs a lot slower when immediately tabbing to another app after clicking on "Run All" in the Unity Editor. The difference was 2.5s focused Unity editor vs 15s Unity edit in background in three subsequent runs. With 6x slower timings this seems just the same difference I was observing running tests in Rider.

Thus this seems to be a Unity issue, or worse, one regarding Windows 11 behaviour slowing down background processes. I will submit a report.

 

WORKAROUND #1: Keep moving the mouse while tests run in the background. There are scripts/tools that can do this for you. HOWEVER this only works when the mouse is moved over the Unity Editor window, so not really applicable as it does not speed up running Unity TestRunner tests from the IDE (Rider).

WORKAROUND #2: I noticed that changing Preferences => General => Busy Progress Delay value by clicking & dragging on the label will make tests run faster in Rider (5s instead of 20s). Note that you have to drag to change the value, entering a value directly won't work. Also it does not matter what the value is or how much you change it. I hope to find out what setting/method this calls behind the scenes in order to call this every time tests start, but had no luck so far.

I also tried a few more Windows settings and opened a superuser question in hopes for a fix or workaround.

I did some tests with Preferences => General => Interaction Mode as I noticed that TestRunner execution speed depends on it. Here are results for my project running all tests:

| Interaction Mode | Time to "Run all" |
| Default | 2.9s |
| Monitor Refresh Rate (8 ms) | 3.7s |
| No Throttling | 2.1s |

No Throttling (or: Custom with 0 ms Frame Throttling) makes tests run about 30-40% faster! However I don't wish throttling to be enabled all the time. I'd rather hope Unity could just generally disable throttling during test runs if we can't do it ourselves.

 

I tried changing the corresponding EditorPrefs key "EditorBusyProgressDialogDelay" before running tests but this apparently does not trigger the code that runs when changing the value in Preferences which apparently makes the tests run faster.

If anyone has any idea how to disassemble the editor code for the General Preferences or where to find it (string search didn't reveal anything) it may help because then I could actually call that (possibly internal) method.

 

UPDATE: I can confirm that the "background slows down tests" issue no longer occurs in Unity 2023.1.0b14 (beta) and 2023.2.0a12 (alpha). TestRunner runs just as fast in background as it does in foreground. Speeding up tests within Rider however seem to require the following script.

I wrote this script for use in test assemblies that automatically sets Interaction Mode to "No Throttling" while tests are running. This reliably speeds up TestRunner execution by 20-40% and beginning with the 2023 tech streams it also considerably (~5x) speeds up running tests from within Rider.

Here's the script in full, it should work if you put it in any folder that's compiled by a unit test Assembly Definition

Code (CSharp):

1. // This work is Public Domain (CC0) - "No Rights Reserved"
2. // License: https://creativecommons.org/publicdomain/zero/1.0/
3.  
4. using System.Reflection;
5. using UnityEditor;
6. using UnityEditor.TestTools.TestRunner.Api;
7. using UnityEngine;
8.  
9. namespace CodeSmile.Tests.Utilities
10. {
11.     [InitializeOnLoad]
12.     public class FasterTests
13.     {
14.         static FasterTests() => ScriptableObject.CreateInstance<TestRunnerApi>().RegisterCallbacks(new Callbacks());
15.  
16.         private class Callbacks : ICallbacks
17.         {
18.             private const string ApplicationIdleTimeKey = "ApplicationIdleTime";
19.             private const string InteractionModeKey = "InteractionMode";
20.  
21.             private int m_UserApplicationIdleTime;
22.             private int m_UserInteractionMode;
23.  
24.             private static void UpdateInteractionModeSettings()
25.             {
26.                 const string UpdateInteractionModeMethodName = "UpdateInteractionModeSettings";
27.  
28.                 var bindingFlags = BindingFlags.Static | BindingFlags.NonPublic;
29.                 var type = typeof(EditorApplication);
30.                 var method = type.GetMethod(UpdateInteractionModeMethodName, bindingFlags);
31.                 method.Invoke(null, null);
32.             }
33.  
34.             public void RunStarted(ITestAdaptor testsToRun) => SpeedUpTestRunner();
35.             public void RunFinished(ITestResultAdaptor result) => ResetInteractionMode();
36.             public void TestStarted(ITestAdaptor test) {}
37.             public void TestFinished(ITestResultAdaptor result) {}
38.  
39.             private void SpeedUpTestRunner()
40.             {
41.                 Debug.Log("Set Interaction Mode to 'No Throttling' during tests.");
42.                 SetInteractionModeToNoThrottling();
43.                 UpdateInteractionModeSettings();
44.             }
45.  
46.             private void ResetInteractionMode()
47.             {
48.                 Debug.Log("Reset Interaction Mode to user settings.");
49.                 SetInteractionModeToUserSetting();
50.                 UpdateInteractionModeSettings();
51.             }
52.  
53.             private void SetInteractionModeToNoThrottling()
54.             {
55.                 m_UserApplicationIdleTime = EditorPrefs.GetInt(ApplicationIdleTimeKey);
56.                 m_UserInteractionMode = EditorPrefs.GetInt(InteractionModeKey);
57.                 EditorPrefs.SetInt(ApplicationIdleTimeKey, 0);
58.                 EditorPrefs.SetInt(InteractionModeKey, 1);
59.             }
60.  
61.             private void SetInteractionModeToUserSetting()
62.             {
63.                 EditorPrefs.SetInt(ApplicationIdleTimeKey, m_UserApplicationIdleTime);
64.                 EditorPrefs.SetInt(InteractionModeKey, m_UserInteractionMode);
65.             }
66.         }
67.     }
68. }