How to pass PlayMode test early instead of waiting for timeout?

In NUnit we can simply call Assert.Pass, but Unity does not seem to handle the SuccessException. Is there anything I need to configure to make this work or is it simply not supported?

Are there other ways of ending a coroutine test during PlayMode early and count the test as passed?

 

I can post the complete test later, but for now, maybe I already found the issue: I have been calling Assert.Pass within a lambda callback hooked up to the SceneManager.sceneLoaded event. I want to test if a special scene is actually loaded within a timeout of e.g. 5 Seconds. Instead of having to wait the entire time, I wanted to pass the test early when I receive the loaded event with the correct scene name, but maybe this is the reason why it did not catch the exception.

 

Here is simplified version of the issue:

Code (CSharp):

1. public class SceneLoaderTests
2. {
3.     [UnityTest]
4.     public IEnumerator DummyTest_LoadSceneAsync_MainSceneIsLoaded()
5.     {
6.         // When a scene was loaded, check if we can quit the test early.
7.         SceneManager.sceneLoaded += (scene, mode) =>
8.         {
9.             // If the correct scene was loaded, simply stop the test successfully.
10.             // However, this does not work and only logs "SuccessException",
11.             // which fails the test.
12.             if (scene.name == "Main" && mode == LoadSceneMode.Single)
13.                 Assert.Pass();
14.         };
15.  
16.         SceneManager.LoadSceneAsync("Main");
17.  
18.         // We can wait the full timeout duration and assert, which works
19.         // but wastes a lot of time waiting for nothing.
20.         yield return new WaitForSeconds(5f);
21.         Assert.IsTrue(SceneManager.GetSceneByName("Main").isLoaded);
22.         yield return null;
23.     }
24.  
25.     [UnityTest]
26.     public IEnumerator DummyTest_SimplePass()
27.     {
28.         // This test passes as expected.
29.         Assert.Pass();
30.         yield return null;
31.     }
32. }

The DummyTest_SimplePass demonstrates, that the Assert.Pass call indeed works in this simple case, however in the test above, when called within the SceneManager.sceneLoaded callback, it does not pass the test. Instead, it logs the SuccessException and fails the test.

Spoiler: Exception Stacktrace

SuccessException
NUnit.Framework.Assert.Pass (System.String message, System.Object[] args) (at <59819be142c34115ade688f6962021f1>:0)
NUnit.Framework.Assert.Pass () (at <59819be142c34115ade688f6962021f1>:0)
Tests.SceneLoaderTests+<>c.<DummyTest_LoadSceneAsync_MainSceneIsLoaded>b__0_0 (UnityEngine.SceneManagement.Scene scene, UnityEngine.SceneManagement.LoadSceneMode mode) (at Assets/PlayModeTests/SceneLoaderTests.cs:17)
UnityEngine.SceneManagement.SceneManager.Internal_SceneLoaded (UnityEngine.SceneManagement.Scene scene, UnityEngine.SceneManagement.LoadSceneMode mode) (at C:/buildslave/unity/build/Runtime/Export/SceneManager/SceneManager.cs:244)

 

You should use this instead of that "WaitForSeconds(5f)".

Note that with the below message, the call to replace that would be:

Code (CSharp):

1.  
2. yield return StartCoroutine(WaitFor(() => SceneManager.GetSceneByName("Main").isLoaded), 5f);
3.  

This will wait for up to 5 seconds for the main scene to be loaded. Or increase the timeout value, but it offers a dynamic wait. Please note that this is part my Trilleon Automation/Unit Test Framework so there will need to be some edits if you use the following code directly in your current framework.

If you switched to Trilleon for your unit testing and automation, none of this will be a problem at all for you. Give it a try. Compare it to what you are using now. I think it will be much easier.

Code (CSharp):

1.         /// <summary>
2.         /// Waits for a single value to not be null or default. Waits for a list of values to return more than 0. Waits for object(s) to be active, visible, and interactable.
3.         /// </summary>
4.         /// <param name="propertyExpression">Lamba expression representing the check you wish to perform with each iteration of the loop. This is syntactically as simple as "() => SomeCondition && SomeOtherCondition", for example.</param>
5.         public IEnumerator WaitFor(Func<bool> condition, string optionalOnFailMessage = "", float timeout = TIMEOUT_DEFAULT, params int[] testCaseIds) {
6.  
7.             PreCommandCheck();
8.  
9.             isDeepDive = true;
10.             float timer = 0;
11.             while(!condition.Invoke() && timer <= timeout) {
12.  
13.                 yield return StartCoroutine(WaitRealTime(1f));
14.                 timer++;
15.  
16.             }
17.             if(timer > timeout) {
18.  
19.                 if(!isTry) {
20.  
21.                     yield return StartCoroutine(Q.assert.Fail(string.IsNullOrEmpty(optionalOnFailMessage) ? "Conditional wait (WaitFor) timed out." : optionalOnFailMessage));
22.  
23.                 }
24.  
25.             }
26.             isDeepDive = false;
27.             yield return StartCoroutine(Q.game.WaitForNoLoadingIndicators());
28.  
29.             if(!string.IsNullOrEmpty(optionalOnFailMessage)) {
30.  
31.                 yield return StartCoroutine(Q.assert.Pass(optionalOnFailMessage));
32.  
33.             }
34.  
35.             if(testCaseIds.Length > 0) {
36.  
37.                 Q.assert.MarkTestRailsTestCase(timer > timeout, testCaseIds);
38.  
39.             }
40.  
41.             PostCommandCheck();
42.             yield return null;
43.  
44.         }

 

I know this won't fix this issue in general but you can use a yield statement

Code (CSharp):

1. yield return SceneManager.LoadAsync()

to wait until the level isn't loaded. An error will be thrown anyway if the level isn't loaded anyway e.g. build settings.

 

Is quit old, but if someone comes here looking for solution mine was not using the troublesome sceneloaded.

Instead just put a simple waiter something like this:

Code (CSharp):

1. TestUtilities.SetupScene(scenePath);
2.  
3. while (SceneManager.GetSceneByPath(scenePath).isLoaded == false)
4. {
5.      yield return null;
6. }
7. //After scene loaded wait one frame to be safe.
8. yield return null;
9.  
10.  
11.