Search Unity

  1. Are you interested in providing feedback directly to Unity teams? Sign up to become a member of Unity Pulse, our new product feedback and research community.
    Dismiss Notice

Run tests before play mode and prevent enter play mode if tests are failing?

Discussion in 'Testing & Automation' started by Xarbrough, Oct 7, 2020.

  1. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,085
    Is it possible with the testing API to run a set of tests before entering play mode? I'd like to do so with only a few tests and if they fail, prevent entering play mode to avoid users having to wait through unity freezing and several loading screens just to realize that something basic broke.

    I've actually implemented this as a custom tool without using the test API. When exiting edit mode, it searches the scene for missing mandatory references, and if any issues are found, the enter play mode is cancelled and an error is logged. This has helped my team in a lot of cases and we'd like to continue expanding on this tool.

    My idea now was to incorporate this is a test that could be run manually via the test runner, but also runs automatically before entering the scene (best as a user preference).

    If this is not possible at the moment, that's my feature suggestion. ;)
     
    JoNax97 likes this.
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,483
    Well, you can run tests via the TestRunnerApi, and you can make them happen synchronously by setting the execution settings appropriately - I demoed this technique in the talk at Unite Copenhagen.

    What I'm less sure about is how to integrate it with the process of entering play mode. You could try subscribing to EditorApplication.playModeStateChanged, but I'm not sure if this callback would fire before entering play mode or after it.
     
  3. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,085
    Thanks for the pointers! I think that’s enough for me to get this working. I’ll try it out tonight and post my results here. :)
     
  4. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,085
    It's working out pretty well:

    Code (CSharp):
    1. using NUnit.Framework;
    2. using UnityEditor;
    3. using UnityEditor.TestTools.TestRunner.Api;
    4. using UnityEngine;
    5.  
    6. namespace Tests
    7. {
    8.     public class SceneValidationTest
    9.     {
    10.         [InitializeOnLoadMethod]
    11.         private static void Initialize()
    12.         {
    13.             EditorApplication.playModeStateChanged += OnPlayModeStateChanged;
    14.         }
    15.  
    16.         private static void OnPlayModeStateChanged(PlayModeStateChange change)
    17.         {
    18.             if (change == PlayModeStateChange.ExitingEditMode)
    19.             {
    20.                 var testRunner = ScriptableObject.CreateInstance<TestRunnerApi>();
    21.                 var filter = new Filter()
    22.                 {
    23.                     testMode = TestMode.EditMode,
    24.                     categoryNames = new[] { "SceneValidation" }
    25.                 };
    26.                 var resultHandler = new ResultHandler();
    27.                 testRunner.RegisterCallbacks(resultHandler);
    28.                 testRunner.Execute(new ExecutionSettings(filter)
    29.                 {
    30.                     runSynchronously = true
    31.                 });
    32.                 if (resultHandler.result.TestStatus == TestStatus.Failed)
    33.                 {
    34.                     Debug.LogError("A validation test has failed. Aborting enter play mode.");
    35.                     EditorApplication.isPlaying = false;
    36.                 }
    37.                 else
    38.                 {
    39.                     Debug.Log("All tests were passed.");
    40.                 }
    41.  
    42.                 Object.DestroyImmediate(testRunner);
    43.             }
    44.         }
    45.  
    46.         private class ResultHandler : ICallbacks
    47.         {
    48.             public ITestResultAdaptor result { get; private set; }
    49.  
    50.             public void RunFinished(ITestResultAdaptor result)
    51.             {
    52.                 this.result = result;
    53.             }
    54.  
    55.             // No interface segregation or separate events?
    56.             public void RunStarted(ITestAdaptor testsToRun) { }
    57.             public void TestFinished(ITestResultAdaptor result) { }
    58.             public void TestStarted(ITestAdaptor test) { }
    59.         }
    60.  
    61.         [Test]
    62.         [Category("SceneValidation")]
    63.         public void SceneValidation_AlwaysFails()
    64.         {
    65.             Assert.Fail("This test is only a demo and always fails.");
    66.             //Assert.Pass();
    67.         }
    68.     }
    69. }
    70.  
    This does exactly what I want in only a few clean lines of code, so I'm really happy with this solution.

    I'm still wondering why the callback listener is an interface and I have to create this container object with the stub implementations for all callbacks, instead of subscribing a specific listener to an event. Not complaining but I'm curious what the design reason is.
     
    Last edited: Oct 9, 2020
    Whatever560 likes this.
unityunity