Search Unity

Question Trouble getting RunStarted callback in editor using attribute TestRunCallback

Discussion in 'Testing & Automation' started by Tomanonymous, Oct 6, 2023.

  1. Tomanonymous

    Tomanonymous

    Joined:
    Mar 25, 2016
    Posts:
    1
    Good day,

    I am attempting to set up a class to receive callbacks using ITestRunCallback during test started in the editor. I registered my class with TestRunCallback, however I do not seem to be receiving the RunStarted callback. The other 3 callbacks from the interface all work (somewhat) correctly: TestStarted, TestFinished, and even RunFinished. I created an class based on the example provided by Unity, but I seem to be missing a call to RunStarted and I believe a call to the project's test group TestStarted.

    Code (CSharp):
    1. using UnityEngine;
    2. using NUnit.Framework.Interfaces;
    3. using UnityEngine.TestRunner;
    4.  
    5. [assembly: TestRunCallback(typeof(TestCallbackExample))]
    6. public class TestCallbackExample : ITestRunCallback
    7. {
    8.     public void RunStarted(ITest testsToRun)
    9.     {
    10.         Debug.Log($"RUN STARTED - Test Count: { testsToRun.Tests.Count }");
    11.     }
    12.  
    13.     public void RunFinished(ITestResult testResults)
    14.     {
    15.         Debug.Log($"RUN FINISHED - Pass Count: { testResults.PassCount }");
    16.     }
    17.  
    18.     public void TestStarted(ITest test)
    19.     {
    20.         Debug.Log($"TEST STARTED - { test.Name }");
    21.     }
    22.  
    23.     public void TestFinished(ITestResult result)
    24.     {
    25.         Debug.Log($"TEST FINISHED - { result.Name }");
    26.     }
    27. }

    Code compiles and runs without errors, but from the output, it would seem that my class is being registered for the callbacks after the tests have already started. I would expect to have seen "RUN STARTED - Test Count: 1" followed by "TEST STARTED - TestProject" prior to the first output in the log below.

    Output:
    TEST STARTED - Tests.dll
    TEST STARTED - MyTests
    TEST STARTED - test1
    TEST FINISHED - test1
    TEST FINISHED - MyTests
    TEST FINISHED - Tests.dll
    TEST FINISHED - TestProject
    RUN FINISHED - Pass Count: 1


    Some additional troubleshooting details:
    I have tried moving the file both in and out of an editor folder and also tried placing it in the same assembly as my tests. I have a breakpoint set at the missing callback and I can confirm that it does not hit. I am currently using Unity 2022.3.10f1. I have tried searching for other help topics on the subject, but so far it does not seem like anyone else is having this issue.

    Some information on alternatives:
    I am able to create a class using the ICallbacks interface and manually register it with TestRunnerApi to receive the RunStarted(ITestAdapter) callback. However, I would like to use the ITestRunCallback if possible for its compatibility in standalone player. It would appear that this should be possible, but if not, I would appreciate the lesson on what I missed that would have told me such. Much appreciated.
     
    Last edited: Oct 6, 2023