Official UTF 1.4

Will 1.4 still work with 2021.3?

 

Installing this worked! That's great, we've been stuck on 2.0.1-pre.18 for ages, as any switch to a 1.x version would cause our project to not compile due to internal compiler errors until we switched back to 2.0.1-pre.18 and deleted the library.

Is there any entries for testing on the Roadmap page? I'm still waiting for any work put into making tests that needs to run in a specific scene not be incredibly painful.

 

Coming from v2.8.1-preview

TestRunnerApi.IsAnyRunActive() and TestRunnerApi.CancelAllTestRuns() have been removed.

How can we know if any run is active and cancel all runs now?

Also, RetrieveTestList no longer accepts an ExecutionSettings object?
How can I retrieve tests for a specific category?

 

hey @Warnecke, thanks for the constant feedback here.
I'm using the version "1.4.3", and running async Task methods on Editor Mode will hang most of the time. I have to constantly move the mouse and click on any view of the Unity Editor to get the Tests to get finished.
Do you know if that is a known issue and if the team is working on a solution for it?

It is exactly the issue posted by @PhantomGingerGames a while ago here:
- https://forum.unity.com/threads/unity-test-framework-2-0-ready-for-feedback.1230126/#post-7910305

Image from the post above:

 

Okay, I've made a simple utility to solve that issue until it gets solved on the Unity side.
I basically added a logic to trigger an infinity loop to execute `EditorApplication.QueuePlayerLoopUpdate()` on every tick whenever the TestRunner starts to run Editor mode tests and then break the loop when the TestRunner tests get finished. Script follows below for anyone experiencing the same issue:

Code (CSharp):

1. using System.Threading;
2. using System.Threading.Tasks;
3. using UnityEditor;
4. using UnityEditor.TestTools.TestRunner.Api;
5. using UnityEngine;
6.  
7. namespace TestRunnerUtils
8. {
9.     [InitializeOnLoad]
10.     public static class TestRunnerHangingOnEditorTestsFixer
11.     {
12.         private static TestRunnerApi api;
13.         private static CancellationTokenSource cancellationTokenSource;
14.  
15.         static TestRunnerHangingOnEditorTestsFixer()
16.         {
17.             if (Application.isBatchMode)
18.             {
19.                 // This script should not be used in batch mode.
20.                 return;
21.             }
22.  
23.             SetupListeners();
24.         }
25.  
26.         private static void SetupListeners()
27.         {
28.             Object.DestroyImmediate(api);
29.             api = ScriptableObject.CreateInstance<TestRunnerApi>();
30.             api.RegisterCallbacks(new TestRunnerCallbacks());
31.         }
32.  
33.         private class TestRunnerCallbacks : ICallbacks
34.         {
35.  
36.             public void RunStarted(ITestAdaptor testsToRun)
37.             {
38.                 Debug.Log($"TestRunner in {testsToRun.TestMode} mode started");
39.  
40.                 if (testsToRun.TestMode != TestMode.EditMode)
41.                 {
42.                     return;
43.                 }
44.  
45.                 cancellationTokenSource?.Cancel();
46.                 cancellationTokenSource = new CancellationTokenSource();
47.                 TriggerEditorUpdateLoop(cancellationTokenSource.Token);
48.             }
49.  
50.             public void RunFinished(ITestResultAdaptor result)
51.             {
52.                 Debug.Log("TestRunner finished");
53.                 cancellationTokenSource?.Cancel();
54.             }
55.  
56.             public void TestStarted(ITestAdaptor test)
57.             {
58.                 // Don't need to do anything here.
59.             }
60.  
61.             public void TestFinished(ITestResultAdaptor result)
62.             {
63.                 // Don't need to do anything here.
64.             }
65.  
66.             private static void TriggerEditorUpdateLoop(CancellationToken cancellationToken = default)
67.             {
68.                 TriggerEditorUpdateLoopAsync(cancellationToken).ConfigureAwait(false);
69.             }
70.  
71.             private static async Task TriggerEditorUpdateLoopAsync(CancellationToken cancellationToken = default)
72.             {
73.                 while (cancellationToken.IsCancellationRequested == false)
74.                 {
75.                     await Task.Yield();
76.                     EditorApplication.QueuePlayerLoopUpdate();
77.                 }
78.             }
79.         }
80.     }
81. }

 

@Warnecke One really important feedback I would like mention is please dun backport PlayerHeartBeatTask at 2.0 which is regression that will fail my long running test. I have test needs to take a lot of time to run.

 

Do async tests work with [UnityTest]? Or now that we don't need to use IEnumerator, can we remove [UnityTest] and put [Test] instead? Because in the docs it doesn't appear nor is it mentioned anywhere:

https://docs.unity3d.com/Packages/com.unity.test-framework@1.4/manual/reference-async-tests.html

https://docs.unity3d.com/Packages/c...1.4/manual/reference-attribute-unitytest.html

 

async Task works only with [Test].
and there is no need to remove [UnityTest] because otherwise everyone would have to refactor their code to change IEnumerator to async Task and that is not good.
Also, there is no reason to remove the IEnumerator option only because now we can use async Task.

 

In my case it is because my tests are async but I was using UniTask.ToCoroutine (before you could run async tests officially) and I want to remove that dependency.

 

In some of my old projects, I was also using a task.ToCoroutine() extension I created, then I started to convert them to use async Task methods with [Test], and it turned out to be an easy refactoring.
What would be the issue with [UnitTest], in your case, when you start changing your UniTask.ToCoroutine to async Task methods?

 

In the section: https://docs.unity3d.com/Packages/com.unity.test-framework@1.4/manual/reference-async-tests.html

There is no mention of OneTime/Setup OneTime/TearDown attributes or the known issues of the first post of this thread:

What is the status? It's fixed on any 1.4.x package version (In the changelog there is no mention of it)? Can these attributes be used with "async Task", or not @Warnecke? Thanks.

Also, if I try to use the [Retry] attribute (I'm using also Test, UnityPlatform and Timeout attributes for what it's worth) with async Task I get the following error: