Async (non-coroutine) tests pass even if awaited tasks throw exceptions

Code (CSharp):

1. using System.Threading.Tasks;
2. using NUnit.Framework;
3. using UnityEngine;
4. public class Tests_AsyncFailure
5. {
6.     [Test]
7.     public async void AsyncTest()
8.     {
9.         await DoSomethingAsync();
10.         await DoSomethingAsync();
11.     }
12.  
13.     async Task DoSomethingAsync()
14.     {
15.         Debug.Log("???");
16.         throw new System.Exception();
17.     }
18. }

This is a simple async test that you'd expect to fail. However, the test is passing. Why is that? Are async tests explicitly not supported in Unity's test framework or something?

Extra information:
1. As you may expect, only one

Debug.Log

occurs here.
2. The thrown exception is also visible in the console
3. If I changed

Debug.Log

to

Debug.LogError

, then the test would fail because an unhandled error was logged

Side question: why does the error logged from Extra information point 2 NOT cause the test to fail, but the error logged from Extra information point 3 does?

 

Your test needs to be typed Task, rather than void, in order for the testing framework to be able to wait for it to complete.

Any strange differences are likely the result of timing issues: what happens first, the test is set to a failed state, or the testing framework is given a chance to run.

 

Ah, I wasn't aware that the "outer" function also needed to be a

Task

.

However, I'm not able to change it:

Code (CSharp):

1. AsyncTest (0.000s)
2. ---
3. Method has non-void return value, but no result is expected

 

Then your testing framework isn't supporting async methods.

NUnit has supported async tests returning Task since v3, so check your version.

Also verify that the testing framework is correctly installed, which for nunit is either a visual studio extension, or a nuget package. Verify it is version 3 or higher.

It's also possible async tests just aren't possible with Unity. I don't know.

This may also be useful to you: https://answers.unity.com/questions/1597151/async-unit-test-in-test-runner.html

 

I've not done any form of a custom installation: in a blank 2020.2.1f1 project I've added Unity's Test Framework package, and used the asset context menus to create the asmdefs and test files.

Looks like their package includes a version of NUnit that they've wrapped themselves, so I'm not sure what the actual NUnit version is

Thanks for sharing that link. Looks like Unity's come some way from then, as I've been able to

yield return

a

Task

without needing any bridging/wrapping code. This has been my fallback for now - I'm just trying to see if I can get "true" async/await working.

I guess I'll assume that it's a no-go in the current state of Unity's Test Framework, unless something new pops up on this thread