Feature Request Changing the bundled NUnit version

Currently, Unity automatically references a built-in NUnit assembly from the Unity installation folder.
I am not 100% sure which NUnit version is that, but is it going to be possible to drop whatever NUnit version i want so my tests can use that ?

 

The latest version is compatible with .NET Standard 2. I don't know if it still references System.Web in that assembly

https://www.nuget.org/packages/NUnit/

Also about your fork and modifications... Did you suggested a pull request at that time with that valid points so you could use the original assembly still? It will be better for everyone.

 

@ElvisAlistar @superpig I see that in 2019.2 alpha Test Runner it's already a separate package.

I have two questions:
1. Have you considered updating NUnit to the latest version? Now that it is "separated from Unity" (a separate package) will it make it easier for you to update NUnit from time to time and be able to be close to the latest version?
2. Will it ever be possible to test with async/await/Tasks? Please see: https://forum.unity.com/threads/async-await-in-unit-tests.671608/

 

I would also like to express my support for a more recent version of NUnit. Particularly because of the Order attribute for fixtures (instead of just methods) that was introduced in NUnit 3.8. This would be useful to ensure that e.g. the tests defined in SmokeTests are run first. Would also be nice to be able to toggle between different sort modes in the test runner (e.g. by Name, by Order)

 

I would like a newer version too. By now Unity's NUnit version is ancient (I cant even find a version that old on NUnit GitHub which goes 5 years back). I'd also like a .Net 4.5 build of it so I can run async tests. Maybe you can leverage the async support to handle coroutines?
The docs for v2 is also way worse than the v3 docs.

 

+1

I also would want to have async tests.

 

I'd be happy to just have

Assert.Multiple

.

 

+1

 

One of the reason why we desperately need an NUnit update on our side is to support async tests correctly. The embedded NUnit version does not accept test methods that returns Tasks, e.g. this does not run:

Code (CSharp):

1. [Test]
2. public async Task SimpleAsyncTest()
3. {
4.     var value = false;
5.  
6.     await Task.Delay(2000).ContinueWith((t) => {
7.         value = true;
8.     });
9.  
10.     Assert.IsTrue(value);
11. }

And using void is not an option since Nunit will think the test is done at the first await:

Code (CSharp):

1. [Test]
2. public async void SimpleAsyncTest()
3. {
4.     await Task.Delay(2000);
5.  
6.     UnityEngine.Debug.LogWarning("This executes after the test completes, making the test a false positive");
7.     Assert.IsTrue(false);
8. }

If Unity does not have the resources to maintain a custom Nunit version, is there a possibility to integrate another test runner ?

 

I've run into the same issue as remi-sormain-vrdirect mentioned. In Unity 2019.3.6f1 version and Test Framework package 1.1.13 version from March 17, 2020.
The tests with Task return type fails by default.
So I wrote task.Wait() in tests for async methods. It seems to work.

 

I would also prefer async unit tests now that Unity has better support for async features in general.

 

A year has passed since this reply. It seems like bandwidth should be made for this, it's quite painful to have such an old version of NUnit.

 

And another 3 months. Poking here in the hopes of getting a Unity dev's attention on this.

 

I assume you proposed a task var with Task.Run(Action to await for), then .Wait() on this task var and then use .Result().
That's not working to me 'cause it says my action to await for is only capable to be executed from the main thread, so I'm on a deadlock thus I can't test async code in my project.
The transient solution I came with is just return void so all tests pass but they log the exceptions. My decision relies on a proper solution updating NUnit bundled so Task can be returned where on a [Test] method.

 

I will too ask for this to be prioritised, lack of proper async/await support is holding us back from adopting it. Plus, with newer APIs like Addressables supporting the pattern and libraries like UniTask getting better with every update, I believe it would be a huge upgrade for many workflows.

 

+1
I really want this to be prioritised, [UnityTest] and other workarounds bloat tests too much and lead to garbage code

 

Another solution I have dwelt on is use SetUp/Teardown in such a heterodox way so SetUp subscribes to Unity console logs then Teardown spreads any error/exception log the test came up with, throwing a new exception, thus the test will not pass.
It doesn't work because Unity does not raise the log event when logging the test exception — having async void as tests signature, of course.

I tried both events regarding Application.LogCallback type.
https://docs.unity3d.com/ScriptReference/Application.LogCallback.html

May someone finds a way.

 

@superpig @Warnecke @ElvisAlistar I have a couple questions:
1. Is there any chance of this being prioritised in the mid-term?
2. Does this repository reflect the current state of the NUnit package? If not, is the current version source available somewhere?
3. If someone could update to a newer version of NUnit, would you be willing to accept a PR?

 

+1
We need answers ASAP. We have been years waiting and it's slowing us down.

 

Bump

 

Any updates regarding this @superpig ?

Alternatively, is there any user developers can choose their own version of NUnit and add it, instead of the one provided out-of-the-box ?