Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

  2. Introducing the new Universal Render Pipeline and High Definition Render Pipeline subforums!

    Dismiss Notice

  3. Unity 2019.3 Beta is out now.

    Dismiss Notice
2019.2 has been released! URP/LWRP forums 2019.3 Beta released

Feature Request Changing the bundled NUnit version

Discussion in 'Testing & Automation' started by liortal, Mar 1, 2019.

  1. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,231
    Currently, Unity automatically references a built-in NUnit assembly from the Unity installation folder.
    I am not 100% sure which NUnit version is that, but is it going to be possible to drop whatever NUnit version i want so my tests can use that ?
     
    liortal, Mar 1, 2019
    #1

  2. ElvisAlistar

    ElvisAlistar

    Unity Technologies

    Joined:
    Oct 2, 2013
    Posts:
    221
    We are currently moving the Test Runner functionality into a package. The built-in NUnit is going to be a separate package referenced by the Test Runner one. So, in theory, you should be able to swap that one with any other NUnit version, but in practice, things might not work as expected if you do that or might not works at all.

    The reason is that the NUnit version we use contains some modifications compared to the original:
    - it removes System.Web.dll
    - changes some internal NUnit APIs to public, so that we could make our own test runner to use instead of their default one (required for dealing with platforms like WebGL, where multithreading isn't supported)
    - NUnit did not respect equality overloading, so checking for null on UnityEngine.Object and other things like that were added
    - maybe other things I can't remember

    I think the version of NUnit we currently use is 2.6.4 and we want to update to at least NUnit 3
     
    ElvisAlistar, Mar 9, 2019
    #2

  3. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    673
    The latest version is compatible with .NET Standard 2. I don't know if it still references System.Web in that assembly

    https://www.nuget.org/packages/NUnit/

    Also about your fork and modifications... Did you suggested a pull request at that time with that valid points so you could use the original assembly still? It will be better for everyone.
     
    Last edited: Mar 10, 2019
    bdovaz, Mar 10, 2019
    #3

  4. superpig

    superpig

    Quis aedificabit ipsos aedificatores? Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,184
    @ElvisAlistar We currently ship Unit 3.5-ish.

    The biggest thing we changed in NUnit was restructuring the execution engine to make it possible to write tests as coroutines (i.e. support for the [UnityTest] attribute). I did look at whether there was a nice way to upstream this but couldn't come up with anything that I thought would be appealing to Charlie and the team. Maybe once it's all packaged up, it'll be easier to figure something out.
     
    superpig, Mar 10, 2019
    #4

  5. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    673
    @ElvisAlistar @superpig I see that in 2019.2 alpha Test Runner it's already a separate package.

    I have two questions:
    1. Have you considered updating NUnit to the latest version? Now that it is "separated from Unity" (a separate package) will it make it easier for you to update NUnit from time to time and be able to be close to the latest version?
    2. Will it ever be possible to test with async/await/Tasks? Please see: https://forum.unity.com/threads/async-await-in-unit-tests.671608/
     
    bdovaz, May 5, 2019
    #5

  6. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    33
    Hey.
    Updating nunit is something that we would like to look into when we have the bandwidth for it. However it is currently unknown for us how feasible it will be to do so and how much will break when we upgrade it. We cannot promise anything yet, but it is on our radar.
     
    Warnecke, May 24, 2019
    #6

  7. DrummerB

    DrummerB

    Joined:
    Dec 19, 2013
    Posts:
    42
    I would also like to express my support for a more recent version of NUnit. Particularly because of the Order attribute for fixtures (instead of just methods) that was introduced in NUnit 3.8. This would be useful to ensure that e.g. the tests defined in SmokeTests are run first. Would also be nice to be able to toggle between different sort modes in the test runner (e.g. by Name, by Order)
     
    DrummerB, Jun 8, 2019
    #7
    liortal and Stormy102 like this.

  8. Yacuzo

    Yacuzo

    Joined:
    Sep 22, 2016
    Posts:
    8
    I would like a newer version too. By now Unity's NUnit version is ancient (I cant even find a version that old on NUnit GitHub which goes 5 years back). I'd also like a .Net 4.5 build of it so I can run async tests. Maybe you can leverage the async support to handle coroutines?
    The docs for v2 is also way worse than the v3 docs.
     
    Yacuzo, Aug 30, 2019
    #8

  9. bdovaz

    bdovaz

    Joined:
    Dec 10, 2011
    Posts:
    673
    +1

    I also would want to have async tests.
     
    bdovaz, Aug 30, 2019
    #9

  10. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    11
    JesseSTG, Sep 6, 2019
    #10

  11. NHals

    NHals

    Joined:
    Dec 23, 2013
    Posts:
    1
    +1
     
    NHals, Sep 15, 2019 at 5:37 PM
    #11