Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Editor test runner (NUnit)

Discussion in '5.3 Beta' started by liortal, Sep 30, 2015.

  1. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Hey,

    Is this any different from the runner in the Unity Test Tools package ? if not, what is different here?
     
    liortal, Sep 30, 2015
    #1

  2. iivo_k

    iivo_k

    Joined:
    Jan 28, 2013
    Posts:
    314
    I'd assume it's just now built-in instead of a separate asset.
     
    iivo_k, Sep 30, 2015
    #2

  3. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I assumed that too, but the link in the original thread (the one listing all the new things in 5.3) didn't say anything about that.

    Can you please make 1 small fix: i think that "run tests in a separate scene" should be made default.

    all too often i run into the following scenario:

    1. Run some tests (before creating a build locally)
    2. Some of the tests have side effects that create new game objects (these are added to the current scene).
    3. Once all tests pass, i create a build.

    This is dangerous as it can get some "test only" objects into scenes.
    I know that unit tests are not supposed to use game objects (since this is not a "pure" unit test), but still this is easier for me to create such tests that run in the editor to exercise some parts of our game.
     
    liortal, Sep 30, 2015
    #3

  4. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,658
    If you're creating game objects, should those not be integration tests?
     
    superpig, Sep 30, 2015
    #4

  5. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    It sort of is, but i'm already used to the syntax of creating unit tests and that's so much easier for me.

    Our test code instantiates some objects (that in turn may lazilly create GameObjects). I am not sure how the equivalent integration test would look like... at the bare minimum i know i should create a new test scene for it, but i haven't investigated it further yet.
     
    liortal, Sep 30, 2015
    #5

  6. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,350
    So now that NUnit is included, UnityTestTools will cause the project to fail to compile due to the two copies of NUnit. Like others, I assumed that this new test runner was just a bundled version of UnityTestTools but apparently it's completely different; this one doesn't use the UnityUnitTest and instead can run any standard NUnit test class? But it does not include NSubstitute or any of the integration tools? Is there going to be some sort of reconciliation of the two different test tools or is this new one going to replace the existing one?
     
    makeshiftwings, Oct 4, 2015
    #6

  7. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    How did you deduce that it's "completely different" ? What is UnityUnitTest? i just use standard NUnit attributes for my unit tests with the currently available test runner...
     
    liortal, Oct 4, 2015
    #7

  8. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,350
    Unity Test Tools contains a class called "UnityUnitTest" which inherits from a standard NUnit test class; I assumed it was necessary to inherit from it in order to get it to show up in the old test runner but maybe not; it could just be there for the helper methods. But this new version is missing things like NSubstitute and the integration test package.

    I got both of them to work together by just deleting the NUnit.Framework.dll from the UnityTestTools package but leaving the other NUnit dll's and the wrapper classes.
     
    makeshiftwings, Oct 4, 2015
    #8

  9. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Sorry guys for late reply, I just came back from vacations.

    The Editor Tests Runner is the Unit Tests Runner from Unity Test Tools package integrated in the editor. There's been small changes to it but overall it's the same runner and mostly the same code. It's mostly the rendering code that's been changed because the runner window is using the TreeView class from the editor.

    The reason it got renamed to Editor Tests Runner is because although it's NUnit driving the test run, you can write integration tests in it as well. I discussed this a little bit in this blog post: http://blogs.unity3d.com/2014/05/21/unit-testing-part-1-unit-tests-by-the-book/

    To address your questions/issues:
    • The UnityUnitTest class had been removed not to confuse people. It wasn't required for running tests and if you still need it you can copy it from the original package
    • Same with NSubstitute, for the moment it's not gonna be included by default.
    • The UTT package needs to be cleaned up in order to work with 5.3.
    • Changing default "run tests in a separate scene" to true might be a good idea, i'll think about it
     
    Tomek-Paszek, Oct 13, 2015
    #9
    GarthSmith, 5argon, Trong Ho and 1 other person like this.