Search Unity

Help Wanted Is it possible to change the order of tests on Unity Test Runner?

Discussion in 'Testing & Automation' started by abeldantas, Nov 3, 2018.

  1. abeldantas

    abeldantas

    Joined:
    Jul 28, 2013
    Posts:
    16
    I like sorting the test from more to less specific, that's how tests are sorted on my classes.

    On the Unity Test Runner they are sorted alphabetically.

    Any way to change the order?
     
  2. ElvisAlistar

    ElvisAlistar

    Unity Technologies

    Joined:
    Oct 2, 2013
    Posts:
    223
  3. gtzpower

    gtzpower

    Joined:
    Jan 23, 2011
    Posts:
    247
    Not sure if this was a new feature added recently, but it does appear that the "order" attribute affects the order in which tests run. For example:
    Code (CSharp):
    1. [UnityTest, Order(1)]
     
  4. ElvisAlistar

    ElvisAlistar

    Unity Technologies

    Joined:
    Oct 2, 2013
    Posts:
    223
    It's a feature added in NUnit 3.8, only works well in 3.8.1. The NUnit that is packaged in Unity is older, so this attribute would not be supported there.

    Relevant discussion and feature request in https://forum.unity.com/threads/changing-the-bundled-nunit-version.637549/#post-4575238
     
  5. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    41
    In the build in version of unity the Order attribute work for customizing the order of the tests within the same fixture, but on a fixture and higher level, it does not work.
     
  6. Stormy102

    Stormy102

    Joined:
    Jan 17, 2014
    Posts:
    495
    This works absolutely fine for us in 2019.1. A lot of our tests are dependent on certain preconditions, such as loading a scene. I'm assuming that this will still be working in 2019.2?
     
  7. tsibiski

    tsibiski

    Joined:
    Jul 11, 2016
    Posts:
    274
    I just want to note that there are several alternative frameworks here.

    In particular, Trilleon is also a unit test framework, and it supports all the ordering you could possibly imagine. From ordering your tests explicitly one-by-one based on the relationships you know each test has. You can mark tests as dependencies of one another across the entire framework, creating a "dependency web". If the test runner gets to a test and sees that a dependency has not yet run, it will defer that test until its dependency has run etc.

    Additionally, the framework validates valid usage of this dependency and ordering system so that you cannot accidentally create loops in dependencies.