Search Unity

Unity Unity Test Framework documentation

Discussion in 'Testing & Automation' started by Warnecke, Jul 16, 2019.

  1. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    27
    With Unity 2019.2, the Unity Test Framework (aka Unity Test Runner) will be a package. As a part of it, we’ve given the documentation a very much needed overhaul.

    The latest version of the documentation is available here: https://docs.unity3d.com/Packages/com.unity.test-framework@1.1/manual/index.html

    Any feedback and questions to the documentation are welcome. We will be keeping the documentation up to date as we release new versions of the package.
     
    Last edited: Aug 7, 2019
    liortal and Stormy102 like this.
  2. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    4
    Beautiful; a hell of an improvement over the current documentation.

    One request; can you describe the order in which external actions within the same category run with respect to each other? E.g. in which order will implementations of
    OuterUnityTestAction
    run with respect to one another? Or implementations of
    IWrapSetUpTearDown
    ?

    EDIT: And maybe while you're at it, why would I want to use certain stages over others? Like, why would I want to use
    OuterUnityTestAction
    over
    IWrapSetUpTearDown
    /
    IWrapTestMethod
    , or vice versa?
     
  3. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    27
    Thanks.
    In regards to the order of each set of actions (e.g. OuterUnityTestAction), then they are run in the same order as other nunit attribute, which again is the order that System.Reflection provide the attributes. It is however not documented.
    In my experience, then the order seems to be first it takes the attributes in the order they are written, staring with the base most class, working upwards.

    Regarding OuterUnityTestAction vs IWrapSetUpTearDown/IWrapTestMethod, then they are a lot alike, but the need for the OuterUnityTestAction arose for performance measurements, which would preferably be at the outer most scope. Hence this can be used as the outer most action.
     
  4. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,222
    Thanks for updating the docs. here is some feedback from me (BEWARE!!! nitpicking ahead! also this is going to be very long).

    https://docs.unity3d.com/Packages/c...@1.1/manual/edit-mode-vs-play-mode-tests.html

    I am not using .asmdef but still, my editor code (which contains my editor tests) reference my "runtime" scripts. This phrasing (to me at least) made it sound as if you can only access editor code in these tests.

    We have a lot of code that is not MonoBehaviour based but still used at runtime in our game. This makes it perfectly suitable to be run as an edit mode test, so it's not really only for editor extensions.

    Not sure why this would be needed, but sounds like a bit of a hack isn't it? creating an edit mode test only to let it go into play mode.

    The page later mentions how to set up playmode vs. edit mode tests.
    For edit mode:
    And for play mode:
    Should probably describe both cases in a similar way to avoid any confusion.

    This section is not clear to me (sorry!)

    https://docs.unity3d.com/Packages/com.unity.test-framework@1.1/manual/workflow-create-test.html

    This is not a category... NUnit allows you to tag tests using the Category attribute. What's shown on the documentation page is the actual class name containing different tests. using this attribute has no effect for me.
    The categories are shown on the top right of the window:
    upload_2019-8-10_21-12-54.png

    https://docs.unity3d.com/Packages/c...al/workflow-run-playmode-test-standalone.html
    "Running as standalone" is a bit confusing, since it sounds like it's only possible to run on a standalone (e.g: Desktop) build.
    The UI itself calls it "Run in player" so i think the docs should be aligned with that.

    TestPlayerBuildModifier
    (not docs related, sorry..) First of all, why do we need this attribute? In order to modify the BuildPlayerOptions we already had this option available for a while.

    Also - this method of setting an attribute to point to a given type is a bit cumbersome. why is it designed that way? if my implementation needs to implement a certain interface, i suppose you can find that out via reflection and instantiate it.

    "Modify the Player build options for Play Mode tests" -- why is this needed? why does the location of the build player matters when executing PlayMode tests?

    Split build and run for Play Mode tests
    The code example is not very clear:
    1. Why do i need to modify the build location?
    2. A bit of a hacky path manipulation going on there
    3. Why are we exiting the app if run from the command line? isn't this provided by the -quit argument ?


    These are my comments so far. again, thank you for creating such a detailed guide with examples and API references. great step indeed :)