Search Unity

Help Wanted How to run some code before running tests, then again when they all finish?

Discussion in 'Testing & Automation' started by JesseSTG, Jul 18, 2020.

  1. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    193
    I run my automated tests in the editor. There's some state I'd like to change for the duration of my test run, but I'd also like to revert this state when these tests finish.

    No matter how many tests I run, or in which assemblies, or how the test runner is started, I want to:
    • Run some code exactly once before any tests are run or initialized
    • Run some different code exactly one when all tests and other cleanup are finished, canceled, or otherwise interrupted
    How can I do this with the Unity Test Framework?
     
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,486
  3. kosigaga

    kosigaga

    Joined:
    Oct 12, 2019
    Posts:
    4
    I have tried this. The PrebuildSetup seems to work just fine, even the PostBuildCleanup is running, but. What I want to achieve is to set the Active Scene to an Empty scene before running the EditorTests.Then set it back after all tests have already run.

    If I try to OpenScene or LoadScene or even Close/RemoveScene they do not seem to work.

    Do you have an easy way to set this? Because always setting the scene to an empty one by hand is a bit tedious.

    Thanks.
     
  4. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    193
    That's what I thought, but it doesn't seem to be working the way I'd hope. I want my Prebuild/Postbuild handlers to run on a per-assembly level. However, given the following code...

    Code (CSharp):
    1. using CorundumGames.Tests;
    2. using UnityEngine;
    3. using UnityEngine.TestTools;
    4.  
    5. [assembly: PrebuildSetup(typeof(ProfileHandler))]
    6. [assembly: PostBuildCleanup(typeof(ProfileHandler))]
    7.  
    8. namespace CorundumGames.Tests
    9. {
    10.     public sealed class ProfileHandler : IPrebuildSetup, IPostBuildCleanup
    11.     {
    12.         public void Setup()
    13.         {
    14.             Debug.Log($"Setup({GetHashCode()})");
    15.         }
    16.  
    17.         public void Cleanup()
    18.         {
    19.             Debug.Log($"Cleanup({GetHashCode()})");
    20.         }
    21.     }
    22. }
    23.  
    ...the two
    Debug.Log
    statements are never called when applied at an assembly level. This class is defined in an assembly called
    Common.dll
    , which contains no tests but has code shared by
    Runtime.dll
    (Play Mode tests) and
    Editor.dll
    (Edit Mode tests).

    If I apply this handler to
    Editor.dll
    and
    Runtime.dll
    , nothing happens. However, if I apply it directly to the test fixture classes then it prints out those statements as expected. Am I doing something wrong, or is this a bug?
     
    Last edited: Jul 19, 2020
  5. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,486
    Wait, so do you want the code to run "exactly once," or do you want it to run once per-assembly?

    Maybe you want SetUpFixture?
     
  6. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    193
    Technically I want it to run exactly once (rehardles, but once per assembly would be good enough since my tests are only in two assemblies. As long as I can be sure that the SetUp/TearDowns aren't stacked (i.e. I want SetUp -> TearDown -> SetUp -> TearDown, not SetUp -> SetUp -> TearDown -> TearDown).

    It sounds like you could use more info about what I'm trying to do.

    I'm using a framework called Trimmer to manage build configurations for my project. These build configurations let me build slightly different versions of the same project, and they can vary in pretty much any way I want them to (including/excluding specific assets,
    #define
    symbols, player settings, etc.). These configurations are represented as assets called BuildProfiles. They can even be used to affect the editor; exactly one is enabled in the editor at a time.

    I want to do the following:
    • Prepare a special BuildProfile used for running in-editor tests (e.g. using certain game configurations), rather than for producing a standalone build.
    • When I run a test, I want the test profile to become active before the tests start.
    • When the test run finishes for any reason (successfully, with errors, canceled, etc.), I want the build profile that was previously active to be made so again.
    I could do this once per test fixture, but then the SetUp/TearDown pair would run for each fixture, right? Changing the build profile is an expensive operation. Also, I might simply forget to keep the attributes up to date. I have around 30 fixtures, but I'm about to do a major rewrite of my test suite.
     
    Last edited: Jul 20, 2020
  7. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,486
    I see. I do think a SetUpFixture outside of any namespace is probably your best bet, then - I think it's your best option for running code that is before/after multiple other fixtures. You'd have to have one in each of your assemblies, but they could both call into shared code.
     
  8. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    193
    I can do that? Huh. I'll be damned. I'll give this a try and report back. Thank you!
     
unityunity