Search Unity

Resolved Automate Unit Tests Before Build

Discussion in 'Testing & Automation' started by Kokowolo, Aug 24, 2021.

  1. Kokowolo

    Kokowolo

    Joined:
    Mar 26, 2020
    Posts:
    60
    How do I automate my UTF unit tests before I create a build such that the tests must pass in order for the build to complete?

    I ask this because I've researched this topic for a day now and haven't really found materials online that pertain to the Unity Test Framework and automation before a build. All solutions I've created myself are incredibly haphazardous and not really streamlined.

    Ideally, the automation would run both play and edit mode tests before a build, but this is a little challenging since UTF incorporates callbacks that yield before a solution using something like IPreprocessBuildWithReport can validate before creating a potentially test-failing build.
     
    Kokowolo, Aug 24, 2021
    #1
    will_unity731 likes this.

  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,659
    As you've identified, this is not necessarily possible using IPreprocessBuildWithReport - you would only ever be able to run synchronous tests in that context.

    What I recommend is to instead focus on creating a custom Editor script which initiates the workflow you want - e.g. uses the TestRunnerApi to launch the test run, subscribes to the run completion event, and if all the tests passed, calls BuildPipeline.BuildPlayer. This script can then be invoked from the command line using the -executeMethod option, and in the Editor you could either bind it to a MenuItem or you could bind it to the 'build' button in the Build Window.
     
    superpig, Aug 24, 2021
    #2
    Kokowolo likes this.

  3. Kokowolo

    Kokowolo

    Joined:
    Mar 26, 2020
    Posts:
    60
    Thanks, @superpig, for your response! I was hoping for an easier solution, but this certainly makes sense and clarifies my next step. I'll start right away with your recommendation :D
     
    Kokowolo, Aug 24, 2021
    #3

  4. Kokowolo

    Kokowolo

    Joined:
    Mar 26, 2020
    Posts:
    60
    My solution was a bit whacky, so I won't post the script here. But here are a couple of 'gotchas' for anyone else who is taking on this issue.
    • To run both types of tests through the 
      TestRunnerApi
      , you must run Edit Mode and Play Mode tests separately and stitch the results together.
    • To get Play Mode tests working, you must adhere to an 
      assembly definition
      hierarchy throughout your project. If you JUST want to use Edit Mode tests, then you can avoid this requirement by placing your Edit Mode tests inside of an Editor folder provided that it is under the 
      Assembly-CSharp-Editor.dll
    • Since the editor will have a domain reload after you run your Play Mode tests, to keep your registered 
      callbacks
      you either must disable the domain reload feature (when starting Play Mode) or save the 
      TestRunnerApi
      data to disk somewhere
    Those were my main issues, good luck to whoever needs this information!
     
    Last edited: Nov 9, 2021
    Kokowolo, Sep 23, 2021
    #4
    JustJunuh and NikMikk like this.