Search Unity

Resolved How do I run tests through the command line?

Discussion in 'Testing & Automation' started by Stormy102, Apr 25, 2019.

  1. Stormy102

    Stormy102

    Joined:
    Jan 17, 2014
    Posts:
    495
    Hi all,

    I've been starting to work with the Unit Test functionality within Unity with the Unit Test Runner. This works pretty well within the editor, but I am unable to get it to work by running Play Mode tests through the command line. I've tried a variety of combinations, but it doesn't help that the documentation is saying two different things. On the one hand, there is the page for Unity's command line, but on the other hand, there is the official Test Runner documentation. Can someone clear up for me how I can execute Play mode tests in batchmode and save the output file to a specified path as of 2019.1?

    Also, Unity staff, it would also be worth cleaning up the documentation. I'm aware that there is going to be a push towards making this a package for 2019.2 with an exposed API for running tests - something which will negate the use of the command line and simplify our CI setup greatly, as we won't need multiple steps to execute Tests and then build. Will this documentation be moved to the package manager documentation?

    Thanks,
    -Matt
     
    AmazingRuss likes this.
  2. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Can you say what is not working for you?

    The documentation on the test runner page should be correct, that is how we are running our tests (we are not using 2019.1, though).
     
  3. Stormy102

    Stormy102

    Joined:
    Jan 17, 2014
    Posts:
    495
    Thanks for your reply. Eventually got it working by copying the arguments used on this blog post. Doesn't help that the documentation isn't expressly clear.
     
  4. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    I know that the docs are not 100% clear or even cover all of the things you can launch via command line
    Out of curiosity, what specifically in the docs prevented you from running tests though? i don't think we struggled to follow the arguments that were specified in the docs to make it work.
     
  5. Stormy102

    Stormy102

    Joined:
    Jan 17, 2014
    Posts:
    495
    I tried using -runEditorTests and I think I also used -nographics as well, but neither of them worked.
    We have run into another bug which seems to be related to Assembly Definitions and Unit Tests, where it seems to produce a compilation failed error message, which we're looking into to attempt to reproduce.
     
  6. elenzil

    elenzil

    Joined:
    Jan 23, 2014
    Posts:
    73
    On windows, when I launch unity.exe for tests from powershell, it returns immediately, with zero feedback about the status of tests, etc. How can I know when my tests are complete, or when/if something has failed ?
    I could monitor for the existence of the output.xml file, but if there's something wrong with the command-line that will never get created. .. As it stands now the output file is not being created, and there's zero feedback on command-line. The tests run fine from editor. How should I diagnose this ?
     
  7. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,659
    This is a Windows/Powershell oddity - Unity is a 'Windows subsystem' exe so Powershell doesn't wait for it to exit before completing the command. You see the same thing with e.g. notepad.exe.

    See this StackOverflow post for tips on how to work around it.