Search Unity

Question Play mode tests and asmdef files

Discussion in 'Testing & Automation' started by liortal, Mar 31, 2020.

  1. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,562
    Our project doesn't use asmdef's yet (while we have a few of them - most of the code is compiled to Assembly-CSharp).

    I'd like to create a few play mode tests that would not be included into the built player.
    To achieve this i can create a new test assembly using asmdef files. This assembly will also include references to test assemblies (e.g: NUnit, NSubstitute, etc).

    The main problem is that this assembly now cannot reference anything else from the project! to solve this i must wrap everything else in asmdef as well and link it as a reference.

    Is there any other solution for this scenario?
     
  2. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,334
    @Warnecke, I'm also running into this.

    We really don't want to have our main code inside an asmdef (turns out that that causes way more problems that it's worth), but the test runner requires us to do just that.

    Am I missing something, or is it impossible to test code in Assembly-CSharp / Assembly-CSharp-Editor? I guess I could copy the test runner packages to our Packages folder and mark them as auto-referenced, but that seems somewhat clunky.
     
  3. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,334
    Okay, I just didn't try hard enough.

    NUnit is referenced by default in the editor assembly, so I can just make an Editor folder and create NUnit tests. That's not at all clear from the docs!
     
  4. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,334
    Back again!

    The solution above only works for edit mode tests. If I want play mode tests, I have to have them in a testing assembly, otherwise Unity seems to not recognize them as such. But now I can't test stuff in assembly-csharp.


    That makes tests clunkier than it has to! It also means that we have to recompile both a "core code" assembly and a unit test assembly every time we update code, which is slow!


    The ideal solution here is:
    - Test assemblies reference all the code in the project
    - Test assemblies postpones compilation until we run tests.
     
    dimmduh1 and xjjon like this.
  5. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    612
    Surely there must be a better way to use PlayMode tests that doesn't require putting all your game code its own assembly definition?
     
  6. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,334
    Looked into this recently. It's possible, but has downsides.

    If you open your projectsettings.json file, find the line "playModeTestRunnerEnabled: 0", and set it to 1, then the test runner will look for tests in Assembly-CSharp as well, so you can just stick them in your Scripts folder.

    Downside is that the tests are now included in your build by default, so you have to disable that by wrapping them in #if UNTIY_EDITOR

    This is not ideal! I also assume it means that the build has a dependency on nunit, though I'm not sure about that.
     
    crekri and xjjon like this.
  7. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    612
    Hmm, how about for any plugins or open source packages? I think that would still require lots of individual asmdef for each script folder that isn't inside your games /Scripts assembly?

    After spending some time with it, most plugins or libraries play well with it.. but if the plugin contains a /Editor/ folder then you have even more things to move around or manage. (This thread here seems to cover a lot of those problems)

    --

    For now we have resorted to just unit tests and in-game tests are "ran" by invoking them as regular monobehavior/classes in play mode and they are in #if UNITY_EDITOR.. not the most ideal and you lose some features of the test framework but still can assert majority of the things we wanted to test.
     
  8. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    515
    I had to put everything in asmdef and manage references. Not sure if there is a viable workaround. The issue faced though is that those assemblies are added to the build.

    [Edit] I used code stripping and it seems to work :
    https://docs.unity3d.com/Manual/ManagedCodeStripping.html

    ```
    <linker>
    <!--
    Don't preserve playmode tests
    -->
    <assembly fullname="___.Client.Tests.PlayMode" preserve="nothing" />
    </linker>
    ```
     
    Last edited: Aug 13, 2021