Search Unity

Can't make the basic PlayMode UnityTest generate an output with Unity 2019.1.9.f2

Discussion in 'Testing & Automation' started by mihl, Jul 18, 2019.

  1. mihl

    mihl

    Joined:
    Apr 22, 2014
    Posts:
    5
    Hi,

    I just started using Unity to build up a flappy dinosaur game. Because I am developping my game following TDD philosophy, I want to have my game tested. Therefore, here's what I tried to do without success on Unity 2019.1.9.f2:

    1. Create new 2D project
    2. In the TestRunner, select PlayMode
    3. Create PlayMode Test Assembly Folder
    4. Create Test Script in current folder

    Here's the test code I want to try:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using NUnit.Framework;
    4. using UnityEngine;
    5. using UnityEngine.TestTools;
    6.  
    7. namespace Tests
    8. {
    9.   public class NewTestScript
    10.   {
    11.     // A Test behaves as an ordinary method
    12.     [Test]
    13.     public void NewTestScriptSimplePasses()
    14.     {
    15.       // Use the Assert class to test conditions
    16.       Assert.Inconclusive();
    17.     }
    18.  
    19.     // A UnityTest behaves like a coroutine in Play Mode. In Edit Mode you can use
    20.     // `yield return null;` to skip a frame.
    21.     [UnityTest]
    22.     public IEnumerator NewTestScriptWithEnumeratorPasses()
    23.     {
    24.       // Use the Assert class to test conditions.
    25.       // Use yield to skip a frame.
    26.       yield return null;
    27.       Assert.Inconclusive();
    28.     }
    29.   }
    30. }
    31.  
    The TestRunner does not provide me with any kind of feedback on the tests, except that they "didn't run" (number of green tests = number of failed tests = 0, number of not run tests = 2). I tried to restart Unity and run the tests again without any success.

    That lack of feedback is interesting because if I set a breakpoint in both of those tests, then they're hit, i.e. the tests are run.

    Same with 2018.4.3f1.

    Now if I reproduce that very experiment with Unity 2018.1.9f2, then

    • NewTestScriptSimplePasses provides no feedback at all
    • NewTestScriptWithEnumeratorPasses triggers an "InconclusiveException" in the console
    If I replace the Assert.Inconclusive() with Assert.IsTrue(false), then I get proper feedback in the 2nd test. If I call Assert.IsTrue(true) in the 2nd test, then I get no feedback. In all cases (also those where I get an exception), the green / red ticks are not shown at all on the test runner.

    What am I doing wrong? In EditorMode, everything is fine.

    Thanks in advance for your help...
     
  2. mihl

    mihl

    Joined:
    Apr 22, 2014
    Posts:
    5
    As a side note, it doesn't work either with version 2019.2.0b9.
     
  3. mihl

    mihl

    Joined:
    Apr 22, 2014
    Posts:
    5
    So ... I was finally able to make it work. It might be in my opinion a bug in Unity, but I can't say with confidence as I am a very inexperienced user of Unity. Here's how I was able to get the green / red ticks: by total coincidence, I pressed the play button (to play the game, so nothing to do with the tests), and then pressed pause, then pressed pause again, then pressed the Step button. At the same time, in the version 2018.4.3f1, I was able to run the PlayMode tests. Then, suddenly the green / red ticks appeared. Since then, I don't need to run my game in play mode any more to get feedback on my tests ... it's a bit strange. Also, inconclusive assertions are not tracked correctly: there are considered passing, they should be considered the same way they are in EditorMode: inconclusive, orange circles.
     
  4. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    92
    Hey. We fixed the issue recently. If you update to a newer version of unity, e.g. 2018.4.5f1, then it should be fixed.
     
  5. mihl

    mihl

    Joined:
    Apr 22, 2014
    Posts:
    5
    Is 2018.4.5f1 newer as 2019.1.9f2? Because I tried it on the latter version.
     
  6. ShadyProductions

    ShadyProductions

    Joined:
    Jul 21, 2014
    Posts:
    18
    This bug is still present in 2019.3.0f6, I also couldn't get red/green tick output until i played, paused un paused, stopped playmode then ran my test.
     
  7. jamesviago

    jamesviago

    Joined:
    Feb 1, 2020
    Posts:
    1
    2019.3.3f1 i seem to have same issue, no tests run at all, just grey circles