Search Unity

[Test Tools] BDD Driven design and test

Discussion in 'Editor & General Support' started by Pixelstudio_nl, Mar 20, 2014.

  1. Pixelstudio_nl

    Pixelstudio_nl

    Joined:
    Jun 22, 2009
    Posts:
    179
    Hello People,

    We are implementing the unity test tools into our project. we migrate our tests from Specflow and NUnit.
    the Unit tests are really easy to put into the test framework.

    the Specflow test we found a little difficult.
    I implement the tests in a bdd specific way (Given, When, Then). For example we have defined the following Specflow scenario (short ctrl-i shows a infodialog box) :

    Scenario: Show about information
    When I activate about
    Then the about message is visible

    I have migrated this into unity like this:

    $2014-03-20 11_56_42-Unity - Tests.unity - OSS - Android_.png

    I have create a new assertion component where i can choose the test :
    $2014-03-20 11_58_18-Unity - Tests.unity - OSS - Android_.png

    The test it self implements a TestBase class that gives me a Given, When, Then like so :
    Code (csharp):
    1.  
    2. public class AboutWindowTests : TestBase
    3. {
    4.     public override void Given()
    5.     {
    6.         TestingProxy.Instance.InitializeTestingMode();
    7.     }
    8.  
    9.     public override void When()
    10.     {
    11.         TestingProxy.Instance.SetAboutActivated(1);
    12.     }
    13.  
    14.     public override void Then()
    15.     {
    16.         var systemInformation = TestingProxy.Instance.GetSystemInformation();
    17.         var messageBox = TestingProxy.Instance.GetMessageBox();
    18.        
    19.         Assert.IsNot(messageBox, null);
    20.         Assert.Is(messageBox.IsVisible, true);
    21.         Assert.IsNot(systemInformation, null);
    22.      
    23.         Assert.Is(messageBox.Title, systemInformation.InformationalVersion);
    24.         Assert.IsContaining(messageBox.Message, systemInformation.Company);
    25.         Assert.IsContaining(messageBox.Message, systemInformation.Product);
    26.         Assert.IsContaining(messageBox.Message, systemInformation.Copyright);
    27.     }
    28. }
    29.  
    This all works now, and i hope to use this more and more. i would like to discuss what you think about this way of testing and perhaps have any suggestions or improvements.

    Second i would like to discuss how you implement tests for GUI related item.
    For example, When i move mouse to position of button and click, then my window is shown.
    There is (from what i know) no way to set the mouse position in unity and trigger a 'click'...

    Best regards,
    Martijn
     
  2. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    Hi Martijn!
    you mean a custom component? or did you reuse the assertion component in some fancy way?

    I like the approach, especially if you say it works :)! Did you build your solution on top of the Integration Tests framework or did you have to modify it? We have an idea of introducing integration tests you could instantiate from code. You would simply put an attribute on a MonoBehaviour class and that class would be added as a test dynamically. Such feature would probably make things even easier for you but I there are many things we need to sort out first.

    Good testing practices prevent you from testing GUI in the way you want to test it. You should rather test what's 'under' the GUI (in your case, just call the same method as you would call after clicking the button and verify what you need).
     
  3. Pixelstudio_nl

    Pixelstudio_nl

    Joined:
    Jun 22, 2009
    Posts:
    179
    I made an custom component which inherits from AssertionComponent, in the start i use the following :
    Code (csharp):
    1.  
    2.   IEnumerator RunTest()
    3.         {
    4.             yield return new WaitForEndOfFrame();
    5.             Action.Given();
    6.             yield return new WaitForEndOfFrame();
    7.             Action.When();
    8.  
    9. #if UNITY_EDITOR
    10.             if (PauseAfterWhen)
    11.                 EditorApplication.isPaused = true;
    12. #endif
    13.  
    14.             yield return new WaitForEndOfFrame();
    15.             Action.Then();
    16.             yield return new WaitForEndOfFrame();
    17.             Action.Finally();
    18.             checksPerformed++;
    19.         }
    20.  
    It's all build on top of the UTT , i try to not touch the original test tools files so that i can update them without loosing edits i make to them.However i had to make one edit . this was caused because i inherent from AssertionComponent and implement Action different ,
    public new TestBase Action .

    this caused an null-ref in the AssertionListRenderer class.

    Code (csharp):
    1.      AssertionListRenderer
    2.         protected virtual void PrintFoldedAssertionLine (AssertionComponent assertionComponent)
    3.         {
    4.             if (assertionComponent.Action == null) return; //ADDED
    5.                 ......
    6.  

    Just made an extra 'feature'
    When i define an test an use public variables, they show up in the editor and can be edited (was harder then it looks):
    i use reflection to get/set them. this way i can reuse the test with different inputs.
    Code (csharp):
    1.  
    2. public class TestPosition : TestBase
    3. {
    4.     public int X = 100;
    5.     public int Y = 50;
    6.  
    7.     public override void Given()
    8.     {
    9.        
    10.     }
    11.  
    $2014-03-20 15_14_05-OSS.Libraries - Microsoft Visual Studio.png
     
  4. Tomek-Paszek

    Tomek-Paszek

    Unity Technologies

    Joined:
    Nov 13, 2012
    Posts:
    116
    I wonder why didn't you create a separate component? How do you benefit from inheriting it from the AssertionComponent?

    I guess you are looking for a DDT like functionality. We are also considering to implement something that would make passing different input to a test easy.