Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Confusion between unit tests, EditMode tests and PlayMode tests

Discussion in 'Testing & Automation' started by alexisnavarro24, Oct 21, 2022.

  1. alexisnavarro24

    alexisnavarro24

    Joined:
    Mar 11, 2022
    Posts:
    1
    I have a couple of questions I really need some help with.

    1. A coworker who's basically in charge of overviewing unit tests told me that EditMode tests are unit tests and PlayMode tests are (exclusively) integration tests. Is this correct?

    2. He wants the EditMode tests to have 100% coverage. Is this realistic? Up until recently this also included testing private methods using reflection, but he dropped that idea after some debating.

    3. Recently it was decided that instead of testing lines of code, we want to test behaviours instead. The idea is: "I don't care if this method calls a method named DoSomething or DoSomethingElse. What I care about is the result. If tomorrow we rename DoSomething to ExecSomething, why should the test fail when the behaviour is still the same?"

    But I find it problematic to test a behaviour in a unit test (EditMode) when the behaviour/result is seen in another class. For example, consider the method LoginManager.EnableLogin:

    Code (CSharp):
    1. public class LoginManager : MonoBehaviour
    2. {
    3.     private LoginView login;
    4.     ...
    5.     public void EnableLogin()
    6.     {
    7.         login.EnablePanel();
    8.     }
    9.     ...
    10. }
    11.  
    12. public class LoginView : MonoBehaviour
    13. {
    14.     protected new GameObject gameObject; // this is necessary to allow set/get using reflection in the tests
    15.     ...
    16.  
    17.     private void Start()
    18.     {
    19.         gameObject = base.gameObject;
    20.         ...
    21.     }
    22.  
    23.     public void EnablePanel()
    24.     {
    25.         gameObject.SetActive(true);
    26.     }
    27.     ...
    28. }
    In the test, the behaviour we want to check is that LoginView.gameObject is set active. This is the test:

    Code (CSharp):
    1. public class LoginManagerTests
    2. {
    3.     private LoginManager manager;
    4.  
    5.     private ILoginView loginView;
    6.  
    7.     [SetUp]
    8.     public void SetUp()
    9.     {
    10.         manager = new LoginManager();
    11.  
    12.         loginView = Substitute.For<ILoginView>();
    13.         SetValue(manager, "login", loginView);
    14.     }
    15.  
    16.     [Test]
    17.     public void EnableLogin_ShouldEnableLoginPanel()
    18.     {
    19.         //Arrange
    20.         GameObject loginGameObject = new GameObject();
    21.         loginGameObject.SetActive(false);
    22.         SetValue(loginView, "gameObject", loginGameObject);
    23.  
    24.         //Act
    25.         manager.EnableLogin();
    26.  
    27.         //Assert
    28.         Assert.AreEqual(
    29.             true,
    30.             GetValue<GameObject>(loginView, "gameObject").activeInHierarchy
    31.             );
    32.     }
    33. }
    Is this really a proper unit test? It seems a bit dirty...
    And then the question arises: Shouldn't this kind of stuff be tested in PlayMode? In the test just load the Login scene, call loginManager.EnableLogin() and check if the panel becomes active. Voilà, behaviour checked. Am I missing something?
     
    Last edited: Oct 21, 2022
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,649
    Not really. I think what your coworker is getting at is that a unit test shouldn't need to be a playmode test because it shouldn't be dependent on interacting with engine systems - if the system under test is truly isolated then it should be possible to test the behaviour you want purely in edit mode. However, there's nothing stopping you from running that same test in play mode, it's just a bit of a waste of time. It's also completely possible to write integration tests in edit mode.

    It may be realistic in the sense that 'it can be done', but it's debatable as to whether it's a good idea.

    Yes, I would not call this example a unit test because it's testing behaviour across two units (LoginManager and LoginView). If you follow traditional unit test rules, you need two tests: one which tests "when LoginManager.EnableLogin is called, then the EnablePanel method on its LoginView member is called exactly once", and one which tests "when LoginView.EnablePanel is called, then the SetActive member of its GameObject members called exactly once with parameter 'true'."

    You certainly could do that, and it brings a different kind of value. For example, that test might fail if the LoginManager/LoginView objects in your Login scene aren't configured correctly. As a result the scope of possible reasons for the test to fail is broader - it's no longer only about the code, but about the Scene, and potentially about other things in the Scene as well, etc. There are pros and cons to that: pro, you potentially catch a broader range of issues with your test; con, when your test does fail you will need to do more investigation to see why. Execution time will also be higher compared to an EditMode test.
     
    apongs_vs and liortal like this.