Search Unity

Feature Request Support for warnings (non-critical tests)

Discussion in 'Testing & Automation' started by Xarbrough, Aug 5, 2020.

  1. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,188
    Would it be possible to add a test result that indicates a non-critical or somewhat optional property? For example, we are validating the content of scenes in our game. Some prefabs must always be present and therefore a test either fails or passes when checking for their presence. However, sometimes, there are fuzzy rules such as "no duplicates". These duplicates should be removed just to keep things clean, but they do not break anything (non-critical). At the moment, this check for duplication test shows dozens of red failure icons, which looks as if something is broken, when it isn't. I'd rather show a warning sign to indicate that this is something that needs to be addressed in the future but isn't blocking the game.

    I realize that this may be out of scope of unit testing in general and the test runner in Unity, but it shows a need for content validation tools. I can implement my own editor window to do this, but maybe it would make sense to support this leveraging the existing test runner framework.
     
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,659
    Have you considered using the `[Category]` attribute to separate your critical and non-critical tests? Then you can filter the Test Runner window to show you only tests in one of the two categories.

    I'm not sure how much sense it makes for us to implement this specifically into the test framework itself, though I think there's a better argument for making the TestRunner window itself more customizable/modular, so you could either customize the way it gets drawn or reuse the pieces of it to build your own window more easily.
     
    Xarbrough likes this.