Search Unity

Using ConditionalIgnore to ignore test on player

Discussion in 'Testing & Automation' started by Xarbrough, Sep 17, 2022.

  1. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,188
    I'm trying to use the ConditionalIgnore attribute to exclude a player test from running when a certain scripting define symbol is set. However, I'm having trouble registering AddConditionalIgnoreMapping early enough to make it work.

    This is what I've tried:

    Code (CSharp):
    1. public class MyTestClass
    2. {
    3.     static MyTestClass()
    4.     {
    5.         ConditionalIgnoreAttribute.AddConditionalIgnoreMapping("MY_FEATURE", !IsFeatureEnabled());
    6.     }
    7.  
    8.     private static bool IsFeatureEnabled()
    9.     {
    10. #if MY_FEATURE
    11.         return true;
    12. #else
    13.         return false;
    14. #endif
    15.     }
    16.  
    17.     [UnityTest]
    18.     [ConditionalIgnore("MY_FEATURE", "Only relevant if feature is enabled.")]
    19.     [UnityPlatform(exclude = new[]
    20.     {
    21.         RuntimePlatform.WindowsEditor,
    22.         RuntimePlatform.OSXEditor,
    23.         RuntimePlatform.LinuxEditor,
    24.     })]
    25.     public IEnumerator MyPlayerTest()
    26.     {
    27.         // Evaluate if some parts of MY_FEATURE are working when running on device.
    28.         // But if the feature is disabled, the test cannot run correctly and should be ignored.
    29.         Assert.Pass();
    30.         yield break;
    31.     }
    32. }
    I want the test to only run if MY_FEATURE is enabled, however, with the shown code it also runs if MY_FEATURE returns false. I tried to debug the issue and it seems as if AddConditionalMapping is called after the ConditionalIgnore.ApplyToTest method checks the test. I don't understand how that is possible, since the static constructor should run first, but maybe something else is off.

    I cannot use InitializeOnLoad because my test runs on device, so only playmode API is allowed.

    A different approach to solve my issue would be to simply do:

    Code (CSharp):
    1. [UnityTest]
    2. [UnityPlatform(exclude = new[]
    3. {
    4.     RuntimePlatform.WindowsEditor,
    5.     RuntimePlatform.OSXEditor,
    6.     RuntimePlatform.LinuxEditor,
    7. })]
    8. public IEnumerator MyPlayerTest()
    9. {
    10.     if (!IsFeatureEnabled())
    11.     {
    12.         Assert.Ignore("Only relevant if feature is enabled.");
    13.     }
    14.    
    15.     // Evaluate if some parts of MY_FEATURE are working when running on device.
    16.     // But if the feature is disabled, the test cannot run correctly and should be ignored.
    17.     Assert.Pass();
    18.     yield break;
    19. }
    But I feel like this is a tiny bit ugly because the test has already started to run although it should be ignored in the first place and now there are two different ways to ignore the test, first via the UnityPlatform attribute, and second via the method call. I'm just not sure that this is the idiomatic and clean way to handle such situations. Especially, because there might be multiple test methods which all need to ignored. I'm certain that I want to ignore the entire TestFixture if the feature is disabled.