Search Unity

Unit Testing Custom Attributes

Discussion in 'Testing & Automation' started by CyberAngel, May 3, 2022.

  1. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    I did a quick search on Google and found nothing.

    I have a custom attribute at the class level, that I would like to Unit Test. But I am running into a road block on what I need to do to get this happening.

    Is anyone able to shed some light on this?
     
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    What aspects of your custom attribute do you want to test exactly?
     
  3. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    Well, 100% coverage would be a start!
     
  4. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    It is a start - a bad one. Focus on the purpose and outcomes of your testing, not on turning lines of code green. Use coverage as a tool to help you realise when you overlooked a property of your system (e.g. 'oh, I forgot to write tests that capture how the code should behave when given invalid inputs'), don't use it as a goal or an indicator of quality.

    Try these questions:
    • What are the situations in which your custom attribute will be used, that you want to verify are working as designed?
    • What properties of the way the attribute works are most at risk of being broken by future evolution of the system?
    • Does the attribute itself have significant functionality, or is most of the functionality actually in the code which uses the attribute?
    Ultimately an attribute is like any other class - you can do
    new MyCustomAttribute()
    and access members on it - and if necessary you can also create dedicated classes in your tests which you apply the attribute to.
     
  5. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    I don't subscribe to the 100 Code coverage is bad.

    Right now, I need to know how to know how to get started, so rather than a lecture on this or that, how about how do I get started!

    I am not new to TDD or BDD, and have used them in other languages as well as C# to some degree. All I want/need is right now a starting point on how I do this in Unity!
     
  6. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
     
  7. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    Can you or can you not provide examples?
     
  8. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    And how would I go about testing how many arguments where used?
     
  9. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    Code (CSharp):
    1. class MyCustomAttributeAttribute : System.Attribute
    2. {
    3.     public int Value { get; }
    4.     public MyCustomAttributeAttribute(int value)
    5.     {
    6.         Value = value;
    7.     }
    8. }
    9.  
    10. class TestClass
    11. {
    12.     [Test]
    13.     public void TestMyCustomAttribute()
    14.     {
    15.         var attr = new MyCustomAttributeAttribute(5);
    16.         Assert.That(attr.Value, Is.EqualTo(5));
    17.     }
    18. }
     
  10. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    And how does that check that it is used?
     
  11. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    You never said anything about wanting to check that it was used. You said you wanted 100% code coverage, which my example provides.

    Which brings us back to "focus on the purpose and outcomes of your testing," and the questions I originally asked you:

    • What are the situations in which your custom attribute will be used, that you want to verify are working as designed?
    • What properties of the way the attribute works are most at risk of being broken by future evolution of the system?
    • Does the attribute itself have significant functionality, or is most of the functionality actually in the code which uses the attribute?
    Neither I nor anyone else is going to be able to effectively help you if you do not explain what you are trying to accomplish. "Unit test a custom attribute" is too vague to be meaningful.

    EDIT: To put it another way: you said that you're familiar with BDD. So what are the BDD testcases that you are trying to encode?
     
  12. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    Well please DO NOT take offence to this, but code coverage includes if the Usage is Valid on say the class, property, field etc. And I guess this is why the Unity Editor is continuously getting broken! I refer to the latest bug that has been around a few months now, 5 updates and still not fix to it, in the Animator. I guess testing the number of transitions would not constitute making sure you could have more than 1 transition and it still works.

    I am very well aware how to test a class, but Attributes go beyond testing a class, it goes into how it is used and what Testing is also about. Making sure that what is intended is indeed intended use!
     
  13. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    That doesn't sound like any definition of code coverage I've ever seen, but never mind, let's focus on the problem you're actually trying to solve here.

    What I think you're saying is: you want to create unit tests that verify that your custom attribute is only applicable to the intended types of target - i.e. that you can put the attribute on a class, but that you can't put it on e.g. a property.

    Is that accurate?
     
  14. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    That would be 100% correct, and why would would you not consider this a definition of code coverage?

    If the Attribute intention is changed, I need to cover it has changed, that is what Code Coverage is for! The same if I expect a string to contain a certain value, and it has changed. By your definition you would not consider that for code coverage!
     
  15. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    In that case, I think the only unit test that makes sense is one that retrieves the AttributeUsageAttribute from your attribute class and checks that its ValidOn property has the value you expect. Testing anything more than that is testing 'does the compiler work correctly', and there's no point in you writing tests for the compiler.

    Code coverage is about measuring how much executable code has been exercised (whether it's counting lines of code, functions, branches, etc). There's nothing executable about your declaration that the attribute only applies to certain program elements - it's just marking up the type for the compiler - so code coverage is not a relevant concept.

    Correct. Measuring "did the code that returns the string get executed" is code coverage, but testing "did the string have the expected value" is simple data testing.
     
  16. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    Thanks, but I eventually found that sample elsewhere and have finished my Unit Test.

    Is it really, I have been programming for 30 years professionally. And companies that I have worked for have even done test coverages on websites using selenium, no code there, except the code we write to make sure that what is meant to be, is!

    And I will say it again. If Unity spent more time writing proper Tests, maybe the current bug with Animation Transitions would not exist! Unity obviously made a significant change that broke something else, and there was no tests to discover it is now broken!


    Who said anything about a string being returned?

    There are such things as

    private const string _invokeTestName = "TestInvoke";

    For anyone else wanting to know how to do this here is what I came up with, feel free to improve upon it.

    Code (CSharp):
    1. public class MyAttributeTest
    2. {
    3.     private IList<AttributeUsageAttribute> attributes;
    4.  
    5.     [SetUp]
    6.     public void Setup()
    7.     {
    8.         attributes = (IList<AttributeUsageAttribute>)typeof(MyAttribute).GetCustomAttributes(typeof(AttributeUsageAttribute), false);
    9.     }
    10.  
    11.     [UnityTest]
    12.     public IEnumerator CheckAttributeCount()
    13.     {
    14.         Assert.AreEqual(1, attributes.Count, "Incorrect attributes used.");
    15.         yield return null;
    16.     }
    17.  
    18.     [UnityTest]
    19.     public IEnumerator CheckNoMultiple()
    20.     {
    21.         var attribute = attributes[0];
    22.         Assert.IsFalse(attribute.AllowMultiple, "Allow multiple is not allowed.");
    23.         yield return null;
    24.     }
    25.  
    26.     [UnityTest]
    27.     public IEnumerator InheritedIsTrue()
    28.     {
    29.         var attribute = attributes[0];
    30.         Assert.IsTrue(attribute.Inherited, "Attribute must be inherited.");
    31.         yield return null;
    32.     }
    33.  
    34.     [UnityTest]
    35.     public IEnumerator AttributeIsClass()
    36.     {
    37.         var attribute = attributes[0];
    38.         Assert.AreEqual(AttributeTargets.Class, attribute.ValidOn, "Attribute Target can only be used on a Class.");
    39.         yield return null;
    40.     }
    41.  
    42.     [TearDown]
    43.     public void TearDown()
    44.     {
    45.         attributes = null;
    46.     }
    47. }
    48.  
     
  17. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    Test coverage is not the same thing as code coverage. There are many different kinds of test coverage; code coverage is only one of them. There's also - for example - state space coverage, environment coverage, data coverage, etc.

    Anyway, I simplified your tests. There is no point having the CheckAttributeCount test because the compiler will prevent you from having multiple AttributeUsage attributes on a single attribute class. You also do not need to use UnityTest because your tests do not need to be executed across multiple frames.

    Code (CSharp):
    1.  
    2. public class MyAttributeTest
    3. {
    4.     private AttributeUsageAttribute attribute;
    5.  
    6.     [SetUp]
    7.     public void Setup()
    8.     {
    9.         attribute = Attribute.GetCustomAttribute(typeof(MyAttribute), typeof(AttributeUsageAttribute), false);
    10.     }
    11.  
    12.     [Test]
    13.     public void CheckNoMultiple()
    14.     {
    15.         Assert.IsFalse(attribute.AllowMultiple, "Allow multiple is not allowed.");
    16.     }
    17.  
    18.     [Test]
    19.     public void InheritedIsTrue()
    20.     {
    21.         Assert.IsTrue(attribute.Inherited, "Attribute must be inherited.");
    22.     }
    23.  
    24.     [Test]
    25.     public void AttributeIsClass()
    26.     {
    27.         Assert.AreEqual(AttributeTargets.Class, attribute.ValidOn, "Attribute Target can only be used on a Class.");
    28.     }
    29. }
    30.  
     
  18. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    Are you saying that this is not a valid use case to test?

    Code (CSharp):
    1.     private const string _TestInvokeMethod = "TestMethod";
    2.  
    3.     private void Awake()
    4.     {
    5.         Invoke(_TestInvokeMethod, 0);
    6.     }
    7.  
     
  19. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,653
    You could certainly write a test for "Does _TestInvokeMethod have the expected value" (though I think it's not likely to be a very valuable test, especially given that in modern C# you'd use the
    nameof
    operator to set the constant value instead of writing a string like that). But tooling which measures 'code coverage' (such as Visual Studio, or JetBrains dotCover) will completely ignore your declaration of _TestInvokeMethod when calculating a coverage percentage, because there's no executable code on that line.
     
  20. CyberAngel

    CyberAngel

    Joined:
    Oct 4, 2014
    Posts:
    128
    That is correct you could do that, but not everyone is up to that level of expertise when they write their code.

    Even though I prefer to not use Invoke, it was shown to only make a point. As CONST in this example is only one way to skin the cat, and could be use elsewhere where the change could have consequences.

    But that is neither here nor there, I needed and got what I needed to test for.