Search Unity

Bug Static constructors not always called when running tests

Discussion in 'Testing & Automation' started by Gladyon, Feb 4, 2020.

  1. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    When running tests in edit mode several times, the static constructor is only called once (except if there's a recompile, in which case it will be called again, one time).


    Context:
    When I exit my game, most of the data are cleared, including some static data.
    Specifically, I have an array which is cleared (filled with 0) when exiting.

    Problem:
    If I run my game in the editor, and then exit the game (not Unity), it works perfectly fine.
    The array is cleared as it should be.
    When I start again the game, it works perfectly, the static constructor initializing the array is called and everything works perfectly.

    But, if I start the tests after having executed the game, the array is full of 0 (which shouldn't happen as it is is filled with non-0 values in the static constructor).
    Adding a log in the static constructor indicates it is not called as the log is never logged.


    If I modify the source code (in order to force a recompile), then it works fine, once.
    In short, it means that the static constructors aren't called when there's no recompilation.
    Meaning that there is a workaround, but if there's a way to fix that it would be better.

    I am using Unity 2019.2.17, and I heard about some optimization to enter in play mode, by avoiding some recompilation in some future version of Unity.
    I guess that it will cause the same problem, which would be a lot more critical as it would mean that entering play mode will not work anymore.
     
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,657
    This is expected behaviour. Static constructors are only called once for the lifetime of a domain; they will not be called again until the domain is reloaded (such as by entering play mode, or after code is recompiled).
     
  3. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    It means that it is the expected behavior to have failed tests when the source code of the tests is perfectly exact.
    It's not very user-friendly.

    I'll add to the documentation that one must force recompilation before executing the tests.
     
  4. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,657
    If you use static/global state and do not reset it properly before your test, then your tests are not isolated. As well as problems like the one you are seeing, you will also see problems such as a test only succeeding when particular other tests have been run before it. This is an inherent problem with the use of static/global state and something that you have to design your tests to account for.
     
  5. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    The problem is that I am relying on the fact that the compiler guarantee that the static constructor is called once and only once.

    Unfortunately, it doesn't work that way for the tests.
    I understand that it is useful as it is a lot faster not to recompile each time we execute the tests, and it's probably linked to the Editor itself, and we absolutely do not want to recompile each time we click somewhere on the Editor.

    It's more an annoyance than a real problem.
    That said, having the possibility to enable/disable automatic rebuild before executing the tests (as it is when we enter Play) would be a QoL option (and probably not a hard one to implement).


    And I am a bit concerned about a Play entering optimization I heard about, where there would be no recompile before entering Play (and I hope I am wrong, because such an optimization would cause some trouble, and not only to me).
    Anyway, I guess that if they do that, I'll have to find a way to automate rebuild when 'Play' is pressed.
     
  6. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,657
    Perhaps it would be an option for your tests to run as Playmode tests? You can still run them in the Editor, but the framework will enter play mode before executing them, which should have the same effects as when you enter play mode normally. It will only do it once at the beginning of the test run, instead of between each test, but perhaps that's enough for your purposes.

    I believe you're talking about the Enter Play Mode Options in 2019.3 - it is indeed possible to disable the recompile before entering play mode, but don't worry, it is entirely optional - we know it would break a lot of projects if we just forced it on for everyone.
     
  7. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    Don't worry, I'll find a trick to automate that if I need to.
    I'll look into the PlayMode tests, it will probably require some efforts but it's not a bad idea.
    In fact, I may even modify my code so that the statics are initialized within the test code itself, I guess it's just a matter of using reflection to find out all the static constructor and to call them all.
    Because in the end, not having to recompile isn't completely bad, as I said previously, it starts a lot faster.


    I believe you're talking about the Enter Play Mode Options in 2019.3 - it is indeed possible to disable the recompile before entering play mode, but don't worry, it is entirely optional - we know it would break a lot of projects if we just forced it on for everyone.[/QUOTE]
    Yes, that was that.
    Thanks for the info, and thank you for making it optional.

    I think I'll try to call the static constructors using reflection, if I manage that it will mean than my initialization is clean, and that's always a good thing.
    Thank you, this little chat gave me that idea and it is very interesting.
     
    superpig likes this.