Search Unity

Feedback Do not use WaitForEndOfFrame

Discussion in 'Testing & Automation' started by javierbullrich, May 5, 2020.

  1. javierbullrich

    javierbullrich

    Joined:
    Jan 31, 2020
    Posts:
    6
    WaitForEndOfFrame is a very nasty utility when creating Unity Tests.

    If you have a simple test like the following:
    Code (CSharp):
    1. [UnityTest]
    2. IEnumerator WaitAFrame()
    3. {
    4.     yield return new WaitForEndOfFrame();
    5.     Assert.IsTrue(true);
    6. }
    This will work on the editor so you'll push it to your codebase, and if you have a CI service, like Unity Cloud Build, or you own solution (Travis/TeamCity/etc) which run your tests in batch mode, you'll discover that this tests won't succeed. In fact, it will fail with a timeout.

    The reason for this is that WaitForEndOfFrame isn't supported in batch mode, and, instead, you should use yield return null.

    Now, the nasty thing here is that Unity doesn't provide much information for this, only the previously linked documentation page and a note in the WaitForEndOfFrame doc page. And, when running this in batch mode, you won't get any log, it will simply yield until it times out, making it a silent error.

    Because we ran our tests in the editor, we won't find this error, and then, when the CI service fails, we'll have the infamous case of "it works on my machine". I have spent years having tests fail with no idea why, always timing out, looking at the logic and finding no problems with it, and no clue in the logs, until one day I discovered by accident that it only timeout on the WaitForEndOfFrame call.

    This could easily be fixed (or simplified) by Unity by adding an exception or a log when a WaitForEndOfFrame object is created in batch mode informing the user that this feature isn't supported in batch mode.
    Or, another alternative, is to handle WaitForEndOfFrame as a yield return null instead.

    Any solution would be better than silently failing and discouraging confused users from using Unity tests.

    I understand that this is used from a rendering perspective, but most Unity projects have some coroutines with calls to WaitForEndOfFrame in their code, or in plugins, and sometimes that problem can be deeply nested instead of being in the first layer of the test, making debugging extremely hard.
     
  2. javierbullrich

    javierbullrich

    Joined:
    Jan 31, 2020
    Posts:
    6
    I'm also aware that the Order of Execution for Event Functions is different, and that WaitForEndOfFrame is executed after every other event has been executed (making it truly wait for the end of the frame) but this still isn't an excuse for rendering it completely useless in batch mode.

    A quick fix for the editor can be
    Code (CSharp):
    1. public static IEnumerator WaitAFrame()
    2. {
    3.     yield return Application.isBatchMode ? null : new WaitForEndOfFrame();
    4. }
     
    mbatutis likes this.
  3. yisusgamer55

    yisusgamer55

    Joined:
    May 10, 2019
    Posts:
    7
    I don't know how to thank you. I have been 5 hours trying to see what was the problem. It is true that this "function" sucks, I do not understand that something that does work in the editor does not work later in the compilation. Thanks again!
     
  4. sang

    sang

    Joined:
    Jul 17, 2013
    Posts:
    8
    I got the same error but it appear randomly in Editor too. Sometimes the code actually passed, sometimes it's not.

    If you press Pause then Unpause, it will continue as expected.
     
  5. lassade

    lassade

    Joined:
    Jan 27, 2013
    Posts:
    86
    I found out that WaitForEndOfFrame doesn't work in the edtior (2020.1.11f1) if the Game window is not opened and rendering the game view, my guess is this function is tighly integrated with rendering.

    This one was a pain to figure out, maybe this is indeted behaviour I dunno;
     
  6. sbergen

    sbergen

    Joined:
    Jan 12, 2015
    Posts:
    21
    It's documented here: https://docs.unity3d.com/2020.2/Documentation/Manual/CLIBatchmodeCoroutines.html
    (but yes, that one can be a pain to figure out, indeed)
     
unityunity