Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Feedback Do not use WaitForEndOfFrame

Discussion in 'Testing & Automation' started by javierbullrich, May 5, 2020.

  1. javierbullrich

    javierbullrich

    Joined:
    Jan 31, 2020
    Posts:
    6
    WaitForEndOfFrame is a very nasty utility when creating Unity Tests.

    If you have a simple test like the following:
    Code (CSharp):
    1. [UnityTest]
    2. IEnumerator WaitAFrame()
    3. {
    4.     yield return new WaitForEndOfFrame();
    5.     Assert.IsTrue(true);
    6. }
    This will work on the editor so you'll push it to your codebase, and if you have a CI service, like Unity Cloud Build, or you own solution (Travis/TeamCity/etc) which run your tests in batch mode, you'll discover that this tests won't succeed. In fact, it will fail with a timeout.

    The reason for this is that WaitForEndOfFrame isn't supported in batch mode, and, instead, you should use yield return null.

    Now, the nasty thing here is that Unity doesn't provide much information for this, only the previously linked documentation page and a note in the WaitForEndOfFrame doc page. And, when running this in batch mode, you won't get any log, it will simply yield until it times out, making it a silent error.

    Because we ran our tests in the editor, we won't find this error, and then, when the CI service fails, we'll have the infamous case of "it works on my machine". I have spent years having tests fail with no idea why, always timing out, looking at the logic and finding no problems with it, and no clue in the logs, until one day I discovered by accident that it only timeout on the WaitForEndOfFrame call.

    This could easily be fixed (or simplified) by Unity by adding an exception or a log when a WaitForEndOfFrame object is created in batch mode informing the user that this feature isn't supported in batch mode.
    Or, another alternative, is to handle WaitForEndOfFrame as a yield return null instead.

    Any solution would be better than silently failing and discouraging confused users from using Unity tests.

    I understand that this is used from a rendering perspective, but most Unity projects have some coroutines with calls to WaitForEndOfFrame in their code, or in plugins, and sometimes that problem can be deeply nested instead of being in the first layer of the test, making debugging extremely hard.
     
  2. javierbullrich

    javierbullrich

    Joined:
    Jan 31, 2020
    Posts:
    6
    I'm also aware that the Order of Execution for Event Functions is different, and that WaitForEndOfFrame is executed after every other event has been executed (making it truly wait for the end of the frame) but this still isn't an excuse for rendering it completely useless in batch mode.

    A quick fix for the editor can be
    Code (CSharp):
    1. public static IEnumerator WaitAFrame()
    2. {
    3.     yield return Application.isBatchMode ? null : new WaitForEndOfFrame();
    4. }
     
    mbatutis likes this.
  3. yisusgamer55

    yisusgamer55

    Joined:
    May 10, 2019
    Posts:
    7
    I don't know how to thank you. I have been 5 hours trying to see what was the problem. It is true that this "function" sucks, I do not understand that something that does work in the editor does not work later in the compilation. Thanks again!
     
  4. sang

    sang

    Joined:
    Jul 17, 2013
    Posts:
    10
    I got the same error but it appear randomly in Editor too. Sometimes the code actually passed, sometimes it's not.

    If you press Pause then Unpause, it will continue as expected.
     
  5. lassade

    lassade

    Joined:
    Jan 27, 2013
    Posts:
    121
    I found out that WaitForEndOfFrame doesn't work in the edtior (2020.1.11f1) if the Game window is not opened and rendering the game view, my guess is this function is tighly integrated with rendering.

    This one was a pain to figure out, maybe this is indeted behaviour I dunno;
     
  6. sbergen

    sbergen

    Joined:
    Jan 12, 2015
    Posts:
    38
    It's documented here: https://docs.unity3d.com/2020.2/Documentation/Manual/CLIBatchmodeCoroutines.html
    (but yes, that one can be a pain to figure out, indeed)
     
  7. doctorpangloss

    doctorpangloss

    Joined:
    Feb 20, 2013
    Posts:
    259
    In a standalone player, `-batchmode` will definitely correctly work with `yield return new WaitForEndOfFrame()`.
     
  8. matthew_unity389

    matthew_unity389

    Joined:
    Sep 30, 2021
    Posts:
    2
    I too was attempting to use WaitForEndOfFrame in batchmode and wasted a whole day trying to figure our why my coroutine wasn't working. Swapped that line with yield return null and all was well. Thanks @javierbullrich
     
  9. dan_ginovker

    dan_ginovker

    Joined:
    Jun 13, 2021
    Posts:
    32
    Thanks a ton for the solution - Can someone explain how the two may play out differently though?

    i.e. what's the difference between running
    Code (CSharp):
    1.     yield return null;
    and
    Code (CSharp):
    1.     yield return new WaitForEndOfFrame();
    Edit: to answer my own question - `yield return null` waits until the next frame to resume execution, while WaitForEndOfFrame continues execution in the same frame, just at the end. So they're similar
     
    Last edited: Feb 19, 2022
  10. dan_ginovker

    dan_ginovker

    Joined:
    Jun 13, 2021
    Posts:
    32
    Nowhere in the documentation does it specify Unity Cloud Build uses WaitForEndOfFrame, so I decided to test it myself and port away from Unity Builder on Github actions.

    Learn from my mistakes - This issue also occurs on Unity Cloud Build.

    Unity Cloud Build logs excerpt:

    Code (csharp):
    1.  
    2. 82: [Unity] OS: 'Windows 10  (10.0.17763) 64bit ServerDatacenter' Language: 'en' Physical Memory: 16380 MB
    3. 83: [Unity] COMMAND LINE ARGUMENTS:
    4. 84: [Unity] C:\Program Files\Unity\Editor\Unity.exe
    5. 85: [Unity] -batchmode // batchmode is used
    6. 86: [Unity] -skipMissingProjectID
    7. 87: [Unity] -skipMissingUPID
    8. 88: [Unity] -buildTarget
    9. 89: [Unity] android
    10. 90: [Unity] -projectPath
    11. 91: [Unity] BUILD_PATH/p
    12. 92: [Unity] -logFile
    13. 93: [Unity] -buildconfig
    14. 94: [Unity] -bvrbuildtarget
    15. 95: [Unity] test-waitforendofframe
    16. 96: [Unity] -runTests
    17. 97: [Unity] -testResults
    18. 98: [Unity] -testPlatform
    19. 99: [Unity] editmode
    20.  
    Test output:
    Code (csharp):
    1.  
    2. Failed - 1 NewTestScriptWithEnumeratorPasses
    3. Timeout value of 180000ms was exceeded
    4.  
    I'm not sure why I spent the time doing this, I did some initial tests and thought it worked, but I must have been confused.
     
  11. Squoktapus

    Squoktapus

    Joined:
    Feb 9, 2014
    Posts:
    4
unityunity