Search Unity

  1. Are you interested in providing feedback directly to Unity teams? Sign up to become a member of Unity Pulse, our new product feedback and research community.
    Dismiss Notice

Why do my tests run so slowly on Jenkins/Cloud Build?

Discussion in 'Testing & Automation' started by andrew-fray, Sep 30, 2020.

  1. andrew-fray

    andrew-fray

    Joined:
    Jul 19, 2012
    Posts:
    147
    I have a few playmode tests that load each level, watch for errors, then unload the level. They're super simple. The RunsCleanly() test just presses Continue on the controller, and walks the player character forwards for 6 seconds. It's sequenced behind other tests so we already know the level is loaded.

    On my local machine, RunsCleanly takes 8 seconds once it's done its setup and teardown. On the jenkins server and cloud build, it takes over a minute and sometimes even longer.

    I'm on Unity 2019.4.5. The jenkins server is a t2.large aws instance using ebs. The command line I'm using to run the tests is
    -batchmode -projectPath "." -runTests -testResults playmodetests.xml -testPlatform playmode -logFile ${WORKSPACE}\playtestlog.txt


    The unity editor log is clean. I can see it starting the level and completing the test. It's just taking forever. I don't think it's loading time, because this test only starts once the level is fully loaded, and ends before we unload the level.

    What could be slowing it down? How can I speed it up? Is there any more info I can get you?
     
    Last edited: Sep 30, 2020
  2. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,505
    does it ever complete? you said locally it takes 8 seconds. how long does it take to complete on EC2 ?
     
  3. andrew-fray

    andrew-fray

    Joined:
    Jul 19, 2012
    Posts:
    147
    Over a minute, and sometimes even longer. It by default times out at 3 minutes.
     
  4. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,505
    And what does the Unity log say? did you try to log the times in the Editor log to see whats taking it so long?
    e.g: use an example code such as this one:
    Code (CSharp):
    1. public class LogWithTime : ILogHandler
    2. {
    3.     private ILogHandler logHandler;
    4.    
    5.     public LogHandlerLior()
    6.     {
    7.         this.logHandler = Debug.unityLogger.logHandler;
    8.     }
    9.    
    10.     public void LogFormat(LogType logType, Object context, string format, params object[] args)
    11.     {
    12.         logHandler.LogFormat(logType, context, $"[{DateTime.Now.ToString("hh.mm.ss.ffffff")}] {format}", args);
    13.     }
    14.  
    15.     public void LogException(Exception exception, Object context)
    16.     {
    17.         logHandler.LogException(exception, context);
    18.     }
    19. }
    20.  
    21. [InitializeOnLoad]
    22. public class EditorClass
    23. {
    24.     static EditorClass()
    25.     {
    26.         Debug.unityLogger.logHandler = new LogWithTime();
    27.     }
    28. }
     
  5. andrew-fray

    andrew-fray

    Joined:
    Jul 19, 2012
    Posts:
    147
    as I mentioned in the OP the log is clean. I can see it starting and stopping the test in the log, with the timestamps that unity inserts, and the timestamp on jenkin's test output is accurate.
     
unityunity