Search Unity

Inconsistent benchmarking results

Discussion in 'Data Oriented Technology Stack' started by jRocket, Jul 28, 2021.

  1. jRocket

    jRocket

    Joined:
    Jul 12, 2012
    Posts:
    607
    I have a simple pathfinding system implemented with Jobs and Burst, but I am running into a problem when I try to benchmark it with the StopWatch class each frame. The first frame I run the pathfinding, I get something like 10ms, but each subsequent frame completes the work in only 1ms. Even putting my code in Start() and hitting the start button in the editor will result in these inconsistent results, but only when Domain Reloading is disabled.

    Is there some kind of caching going on here? If so, how do I fix it for the purposes of profiling my code without having to reload the domain every time?

    Code (CSharp):
    1.     void Update ()
    2.     {
    3.         var sw = new Stopwatch();
    4.         sw.Start();
    5.  
    6.         var jobs = new NativeList<JobHandle>(Allocator.Temp);
    7.         for (int i = 0; i < 100; i++)
    8.         {
    9.             var job = new FindPathJob
    10.             {
    11.                 StartPosition = new int2(0, 0),
    12.                 EndPosition = new int2(99, 99)
    13.             };
    14.             jobs.Add(job.Schedule());
    15.         }
    16.  
    17.         JobHandle.CompleteAll(jobs);
    18.         jobs.Dispose();
    19.  
    20.         sw.Stop();
    21.  
    22.         Debug.Log($"Pathfinding took {sw.ElapsedMilliseconds} ms");
    23.     }
     
  2. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,069
    Burst compiles asynchronously in the background. It's likely your first frame is still running the job on mono and hasn't switched over to burst yet.
     
  3. jRocket

    jRocket

    Joined:
    Jul 12, 2012
    Posts:
    607
    That doesn't sound very good. Should I do some kind of pre-warming to make sure everything is burst compiled? How do I know when burst is ready?
     
  4. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    2,628
    The asynchronous compilation happens only in the editor. It is pre-compiled in builds.
     
    jRocket likes this.
  5. jRocket

    jRocket

    Joined:
    Jul 12, 2012
    Posts:
    607
    To answer my own question, I just needed to turn on Synchronous Compilation in the Jobs->Burst menu.
     
  6. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,069
    The reason this isn't on by default is it could take over a minute to enter play mode once you have 1000+ jobs.

    You can mark individual jobs to be synchronously compiled if required.
     
  7. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    473
    As far as I know, Unity stores burst compiler results. So everything is not recompiled every time.
     
  8. Lee_Hammerton

    Lee_Hammerton

    Unity Technologies

    Joined:
    Jul 26, 2018
    Posts:
    92
    To quickly explain how burst compilation works :

    In editor, compilation is scheduled asynchronously whenever a job is to be executed. If the job has previously been compiled (the code has not changed etc), the job will execute the burst compiled method (compiled methods are cached to disk - currently 1 dll per job). If the job has not been previously compiled, the code will run the mono version of the job. Synchronous compilation forces the execute method to wait for the burst compilation to finish (stalling until its ready).

    In player, all burst compiled code is compiled at the point the player is built into a single dll containing all jobs, synchronous has no affect in players.


    Eager compilation (relatively new) in Editor, will compile things as soon as possible when changes are detected and you are not in play mode. This attempts to ensure code is compiled and available by the time you enter playmode.
     
unityunity