Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question UpdateNavMeshDataAsync Locking Main Thread

Discussion in 'Navigation' started by metallibus, Mar 12, 2023.

  1. metallibus

    metallibus

    Joined:
    Jun 1, 2019
    Posts:
    11
    If I submit a bunch of mesh data to NavMesh.UpdateNavMeshDataAsync(), the function itself does in fact run asynchronously, but it's queueing up a bunch of job work that seems to lock up the main thread either on that frame or a subsequent frame shortly after, while some large amount of the work completes. The returned AsyncOperation still isn't marked complete for a few more frames, so some amount of it is happening separately, but I'm consistently seeing one unbearably-long frame along the way.... Is there some way to remove this frame locking/bottleneck and allow the main thread to continue rendering without waiting on parts of the nav mesh jobs?

    These calls seem to be dominated by NavMesh.TileMesh... It seems that the frame doesn't start rendering until a bunch of these are finished, though some do seem to persist beyond the frame render, so not sure if this is really the culprit.

    upload_2023-3-12_15-35-12.png upload_2023-3-12_15-35-37.png

    Basically everything else is idle at this point, so I don't think there's anything else holding up the render timing:
    upload_2023-3-12_15-39-45.png
     
  2. metallibus

    metallibus

    Joined:
    Jun 1, 2019
    Posts:
    11
    So, a little bit more info here...

    After drilling a little deeper, and closing the Scene view while profiling, it looks like the main thread is getting hung up trying to execute some rendering job...

    upload_2023-3-16_9-44-43.png

    It looks to me like the rendering system is trying to use a job for some portion of its logic, seemingly related to shadows, and waiting on the result.... But since the NavMesh system has submitted so many jobs, all the workers are full, so it looks to be switching to help finish those jobs, until its own is then executed....

    This seems insane.... If the render system needs jobs, why does it not have its own worker pool? Why can't it increase its own priority? It seems pretty crazy that submitting enough work to a system that's meant to help unblock main thread work, can in itself block the main thread...

    That being said, maybe this becomes more of a jobs question and less of a NavMesh question... unless there's some way to control how many workers the NavMesh calls use? I don't see anything clear, but since lots of these workers are stuck in mesh tiling, maybe submitting fewer meshes than there are job workers would alleviate this? But it would greatly slowdown NavMeshing performance, especially because I should be able to process multiple meshes per worker per frame, but wouldn't be sure that I won't bottleneck something if I try doing so. This seems a little silly to juggle, and also will mean I have to be careful about what other jobs I'm submitting as well....
     
  3. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,113
    @metallibus I suggest u submit bug report for this issue
     
  4. DwinTeimlon

    DwinTeimlon

    Joined:
    Feb 25, 2016
    Posts:
    300
    It doesn't solve the overall problem I guess, but you actually can set MaxJobWorkers in NavMeshBuildSettings.
     
  5. Whipexx_DigitalSun

    Whipexx_DigitalSun

    Joined:
    Aug 8, 2017
    Posts:
    17
    For a while I wondered if you are someone working on my team, we've been looking at this for a couple days.
    As DwinTeimlon said setting max job workers to less than JobsUtility.JobWorkerCount will free some threads from the greedy Navmesh allocation.
    However this will only reduce the problem: I've seen that this happens a lot when some task A in a thread creates one or more threads and has to wait for them to complete to finish the work. If A goes iddle the threading system will use that thread for other tasks, like TileMesh; this causes A to have to wait until TileMesh completes to end the work. Even if the update to the navmesh is async and does not block the main thread thad task A might not be and that's what's causing the lag spike.

    I believe there might be something causing really long TileMesh executions when they shouldn't, but I'm still trying to blame it on my project and not navmesh. Are you also experiencing abnormally long calls to UpdateNavMeshDataAsync while others are much faster? If that's the case there might be something that's up.
     
  6. metallibus

    metallibus

    Joined:
    Jun 1, 2019
    Posts:
    11
    Just looped back and saw this thread... I had opened a bug report and Unity didn't respond for months and then said it was "not a priority" to fix and that the issue can be "circumvented by using the recommended workflows in the Navigation package". I don't know what "recommended workflow" they mean besides just not doing this at runtime or only using small navmeshes.

    There is another comment from Unity in another thread saying their prioritization changed or something, but I cannot confirm if this actually fixes this issue or not.

    https://forum.unity.com/threads/bad-job-scheduling-halves-performance-in-35504.1145261/#post-8972248
     
  7. DEBBAJfarouk

    DEBBAJfarouk

    Joined:
    Jun 21, 2021
    Posts:
    6
    This problem still persists , when making a procedural infinite terrain the navMeshSurface take about 2 seconds to calculate which freezes the game even tho when I open the profiler it says it running on a worker thread from jobs system and main thread.