Search Unity

Question Main thread is waiting on the rendering thread, causes FPS drop from 72 to 40 in my Android game

Discussion in 'VR' started by masta-yoda, Feb 28, 2021.

  1. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    I have an Oculus VR game with a mostly static scene with some objects, light, animations, scripts processing, etc. I noticed that the frame rate is not perfect, it doesn't run at the desired 72 FPS and when I look at objects in the scene drops to 40 FPS, if I look away from all objects in the scene, I have a steady 72 FPS.

    I run the profiler with the game running on the Oculus (Android) device and got the following picture. When I have the FPS drop, my Main Thread is wasting most of the time waiting for the render thread which is still busy with the previous frame processing. And also, when the render thread starts processing the current frame, it waits for the Main thread which was delayed earlier. So they kind of waiting on each other but mostly render thread is the bottleneck here (did I read the profiling results correctly?). Unfortunately, I don't have any visibility on what is lagging here, is it high poly objects, lights, other rendering elements? How can I get that visibility in the profiler results to understand what to optimize in my scene?

    shot_210228_023324.png
     
  2. tree_arb

    tree_arb

    Joined:
    Dec 30, 2019
    Posts:
    323
    If you look at objects and the fps drops a lot, try swapping those objects to mobile shaders if applicable. That will make a big difference. Also baked lights and try quaility= low, disable shadows. In my personal situation when i look at an object and it drops fps, its usually the shader.
     
  3. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    Thanks! I experimented with specific objects and found that it's partially related to the triangles count and the other big part was real-time light (thanks for the tip!). I have many more scenes to check, is it possible to clearly say from the profiling session result that FPS drop is related to light for example?
     
  4. colinleet

    colinleet

    Joined:
    Nov 20, 2019
    Posts:
    189
    You can turn on GPU profiling in editor/dev build, Google it. I wasted so much time last year trying to figure out how to diagnose performance issues like this. Long story short VR is unrelenting and brutal: if you go over your GPU budget by like 0.1 ms it will simply not render the next frame/half your frame rate. You need to plan on having at least a half ms of extra time for when the device warms up and clocks down, or best several ms clear overhead to allow for lower end device/bad thermals (yup 10-25% to be safe).
     
    Last edited: Mar 3, 2021
    masta-yoda likes this.
  5. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    >Long story short VR is unrelenting and brutal: if you go over your GPU budget by like 0.1 ms it will simply not render the next frame/half your frame rate

    Oh, it sounds like exactly what happens to me! Thanks for making me confident about GPU profiling. I'm using Vulkan and the Unity profiler says that it's unable to profile GPU with Vulkan enabled, so switch from Vulkan and give it a try!
     
  6. tree_arb

    tree_arb

    Joined:
    Dec 30, 2019
    Posts:
    323
    My findings are that OpenGLES 3 is running faster than vulkan on my android devices (im using default render pipeline).
     
    masta-yoda likes this.
  7. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    Hm.. interesting observations! I switch to Vulkan to run VFX on Android (Oculus VR), it wasn't working on OpenGLES 3 (don't recall what was the limitation), but probably it's a good time to switch to OpenGLES 3 and see if I can run it
     
  8. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    I have switched to OpenGLES3 and at scenes, without VFX the performance was better, I saw the FPS increase from 50 to 70, but when I activate VFX, it just renders it incorrectly and FPS drops to 20. I'm curious if it even possible to run an Oculus Quest game with VFX on OpenGLES3 instead of Vulkan?

     
  9. tree_arb

    tree_arb

    Joined:
    Dec 30, 2019
    Posts:
    323
    Maybe that's where vulkan shines.

    I don't have a quest or experience with vfx graph so my experiences my differ.

    If by VFX you mean particle systems, i have downloaded many vfx/particles systems from the assets store and never encountered anything that doesn't renter on my android phones using OpenGLES3 default render pipeline.

    Maybe vfx graph is different, to be sure you aren't overloading the gpu like colinleet said, maybe export a new empty project with just the vfx? Or, now with opengles3 maybe you can get gpu data in the profiler.

    When i download any vfx particle systems from the asset store, i make all efforts to change the shaders to (mobile>particles>.. ) it really makes all the difference in the world for performance... sometimes that's not an option and if so, i try not to use them.

    Did you Set quality to low and disable shadows and anti aliasing?

    Also, try a test with 'optimize frame pacing' on and off. Im finding i get better performance with it off on Android, which is not expected behavior.
    Project Settings > Player > Settings for Android > Resolution and Presentation > optimize frame pacing (check box)
     
    masta-yoda likes this.
  10. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    Thanks for the great recommendations, I'll try all of them! I think I start from a sample project and see if I can repro. Also, I saw similar complaints from other devs, and it somehow related to URP (I'm also on URP). I'll try to repro on a simple VFX (partial system drawn on GPU).
     
  11. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
  12. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    Even though my VFX doesn't work on GLES3, I tried to profile my Lobby scene with this Graphic API enabled and got interesting results. My FPS is still low (around 40) and I can still see that my Main Thread is waiting for Render Thread (~20 ms) but GPU is just doing profiler EndQueries work (~11 ms) so even less than the wait time. I can see the same poor performance with profiler disabled so I'm not sure how to read this data:

    upload_2021-3-15_0-28-44.png

    If I inspect GPU work, it spends this 11 ms rendering opaque and transparent objects:
    upload_2021-3-15_0-33-49.png

    These numbers don't add up. @Treecrotch it sounds like you have good experience with the profiler, could you please advise?
     
  13. tree_arb

    tree_arb

    Joined:
    Dec 30, 2019
    Posts:
    323
    I wish i could advise but i doubt i know any more about profiling then you do. I just try to target the expensive thing and temp disable objects if needed to narrow it down if i can't figure it out by the profiler alone.

    Sometimes ill put toggle buttons in the menu to turn off various things at runtime to see what happens to frame rate or profiler. Including turning off the UI which mine has a lot of transparent items...at least target which things are the root cause

    Im working on a mobile (non vr) game on android and those tips i posted earlier where things that happen to help my framerate, but I can't say im all that knowledgeable yet w deep profiling.

    I was unable to get the same high framerate on URP on Android as i was the default render pipeline. So i gave up and stuck with default render pipeline for now.

    Our projects differ in that, if you use vfx graph and urp, i have default pipline and mostly basic mobile shaders.

    Im trying to stick my self in this limited box because it seems to keep high framerate but sounds like you don't have that option with your type of project.
     
    masta-yoda likes this.
  14. colinleet

    colinleet

    Joined:
    Nov 20, 2019
    Posts:
    189
    It's still saying that your GPU time is around 11.6 ms... The threshold for 90 fps == 11.1.1111 ms -- which means that it is waiting the full extra frame. You'll need to shave about 1 ms before the GPU timing will stop driving your frame rate off a cliff (allowing for minimal fluctuations where it needs with the extra .5 ms)... Go for 8.5/9.5 ms on the GPU if you want to to be in smooth/occasionally rough waters.
     
    tree_arb likes this.
  15. masta-yoda

    masta-yoda

    Joined:
    Apr 19, 2020
    Posts:
    91
    Yeah, I had that option before I knew the implication of URP + VFX, now I have to deal with it on Android. Thank's for the great insights!
     
    tree_arb likes this.