Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug GPU Profiling not supported by Oculus Quest with URP 11-12 for Unity 2021.1 and 2021.2

Discussion in 'VR' started by SoumyaRipan86, Aug 12, 2021.

  1. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    Hi,

    When I am using profiler for Oculus Quest, it shows that GPU profiling is not supported by this device. It has been very helpful to use the Unity's GPU profiler for understanding issues and saving a ton of time.

    I have tried with 2021.1.7f1 with URP 11 and 2021.2.0b6 with URP 12. But no luck.

    Is there any ways to make it work or which version of Unity it will work with ?


    Screenshot 2021-08-12 103811.png
     
  2. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Hi there. We are just now getting to the point where we can take GPU timings on all platforms in 2021.2/2022.1 but this so far only works for the ProfilerRecorder API. Revamping the GPU profiler to properly represent those measurements is going to still take a while longer.

    In the meantime, the only way to get GPU profiling going on the unsupported platforms is to use a platform or GPU manufacturer specific profiler.
     
    Rory_B and hippocoder like this.
  3. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Facing the same issue. Is there any preferred workaround, perhaps a user created tool that works? I've tried using the Snapdragon profiler but it gives rendering information at an OpenGL level which is hard to understand.
     
  4. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Maybe RenderDoc can help?
     
  5. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    It was so convenient with the GPU profiler to find bottlenecks without much guess work and efforts. Also the way it showed the details at high level was very useful.

    Knowing those GPU timings(for different parts of the renderings) was so helpful before further debugging and optimizing things with renderdoc/frame-debugger. I haven't found any good alternative to the GPU profiler yet. :(

    Hope you guys fix it on priority. Sure a lot of devs who switched to URP are praying for this.
     
    Last edited: Aug 13, 2021
  6. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    @MartinTilo
    Hey does the GPU profiler work by any chance with 2021.1.7f1 version of Unity in a builtin project ?
    I tried it with a builtin project with the above version and it doesn't work.
     
  7. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    The GPU Profiler will tell you if it worked and if not, it tries to give as actionable a reason as possible (e.g. that it doesn't work with a given graphics API (so you could switch to another one) or with Graphics Jobs (so you can turn of Graphics Jobs))

    The only thing that changed over the course of 2021-2022.1 is that ProfilerRecorder can get GPU timings. The GPU Profiler Module still needs to be reworked because it's build as if the GPU works the same as a CPU, which was kinda ok back in the day to structure the information in an accessible way, but is now standing in the way of displaying the measurements the new GPU instrumentation emits in a proper way.

    In short: Nothing changed for the GPU Profiler Module, yet.

    Keep an eye out for release notes or the documentation for updates on when that changed or check out the profiling roadmap page.
     
  8. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    I'm still quite confused @MartinTilo .

    Does this mean that any Android/ARM project using Unity 2021+ (regardless of rendering pipeline) won't be able to use the GPU Usage part of the profiler? Assuming it's on OpenGL and Graphics Jobs is turned off.

    Because that's the case for me when I try debugging the GPU usage with OpenGL, Graphics Jobs off.

    I have tried using Unity 2021 + URP 11 + Oculus Quest -> GPU usage just says "This device is not supported".
    I have tried using Unity 2021 + Built-in + Oculus Quest -> GPU usage just says "This device is not supported".

    Sorry, I'm just trying to understand this better.

    To clarify, this worked in 2019.3.11f1 + Builtin + Oculus Quest
     
    Last edited: Aug 19, 2021
  9. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    It is indeed confusing o_O
    The more I've thought about it and chatted with colleagues, the less it makes sense that this would just suddenly change for the same device, Graphics API and particularly the mostly unchanged built in RP...

    Could you please file a bug for this and post the issue ID here so I can follow up on it? We need to bisect this to find out what could have regressed that support and figure out if that was intentional/ unavoidable or something that could be fixed...
     
  10. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Done. id - 1359552

    Thanks for taking the time to help.
     
    Last edited: Sep 8, 2021
    MartinTilo likes this.
  11. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    @MartinTilo Hi, sorry for the ping but any updates on this issue?
     
  12. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    It hasn't been processed by QA yet. Might be slow going due to summer vacations.
     
    colinleet and mutp like this.
  13. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Did some more testing - GPU usage turns up for these
    Unity 2020.3 + URP 10.6 + Oculus Quest
    Unity 2020.3 + Built in + Oculus Quest

    At this point I have to strongly consider going to back to Unity 2020, because the Unity GPU usage is a crucial feature for us. Would love to know why this is happening in 2021.1 asap so that I can revert back if I have to with a clear conscience. Is it be possible for you to test this directly or if there's any other way to quicken this process.
     
  14. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    I sadly don't have an Oculus Quest device handy :/
    QA has picked it up now but no further insights yet. I checked out the Project and found that it has Graphics Jobs enabled. Technically the GPU Profiler should tell you that that is not supported, so I'm not sure if that is indeed the setting your builds ran with and the issue at hand. If it is, at least something broke with regards to how we report GPU Profiling availability and reasons for it not to be available.
     

    Attached Files:

  15. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    @MartinTilo
    Hi, thanks a lot for helping.

    So I have tried out with Graphics Jobs disabled and other Project Settings options as well, following the unity forums regarding gpu profiler issues. But in all these cases it just doesn't work and empty.

    Even if I disable Graphics Jobs in settings, then the empty GPU profiler just shows that the device is not supported.
     
    Last edited: Sep 8, 2021
  16. colinleet

    colinleet

    Joined:
    Nov 20, 2019
    Posts:
    188
    For anyone else reading this wondering how to use the currently working and available URP/HDRP profiling tools in 2020.1+ @MartinTilo was able to help me though all of the step on getting a few basic profiler recorders working for GPU last winter here:

    https://forum.unity.com/threads/how-to-use-gpu-profiler-stats.1058645/

    I ended up making a script which logs all the currently available gpu profilers and let's you see when they become available to the developer (a few seconds after launch normally), which I posted in the above thread. (Note: These only work in editor and dev builds for anything pre-2021.1). This definitely isn't a strong replacement for a full GPU profiler but I was able to make it into a makeshift frame rate graph in game, which has been night and day for allowing me to see when I'm creeping up to 90hz in game (super important for VR, since going beyond simply half's the frame rate).
     
    Last edited: Sep 8, 2021
    MartinTilo likes this.
  17. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    I guess that included turning off the Player Setting "Enable Frame Timings" and making sure the graphics API is a supported one?
    Then I remain stumped and we'll have to wait how the bug report progresses...
     
  18. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    Yes, I had tried with this before. Just to be sure, I tried it again now with both "Graphics Job" and "Enable Frame Timings" turned off. Doesn't work.

    For graphics API I have been using OpenGL which is the supported one for GPU usage Profiler. (So I disable Auto Graphics API option to remove Vulkan and keep OpenGL API only)
     
    MartinTilo likes this.
  19. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    MartinTilo likes this.
  20. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    @MartinTilo

    Hi, is there any update on this, has the devs already picked up this issue after the above QA check?
    Sorry, for asking, but will be helpful to know if there is any progress on this.
     
  21. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    QA bisected it to a version in which we fixed a different bug related to Frame Timing Manager ([Case 1274505] FrameTimingManager.GetLatestTimings returns a smaller gpuFrameTime than it actually is, fixed in 2021.1.0a8, 2020.3.18f1):oops:. We haven't 100% confirmed that this bugfix caused the GPU Profiler to fail for Open GLES, but it seems rather likely. Open GLES only allows one system to insert time queries as it's working on a stack for these and insertions from other systems would through the whole thing apart. So it seams reasonable that fixing FrameTimingManager and GPU Recorders would throw the GPU Profiler off for Open GLES.

    We'll keep digging and looking for a resolution to this and apologize for this slipping through the cracks and causing this regression and pain on your side. A likely resolution might be that you'll have to choose in the Player Settings which of the three GPU measuring systems you'll want to have working (GPU Profiler, Recorder / ProfilerRecorder APIs, or FrameTimingManager) as, for the time being, only one can work at a time. Future facing, the GPU Profiler Module UI will eventually be redone to use the same back-end as ProfilerRecorders. Also FrameTimingManager is being rewired to use ProfilerRecorders under the hood. With these changes the conflict between these would be fully resolved, but that's not something we'd be able to backport, thus the likelihood for this to be fixed via an option, if a smarter, more dynamic solution isn't possible.

    So, we'll look into this and have a rough idea on how to fix it, so I wouldn't yet downgrade because of this bug. As a really ugly workaround, you might be able to use FrameTimingManager.GetLatestTimings() in the meantime, as that should now be working...:confused:

    Sidenote: looks like this is not just a Quest issue, but a general Android Open GLES one.:oops:
     
  22. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    Thank you for explaining why this bug started happening and what's the possible resolution being planned.

    Yes, having an option to select any of the three GPU measuring systems will be helpful if it runs these systems one at a time and the GPU profiler starts displaying the data soon. This will save us a lot of time and will be able to use the GPU profiler again soon, while the full GPU Profiler Module with Profiler are being redone.

    After researching on the forums and a lot of consideration we had upgraded the game from 2019.3.11f1 builtin to 2021.1.7f1 urp but sadly didn't expect this bug and the GPU profiler has been very helpful to debug the specific rendering costs in the levels. The project is too big now and downgrading only for this signle issue will take up a lot of time and uncertainty of breaking the project. I will take your advice and stick to this version of unity hoping for a GPU Profiler quick fix sooner in the upcoming Unity 2021.x updates.

    I had also tested with an android build on mobile phone and PC build sometimes back, to know if the issue is device specific or platform. Yes, the GPU profiler didn't ran for android mobile builds as well. Then wondered, this should have been a common issue between mobile(android) game devs but there was not many posts about it on the forums.:confused:

    Thanks again for helping out. :)
     
    Last edited: Sep 21, 2021
  23. SoumyaRipan86

    SoumyaRipan86

    Joined:
    Aug 19, 2019
    Posts:
    12
    @MartinTilo

    Can you please give a rough time estimate for the fix to the GPU Profiler, with the "Player Settings" option you mentioned? That will help a lot in making progress and decisions with the game.
     
  24. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    I sadly can't yet give one, no :/
     
  25. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Could you give provide code using this api that will show a similar GPU frame timing breadown as the profiler? Specifically, I'm looking for a way to get the detailed info that the GPU usage UI gives like Skybox render time, Render.OpaqueGeometry etc, not just set pass calls.

    I've been scourging the forums to get a better understanding of this and understand it's limitations since I really don't want to implement yet another Unity workaround only to get stuck again. Posts like this -> https://forum.unity.com/threads/fra...esttimings-always-returning-0-timings.544325/ don't give the highest confidence that this will work for Android. Also, is there any known issue with this approach that we should be aware of?

    Also, in one of your responses on that thread, you mentioned
    >Btw, as of 2020.1 Recorder can already get you GPU timings where the profiler UI can too, but only on Development builds. The newer ProfilerRecorder API allows us to also expose some of the stats in Release.

    Is that also an option to get the same info as the GPU profiler UI? I'm still confused by all this.
     
    Last edited: Sep 30, 2021
  26. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    I understand things a bit more, especially the ProfilerRecorder API.
    I'm able to get the timing info of some markers using the code provided from this talk


    Am I doing this right? And, is there an easy way to list everything (hierarchically) the way the GPU usage profiler does or do I need to code that myself? Where is the hierarchical relationship available between the statistics? GetAvailableStats doesn't have this information.

    Furthermore, I can view the stats on my PC in the game view, whereas in the Oculus Quest (which I haven't tested yet), I'll still only be able to view the stats in the headset (assuming it works). Connecting via IP address to a Unity window seems like a pretty big task in itself, so I'm wondering if there any other options for this. Perhaps logging to file from the Quest could work.
     
    Last edited: Sep 30, 2021
  27. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    The Recorder API works for GPU timings on 2020.1 and up, ProfilerRecorder API works for GPU timings on 2021.2 and up, if this option is passed to the constructor.

    Since the GPU Profiler assumes a tighter coupling of CPU and GPU calls, the hierarchy view in the GPU is a bit of an odd mix of CPU and GPU markers, not everything is profilable as GPU markers. In 2021.2 you can use ProfilerRecorderHandle.GetAvailable() to get a list of all available GPU markers (Categorized as Rendering but sadly indistinguishable from pure CPU markers), keeping in mind that they'll only be listed 3 frames after the code they instrument first ran, due to the 3 frame GPU delay.

    And no, the recorders record flat time without hierarchy. Construction of the hierarchy needs full profiling to be on and is constructed in the receiving profiler, based on the begin/end times of samples in the stream. That's too much overhead for the recorders to deal with. Besides, the hierarchy approach is helpful but kinda not accurate for how the GPU works anyways. One reason why the GPU Profiler module needs rebuilding.

    As for sending the data to the Editor, you can use the PlayerConnection on the Player side and the EditorConnection on the Editor side, leveraging the same connection as Console & Profiler use.
     
    Last edited: Oct 1, 2021
    mutp likes this.
  28. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Thanks for the the response!

    One more question - using ProfilerRecorderHandle.GetAvailable() I get all the Rendering markers of which some do seem like GPU markers (I'm on 2021.1). Does this mean that while querying these, I'll just get 0 as the return value since I'm not on 2021.2?
     
  29. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    When recording them with a GPU enabled ProfilerRecorder (2021.2+) or a Recorder (2020.1+ via it's .gpuElapsedNanoseconds property), you should be getting GPU timings for them, unless the same conflict that disabled the GPU Profiler Module has affected them too.
     
    mutp likes this.
  30. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Oh, yeah, another way to report the numbers back to the Editor would be via ProfilerCounters.
     
  31. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    So, I'm not getting any GPU markers in an Oculus Quest build -> only CPU stats.
    Getting GPU, CPU stats in a PCVR build. Using Recorder API (2021.1).

    PCVR build -
    upload_2021-10-2_19-53-38.png

    Quest build (attachment)
     

    Attached Files:

    Last edited: Oct 2, 2021
  32. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Hmm, right so the GPU recorders also have a hard conflict with FrameTimingManager... So yeah, looks like the only GPU stat you can get until we fix it (we've picked the bug up by now btw) is the GPU frame time stat on the FrameTimingManager
     
    mutp likes this.
  33. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Okay, that's unfortunate. So confused what to do now. Downgrading Unity, URP versions to 2020.3 will be a nightmare.

    Do you have an estimate for when a fix will land? Also, will it be backported to the earlier Unity versions?
     
  34. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    I do not have an ETA yet but yes, we do plan on backporting the fix to all affected versions that will still be supported by the time was have a fix, as it is a regression on LTS and those versions.
     
  35. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    I'd hold out on nightmare scenarios for as long as you can, as we'll eventually get there, and see if you can bridge the gap with native platform tools like these or GPU manufacturer specific ones?
     
  36. mutp

    mutp

    Joined:
    Oct 1, 2018
    Posts:
    79
    Got it! Thanks a lot for everything, by the way. You've answered all my questions so quickly and intelligently, I really appreciate it! I certainly understand profiling a lot more now!

    Looking forward to the fix :)
     
    MartinTilo likes this.
  37. M-Hanssen

    M-Hanssen

    Joined:
    Sep 8, 2014
    Posts:
    34
    @MartinTilo
    *bump* GPU profiling on Android devices still not working in the latest stable Unity versions...
    Any news on the fix?
     
  38. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Hello,
    Yes, I just checked in with the dev working on the fix and it is in a merge request but there are still remaining issues with the current fix that keep it from landing. But it seems to be down to one issue on WebGL now, so hopefully soon it'll land and then get backported
     
  39. SynodicArc

    SynodicArc

    Joined:
    Mar 7, 2015
    Posts:
    3
    Going to add that this issue is definitely blocking us a bit, so fingers crossed you guys will be able to get that fix in soon!
     
  40. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Update: the outstanding issues have been fixed, the merge request is getting another look and testing and then will hopefully land soon™
     
    mutp likes this.
  41. JeffWTD

    JeffWTD

    Joined:
    Dec 17, 2012
    Posts:
    6
    I see on the Issue page that this is fixed in 2022.1.X, does this mean that it will not ever be fixed for 2021.2 or the upcoming 2021 LTS?
     
  42. GrahamHuws

    GrahamHuws

    Unity Technologies

    Joined:
    Mar 14, 2022
    Posts:
    20
    Hi @JeffWTD - We intend to add this to 2021.3, though too early to give you an ETA. We'll be looking at the feasibility for 2020.3 as well. Thank you for your patience!
     
    MartinTilo likes this.
  43. Hombresinombre

    Hombresinombre

    Joined:
    Jun 6, 2020
    Posts:
    2
    I am right now trying to profile in 2021.3 and it is still a problem. Isn't this implemented yet?
     
  44. GrahamHuws

    GrahamHuws

    Unity Technologies

    Joined:
    Mar 14, 2022
    Posts:
    20
    Hey, this should be fixed in 2021.3.2f1. If the profiler is unable to record data because of the timing incompatibilities, it should give an error message linking you to the appropriate settings to change:

    Screenshot 2022-06-07 160302.png
     
    Last edited: Jun 7, 2022
    Hombresinombre likes this.
  45. Hombresinombre

    Hombresinombre

    Joined:
    Jun 6, 2020
    Posts:
    2
    Oh thanks! Sadly, I am using Vulkan for better perfomance and it seems to be incompatible. Is there any ETA for the profiling working with it? Them same with Graphics Jobs
     
  46. GrahamHuws

    GrahamHuws

    Unity Technologies

    Joined:
    Mar 14, 2022
    Posts:
    20
    Sorry, no ETA yet for Vulkan compatibility but we're eager to rectify that!
     
    AldeRoberge likes this.