Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

ARCore EarlyUpdate.TangoUpdate 15-20 msecs per frame

Discussion in 'Handheld AR' started by Flarup, Sep 5, 2019.

  1. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    While doing profiling for our ARCore based game, I noticed that EarlyUpdate.TangoUpdate have started taking about 15-20 msecs per frame. I remember this being an issue back in the early days of ARCore (https://forum.unity.com/threads/earlyupdate-tangoupdate-performance.522400/), but I haven't seen that problem for a long time - until now.

    In order to debug it, I profiled the HelloAR scene from the ARCore SDK unity package, and noticed that the problem was there as well.

    Anyone knows what's causing this problem? Or it is really expected that ARCore takes so much CPU?

    Some more info:

    Device: Samsung Galaxy S8
    Unity: 2019.1.14f
    ARCore SDK: 1.11.0
    IL2CPP scripting backend
    64 bit Android builds

    Thank you in advance very much for all help.


    Kind regards,
    Uffe Flarup
     
  2. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    553
    What is the entire frame time?

    By default, the ARCore update will block until the next frame is received. So it can appear to take significant time when it isn't really doing anything. If the entire frame time is over 30 ms, then it sounds like it could be a different problem.
     
  3. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    First, I downgraded to ARCore SDK 1.9.0, because that seemed to work better than 1.11.0. But the issue is still noticeable there. Here's a typical frame:

    upload_2019-9-8_14-51-3.png

    I have disabled "Match Camera Framerate" in the ARCore session config, because I would like the game to render at 60 FPS if possible, even though the camera may only be 30 fps.

    Edit : The same game runs almost 60 fps at an iPhone SE using ARKit. So gameplay code and rendering seems to be ok.
     
  4. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    553
    Your original post mentioned "EarlyUpdate.TangoUpdate" taking 15-20 ms per frame. In the screenshot you posted, it only takes ~7ms. Is that typical, or are you still seeing 15-20 ms? Also, what features have you enabled in ARCore?
     
  5. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    Thanks a lot for the quick reply @tdmowrer.

    Regarding "EarlyUpdate.TangoUpdate", I was a bit unclear in my previous post. But I discovered that downgrading from ARCore SDK 1.11.0 to 1.9.0 improved this. It took around 15-20 msecs when I used SDK 1.11.0, but when switching to SDK 1.9.0 it now "only" takes around 6-8 msecs. And the picture above is taken from the profiler when using ARCore SDK 1.9.0.

    My ARCore 1.9 session config looks like this (in my own game, not for the HelloAR sample scene):

    upload_2019-9-9_10-40-48.png

    I have tried enabling "Match Camera Framerate", but that didn't change a whole lot.



    When testing the HelloAR scene from the ARCore SDK (in a separate Unity project), the results are like this:

    When using ARCore SDK 1.11.0 I would get TangoUpdate times around 15-20 msecs even for the HelloAR scene from the SDK.

    When using ARCore SDK 1.9.0 I would get TangoUpdate times of only 1-2 in the HelloAR scene, which I would assume is "expected".

    However, I have so far been unable to figure out which setting is causing TangoUpdate to take 6-8 msecs in my own game. Or even worse, 15-20 msecs when upgrading to ARCore SDK 1.11.0.
     
  6. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    553
    I just ran HelloAR on a Pixel 1 with the latest https://github.com/google-ar/arcore-unity-sdk in 2019.2.2f1. If I disable the "Match Frame Rate" option in the session config, the TangoUpdate takes very little time (less than 1ms). Have you tried any other devices besides the S8?

    Edit: I got the same result on an S9. I'll see if I can track down an S8.
     
    Last edited: Sep 10, 2019
  7. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    I have tried with a few other devices as well, with same problems (a Sony Xperia and a Samsung A50).

    However, we're using Unity 2019.1.14 currently, so there's a difference there in our setup.

    I'll try with 2019.2 in the very near future and see if that affects something.
     
  8. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    I have just tried ARCore SDK 1.11.0 and Unity 2019.2.4 and the HelloAR scene from examples (with "match camera framerate" disabled). Even though it's my best test profiling so far, there are still a lot of spikes in TangoUpdate, as shown in this Profiler picture:

    upload_2019-9-10_19-38-16.png

    In many frames TangoUpdate takes less than 1 msec, which is super nice, but there are still a lot of spikes where TangoUpdate takes 8-10 msecs, making it difficult to maintain a solid 60 fps for the game as a whole.

    The above Profiler picture was samples from a Samsung Galaxy S8. I tested on a Samsung Galaxy S50 as well, and that one had most frames TangoUpdate taking less than 2 msecs, but still quite a few spikes up to 8 msecs.
     
  9. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    Did some more testing, still using Unity 2019.2.4 and ARCore SDK 1.11.0.

    Turns on the type of device is making a big difference. Testing on a Sony Xperia XZ1 (which has lower specs than the Galaxy S8), and the profiling looks like this for the Sony:

    upload_2019-9-12_9-50-38.png

    And running the same apk on a Samsung Galaxy S8 looks like this:

    upload_2019-9-12_9-51-13.png

    The ARCore service is updated to latest version on both devices.

    Any idea what possibly could cause this?
     
  10. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    Hi again @tdmowrer. I have made a repro project and uploaded it here:

    https://drive.google.com/drive/u/0/folders/0B2qTW6czfX_FSFFNczdYbzB0WWs

    There's also an apk in the archive, build from my machine. As mentioned earlier. It runs super smooth on our Sony device, but quite bad on various Samsung devices.

    All help on figuring out why TangoUpdate is consuming so much CPU is very highly appreciated.

    Thanks a lot in advance for all help.
     
  11. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    When testing with ARCore SDK 1.12.0 I can in the profiler see Semaphore.WaitForSignal showing up, as a part of TangoUpdate:

    upload_2019-9-17_14-42-11.png
     
  12. fredsa43

    fredsa43

    Official Google Employee

    Joined:
    Aug 31, 2016
    Posts:
    8
    @Flarup As an aside, note that currently only Pixel 2 / 2 XL / 3 3 XL devices support 60 fps camera capture frame rates.
     
  13. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    @fredsa43 Thanks for the info. Getting it running smoothly at stable 30 fps would also be fine for us.

    The main problem we're having right now is that EarlyUpdate.TangoUpdate takes so much time on Samsung devices, and after few minutes of gameplay you can really start to feel the phones are getting warm, it drains the battery, and reduces performance (maybe the phone reduces how many resources the app gets, since it can detect that the app drains the battery). After a few minutes of gameplay it's very common for us to have EarlyUpdate.TangoUpdate take more than 20 msecs per frame.

    However, on the Sony Xperia test device we have it runs much better, as the above pictures from the Unity profiler shows.

    The game itself runs super smooth on an iPhone SE using ARKit, which have much less power than the Galaxy S8 device.
     
  14. fredsa43

    fredsa43

    Official Google Employee

    Joined:
    Aug 31, 2016
    Posts:
    8
  15. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
  16. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    @fredsa43 Is there any news regarding this performance issue? It has been there since ARCore SDK 1.9 at least, and is really holding us back. Thank you very much in advance for all help regarding this issue.
     
  17. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    553
    Catching up on this thread, it sounds like the issue is seen when trying to run at 60 fps. ARFoundation assumes ARCore runs at 30 fps, so I would expect "spiky" behavior if running at 60 fps as devices that only support 30 fps will have a frame available (and thus take little time) on one frame and then do lots of work on the next frame.

    Beyond that, it depends a lot on the features you've enabled. Face tracking, for instance, can take upwards of 40 ms on older devices. The "EarlyUpdate.TangoUpdate" profiler time you're seeing maps directly to ArSession_update so there's not much we can do to improve that.
     
  18. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    We're not (yet) using ARFoundation for this sample. It's based on ARCore Unity SDK directly.

    Also, we have very few features enabled. Basically, it's just plane tracking. Light estimation, face detection, image detection have ALL been disabled.

    As you can see in the 2 profiler screenshots above, the time taken en EarlyUpdate.TangoUpdate is MUCH longer on the Samsung Galaxy S8 device than on the Sony device (which is a less powerful device). And after 1-2 minutes of play time on the Samsung device I can feel the device is getting really warm and is draining battery.

    I'm pretty sure the problem is in the ARCore library.


    Edit : When testing it in our game we can see that iPhone SE using ARKit runs smoother than a Samsung Galaxy S8 using ARCore.
     
    Last edited: Oct 5, 2019
  19. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    @fredsa43 Have you had time to look at our bug repro project (https://drive.google.com/drive/u/0/folders/0B2qTW6czfX_FSFFNczdYbzB0WWs) and profile it on a Samsung device?

    I tested it again earlier this week, and performance is clearly much worse on Samsung devices than it is on Pixel or Sony devices. Since Samsung devices are very popular among Android users, it would be really nice to have it working properly on Samsung devices as well.
     
  20. Flarup

    Flarup

    Joined:
    Jan 7, 2010
    Posts:
    106
    Hi @tdmowrer and @fredsa43. I have just tested it again with the newly released version of ARCore SDK 1.3, on the latest stable version of Unity (2019.2.10).

    EarlyUpdate.TangoUpdate behaves a bit differently now. Some of the time it takes 1-2 msecs on a Samsung Galaxy S8 device, which is what I fully would expect when I compare it to an equivalent Sony or Google Pixel device (where it runs perfectly). However, after a little while it all of the sudden takes 8-12 msecs instead, which causes big performance problems on Samsung Galaxy devices.

    It's like the ARCore library "gets out of sync" with the camera on the Samsung device for some reason. Nothing changes in the test scene, and nothing changes with respect to real world floor or lightning that explains the sudden increase in time that EarlyUpdate.TangoUpdate takes.

    Any help on how to fix this performance problem on Samsung devices would be much appreciated.

    upload_2019-10-29_21-39-35.png