Search Unity

Audio When will Unity support low-latency audio on Android?

Discussion in 'Audio & Video' started by jiyongman, Oct 13, 2019.

  1. jiyongman

    jiyongman

    Joined:
    Dec 10, 2018
    Posts:
    29
    1. As a Google Android audio developer pointed out long ago, OpenSL ES is always the correct choice over AudioTrack.

    2. As the developer above said, matching buffer size is really important to achieve low-latency and to avoid a bug on Android M.

    3. Google announced AAudio to replace the outdated OpenSL ES, and they are encouraging developers to use it.

    But in reality, in 2019, not only doesn't Unity support AAudio, but Unity is still struggling to support OpenSL.

    Your link says :
    "Android: Use OpenSL instead of AudioTrack on more devices. This reduces audio input and output latency."

    What do you mean by MORE here?
    It had to be ALL.
    In the beginning, there's no performance benefit just by using OpenSL ES, but starting Android 4.1,
    OpenSL ES got better and better.
    Unity once used to support old Android devices, so AudioTrack was useful back then, but now Unity only supports Android 4.1 and up.
    So why Unity still uses AudioTrack on some devices in 2019?
    Really don't get it.
    Many third-party libraries including FMOD Studio API always use OpenSL ES on Android 4.1 and newer.

    Also, it doesn't seem like Unity sets the output buffer size properly, at least on my Samsung Galaxy S7 Edge.
    (I'm not talking about the DSP Buffer Size settings in the Project Settings menu here. I'm talking about OpenSL ES settings.)
    When I use audio libraries written by me or third parties on my S7 Edge, I could achieve very low latency compared to Unity's built-in audio feature.
    I added "AT LEAST" above, because, on my S10 Plus, Unity's built-in audio worked almost the same as mine.

    As for AAudio, it first came out two years ago, and now almost all audio libraries from third-parties already support it.
    Even FMOD Studio API added it about two months ago.
    (I know Unity uses the older version, FMOD Ex, internally.)
    AAudio is just like Vulkan for audio on Android, and it came out to replace the old OpenSL ES,
    so the Unity team really needs to support this.
    Imagine Unity without Vulkan. How ridiculous. The same thing is happening with your audio feature.

    As you may know, Android's audio latency issue is really serious.
    We can't develop high-quality audio related apps and games for Android forever using Unity if this is not fixed by you.
    Not all Unity users know well about the Android platform like me.
    I know you, developers there, are really busy doing super cool creative things, but I think if you really care about Android users, you can partner with the Google Android audio team.
    Because they are really willing to improve their user experience, they are working hard educating app developers and already partnered with many audio app and hardware companies to solve their problems.
    Unity is the world's best development engine especially for Android, so Google will help you if you just ask them.
    I hope Unity will bring good news soon.

    NOTE : You don't want to use AAudio on Android 8.0. AAudio should be used on Android 8.1 and later, because Android 8.0's AAudio has a bug related to a race condition with the Java garbage collector.
    Otherwise, the app will likely crash.
     
    Last edited: Oct 13, 2019
    Vectrex likes this.
  2. Vectrex

    Vectrex

    Joined:
    Oct 31, 2009
    Posts:
    267
    Bump. I've been looking this up and it really it something Unity needs to look into. Also they could use the "Oboe" opensource lib as it abstracts AAudio and the fallback OpenES and uses whatever the device supports. https://github.com/google/oboe
     
  3. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,305
    well there you have one of the reasons, right there
    It might be possible they will change and update their 'last mile' platform abstraction facing HW in DOTS/DSPGraph, but I personally don't have high hopes for that tbh

    I suggest you both @jiyongman and @Vectrex file a bugreport / feedback for this issue, though I can assure you they have good overview of existing resources and libraries available ;)
     
  4. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    the year is 2024, cars fly and robots do the dishes... no, wait, did Unity take care of this Android audio latency thing?
     
  5. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,717
    Last edited: Jan 13, 2024
  6. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Initially the low-latency mode was disabled because the AAudio code path caused panning issue on certain devices. That last issue was recently mitigated by work on the device manufacturer side and ours, so low-latency mode has recently been enabled.
    As usual, it takes some time for these updates to trickle from the latest branch to the LTS versions. It should be there soon if it's not already there.
     
  7. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    Panning issue as in erratic panning left-right? Also in mono clips with spatialization turned off?
     
  8. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    I think it was the left-right issue, on some Pixel phones IIRC
     
  9. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    By the way, how do you measure audio latency?
    I've taken 240 fps video of stuff happening on screen in response to audio input.
     
  10. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Generally it's by adding up how many samples are buffered before being sinked by the hardware, plus the hardware's "reported" latency. It's super hard to get precise measurements for bluetooth stuff! Also, generally that doesn't take into consideration the time between the `Play()` call and the audio callback, which depends on the game-engine-to-audio-engine implementation.
     
    laurentlavigne likes this.
  11. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    And aren't sound devices connected to the CPU via USB? That's a bit of lag isn't it. I'd be interested in seeing your numbers, from CPU command to sound out with all the software layers and devices in between.
     
  12. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Well... I don't have serious benchmark to offer right now, in the sense that we did not conduct a serious investigation on the matter since I've joined.

    What I can tell you though, is that our current Unity implementation generally uses 4 buffers (so 4 x the buffer size you're setting) and in the case of Windows, WASAPI is used in shared mode, which gives roughly a [20, 40] ms latency (could be less in exclusive mode, because you would skip mixing layers and probably other stuff). I can't provide anything precise about hardware-induced latency... something like[10, 50] ms from specs I remember.

    Some napkin maths:
    • With a 48000kHz, 256 samples buffers, you'd get 256 * 4 / 48000 -> ~20 ms
    • +=WASAPI -> ~[40, 60] ms
    • += some hardware -> ~[50, 110] ms
     
    marcoantap likes this.
  13. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    That's interesting, thanks for sharing.
    And wow! That's tons of lag.
    I was experimenting with the ASIO driver that keijiro made and get 12ms from input to visual so I'm wondering, why isn't Unity using ASIO internally?
     
  14. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Probably because of compatibility... You can certainly setup custom audio patterns on your own hardware, but when it comes to distribute a game, you couldn't guarantee the availability of reliable ASIO drivers.

    mais bon... that's only my two cents!
     
    laurentlavigne likes this.
  15. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    Oh I didn't know ASIO drivers were fickle, then the choice of super laggy but reliable drive that makes sense, after all, not everyone is making rhythm games.
    How is the lag on Quest 2 and 3?
     
  16. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Good question.. I can't actually answer that as I've never touched this area yet. :confused:
     
  17. wildgoose789

    wildgoose789

    Joined:
    Feb 7, 2023
    Posts:
    5
    Hi,
    I think I may have the same issue here. From which version should the latency be fixed for android? For me it's an issue when building to a Quest 3 - absolutely no latency when running it in the editor on a desktop but as soon as built to quest - noticeable lag. 'Best Latency' is of course selected in audio settings. I have tried this on multiple versions including the 2022.3.20f1, 2023.2.9f1, 2023.2.12f1.... Any thoughts would be welcome!
     
    LevonVH likes this.
  18. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,321
    Are they staffing your team during this big reset?
     
  19. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    409
    Hi sir, sorry just got back from a month off. We have to replenish our original headcount first, then I hope we'll see some additional investment. Can't say for sure actually..