Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question AudioClip.PCMReaderCallback has insane latency

Discussion in 'Audio & Video' started by gwenvis, Dec 14, 2020.

  1. gwenvis

    gwenvis

    Joined:
    Aug 1, 2013
    Posts:
    18
    EDIT:
    For anyone finding this post in need of some extra context or a solution. I gave up with trying shortly after I posted this thread. After a year, I decided I would try again. Here is what I found:

    the OnAudioFilterRead callback works fine. The delay was because of the way I read the microphone. I didn't wait for the microphone to reach position 0 in samples. (With Microphone.GetPosition) before reading the data. Resulting in a delay.

    For the PCMReaderCallback I could not find a solution. The latency of half a second still exists, even with a small DPSBufferSize or low sample count. It always seems to pre-buffer. This is unfortunate, because this callback works properly with spatializing (with Steam Audio too!) while the OnAudioFilterRead needs some extra effort to make that work.


    original post
    Hello, I'm trying to implement VOIP in my game using Opus and Mirror. My efforts so far don't seem to be making any progress in solving the latency issue that I get.

    When the audio is received in the client, I play it using the AudioClip.PCMReaderCallback where I just change the data to the latest info, but this seems to have a big latency of about 500ms/1s, which is just way too long.

    I've also tried using the OnAudioFilterRead method, but the samples there are completely wrong and makes the voice sound very choppy, and like a chipmunk, but then the latency is minimal.

    What's up with this? Does anyone know how to fix the delay issue that I get with the callback?
     
    Last edited: Dec 23, 2021
    achimmihca likes this.
  2. achimmihca

    achimmihca

    Joined:
    Feb 13, 2016
    Posts:
    288
    I just stumbled upon this issue.

    This script demonstrates the huge delay of PCMReaderCallback.
    - sound is played via left mouse button
    - if using OnAudioFilterRead then there is minimal delay
    - if using PCMReaderCallback (OnAudioRead method) then there is a huge delay

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. [RequireComponent(typeof(AudioSource))]
    4. public class PcmReaderCallbackTest : MonoBehaviour
    5. {
    6.     public int sampleRate = 44100;
    7.     public int outputChannelCount = 2;
    8.     public int lengthInSamples = 2048;
    9.    
    10.     private AudioSource audioSource;
    11.     private int totalSampleIndex;
    12.  
    13.     private bool shouldPlay;
    14.     private bool oldShouldPlay;
    15.  
    16.     private bool filledSound;
    17.     private bool oldFilledSound;
    18.    
    19.     private void Awake()
    20.     {
    21.         audioSource = GetComponent<AudioSource>();
    22.        
    23.         // Comment out this line to use OnAudioFilterRead instead of OnAudioRead
    24.         audioSource.clip = AudioClip.Create("PcmReaderCallbackTest", lengthInSamples, outputChannelCount, sampleRate, true, OnAudioRead);
    25.        
    26.         audioSource.Play();
    27.     }
    28.  
    29.     private void Update()
    30.     {
    31.         shouldPlay = Input.GetMouseButton(0);
    32.         if (shouldPlay != oldShouldPlay)
    33.         {
    34.             Debug.Log("shouldPlay: " + shouldPlay);
    35.             oldShouldPlay = shouldPlay;
    36.         }
    37.     }
    38.    
    39.     private void OnAudioRead(float[] data)
    40.     {
    41.         FillBuffer(data, outputChannelCount);
    42.     }
    43.  
    44.     // Uncomment this method to use OnAudioFilterRead instead of OnAudioRead
    45.     // private void OnAudioFilterRead(float[] data, int channelCount)
    46.     // {
    47.     //     FillBuffer(data, channelCount);
    48.     // }
    49.    
    50.     private void FillBuffer(float[] data, int channelCount)
    51.     {
    52.         if (!shouldPlay)
    53.         {
    54.             for (int i = 0; i < data.Length; i++)
    55.             {
    56.                 data[i] = 0;
    57.             }
    58.  
    59.             filledSound = false;
    60.             if (filledSound != oldFilledSound)
    61.             {
    62.                 Debug.Log("filledSound: " + filledSound);
    63.                 oldFilledSound = filledSound;
    64.             }
    65.             return;
    66.         }
    67.  
    68.         int frequency = 440;
    69.         for (int sampleIndex = 0; sampleIndex < data.Length; sampleIndex += channelCount)
    70.         {
    71.             for (int channelIndex = 0; channelIndex < outputChannelCount; channelIndex++)
    72.             {
    73.                 data[sampleIndex + channelIndex] = Mathf.Sin(2 * Mathf.PI * frequency * totalSampleIndex / sampleRate);
    74.             }
    75.             data[sampleIndex] = Mathf.Sin(2 * Mathf.PI * frequency * totalSampleIndex / sampleRate);
    76.             totalSampleIndex++;
    77.         }
    78.        
    79.         filledSound = true;
    80.         if (filledSound != oldFilledSound)
    81.         {
    82.             Debug.Log("filledSound: " + filledSound);
    83.             oldFilledSound = filledSound;
    84.         }
    85.     }
    86. }