Audio Playing float samples and recording some.

Hello, everyone.

I'm currently sending and receiving audio float samples over the network. Using a coroutine, I empty the execution queue (the play queue if you prefer) As much as possible I'd like to use the Unity Framework for compatibility.


The playing code. The problem there is that the sound is choppy, but the execution time is less than 5ms.

Code (CSharp):

1.  
2.             Enqueue(() =>
3.             {
4.                 try
5.                 {
6.                     m_RecordedClip = Microphone.Start(DefaultMicrophone, Loop, LengthSec, Frequency);
7.                     float[] samples = new float[m_RecordedClip.samples];
8.                     m_RecordedClip.GetData(samples, 0);
9.                     while (!samples.Any(a => a != 0))
10.                     {
11.                         m_RecordedClip.LoadAudioData();
12.                     }
13.                     callback(samples);
14.                 }
15.                 catch (Exception ex)
16.                 {
17.                     Debug.Log(ex);
18.                 }
19.             });
20.  

The recording code. If I don't comment the loop, it loops forever, because somehow there is no data loaded. Any ideas why ?

Code (CSharp):

1.            
2. Enqueue(() =>
3.             {
4.                 m_AudioClip.SetData(samples, 0);
5.                 while (!(m_AudioClip.GetData(samples, 0) && samples.Any(a => a != 0)))
6.                 {
7.                     m_AudioClip.LoadAudioData();
8.                 }
9.                 m_AudioSource.clip = m_AudioClip;
10.                 m_AudioSource.PlayScheduled(0d);
11.             });
12.  

 

Given the lack of response I expect you have either given up or found another way, but in general:

- "Busy-while" loops such as the one you have there basically break Unity; they starve the Unity game loop since Unity is basically single-threaded in the game engine. See https://answers.unity.com/questions/1466599/microphonegetpositionnull-hangs-forever.html

- In general trying to capture audio data from the network is almost certainly going to have lots of issues with packet dropping, glitching, and other problems. There are reasons why media streaming software has long buffers (1/2 second or more). You don't seem to be doing or planning any buffering here, so you're definitely vulnerable to glitching.

- Driving your Unity sound capturing from the Unity game loop is potentially a problem because of Unity garbage collection pauses and other issues.

Unity is probably not your best bet for streaming audio over the network, even if it means less portability.