Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Help Wanted Playback pcm-data (float-array) on audiosource

Discussion in 'Audio & Video' started by smaschmann98, Sep 16, 2020.

  1. smaschmann98

    smaschmann98

    Joined:
    Dec 10, 2019
    Posts:
    1
    Greetings everyone,

    I'm currently working on a voice-chat solution for a game in WebGL. Since there is no asset nor Unity natively supporting this kind of feature in any way, I had to come up with my own solution. I started by figuring out how to record the users microphone in the browser and send in via WebRTC to multiple participants. On each end I'm resampling the audio and converting the data to a float-string-array in Javascript, which is then send back in to Unity via SendMessage(GameObject, String).
    This part works fine and I can confirm that the data is transmitted correctly to each an every participant, since I played back the received stream with an native html-audio-element. I need to play back the audio in Unity instead of doing it this was is, that I want spatial-audio for each person, since the should feel like talking in a room.
    Now, lets look into the problem: The send data is received by the corresponding "Speaker"-GameObject of the participant as string of floats and converted to a float-array, which I play back as soon as the data is received.
    But unlike in the native html-audio-element the audio is "cracking" and "stoping" at some times. My guess is that it is a timing / data set issue on Unitys-Audioclip / -Audiosource, because of my misunderstanding on how to use it right. Furthermore this also happens when I play a sound (float-array with the same value for each element) just to confim it is not the false/wrong data recorded in Javascript.
    This is the code of the class doing the work in Unity. If needed I can also provide a link to the running WebGL-Version.

    Code (CSharp):
    1.  
    2. using Photon.Pun;
    3. using System;
    4. using System.Linq;
    5. using System.Runtime.InteropServices;
    6. using UnityEngine;
    7.  
    8. [RequireComponent(typeof(AudioSource))]
    9. public class Voice : MonoBehaviourPun
    10. {
    11.  
    12.     private AudioSource source;
    13.     private int samplePos = 0;
    14.  
    15. #if UNITY_EDITOR
    16.     private float[] test = new float[256];
    17. #endif
    18.  
    19.     [DllImport("__Internal")]
    20.     private static extern void connectAudio(string userid);
    21.     [DllImport("__Internal")]
    22.     private static extern void disconnectAudio();
    23.  
    24.     private void Start()
    25.     {
    26.         gameObject.name = $"VoiceHandle:{photonView.ViewID}";
    27.         source = GetComponent<AudioSource>();
    28.         source.clip = AudioClip.Create("Voice", 44100 * 1, 1, 44100, false);
    29.         source.Play();
    30.         if (photonView.IsMine)
    31.         {
    32. #if UNITY_WEBGL && !UNITY_EDITOR
    33.             Debug.Log($"onstart {gameObject.name}");
    34.             connectAudio(photonView.ViewID.ToString());
    35. #else
    36.             for (int i = 0; i < test.Length; i++)
    37.             {
    38.                 test[i] = -.1f;
    39.             }
    40. #endif
    41.         }
    42.     }
    43.  
    44. #if UNITY_EDITOR
    45.  
    46.     private void Update()
    47.     {
    48.         PlaybackAudio(test);
    49.     }
    50.  
    51. #endif
    52.  
    53.     private void OnDestroy()
    54.     {
    55.         if (photonView.IsMine)
    56.         {
    57. #if UNITY_WEBGL && !UNITY_EDITOR
    58.             disconnectAudio();
    59. #endif
    60.         }
    61.     }
    62.  
    63.     private void WriteAudioData(string data)
    64.     {
    65.         Debug.Log("onwrite");
    66.  
    67.         PlaybackAudio(StringToFloatArray(data));
    68.     }
    69.  
    70.     private float[] StringToFloatArray(string value)
    71.     {
    72.         return value.Split(',').Select(f => Convert.ToSingle(f)).ToArray();
    73.     }
    74.  
    75.     internal void PlaybackAudio(float[] data)
    76.     {
    77.         Debug.Log($"onplay {gameObject.name}:{data.Length}:{source.isPlaying}:{samplePos}");
    78.         source.clip.SetData(data, 0);
    79.         //samplePos = (samplePos + data.Length) % 44100;
    80.         source.Play();
    81.     }
    82.  
    83. }
    84.  
    Thanks for any help!
     
unityunity