Search Unity

Video Recording using mobile device camera and microphone

Discussion in 'Audio & Video' started by 3dprasad, Oct 16, 2019.

  1. 3dprasad

    3dprasad

    Joined:
    Aug 12, 2010
    Posts:
    6
    Hi,

    As per our requirement we need to be able to record a video using device camera and it should also contain audio from device mic.
    We have gone through several solutions regarding this including NatCorder, pmjo's Next Gen Recorder, OpenCV for Unity etc. But none of them so far seems to completely satisfy our requirements.

    What we are trying to achieve is as follows.
    1. Record a video using device camera and mic.
    2. Functioning entirely in Unity (no native dialog or popup)
    3. Audio and video settings being configurable.
    4. Getting output as a file from device storage (its ok if its a temporary file)
    Please suggest a solution in which we can achieve this.
     
  2. pmjo

    pmjo

    Unity Technologies

    Joined:
    Sep 17, 2013
    Posts:
    245
    There is no way to avoid the native permission dialog, you cannot access the microphone without asking the user (even when using Unity microphone). Assuming you are talking about iOS or Android :)
     
  3. pmjo

    pmjo

    Unity Technologies

    Joined:
    Sep 17, 2013
    Posts:
    245
    If you only need to record the microphone audio (not microphone mixed with app audio) you can record the Unity microphone by adding the Next Gen Recorder (iOS only) AudioSourceRecorder component on top of your microphone AudioSource. To get rid of the possible echo, follow this workaround link. Unity microphone is not good, you need to tweak many settings to make it work.. and still the result is not very good.
     
  4. ArthurWammes

    ArthurWammes

    Joined:
    Oct 23, 2018
    Posts:
    1
    Hello Pauli, can you explain more how to use the AudioSourceRecorder. I have downloaded the NextGenRecorder. But there is no example file. Do you have a link to a sample?
     
  5. pmjo

    pmjo

    Unity Technologies

    Joined:
    Sep 17, 2013
    Posts:
    245
    Hi! You could implement a regular microphone capture code like described in many Unity forum posts and then drop the AudioSourceRecorder component on top of the AudioSource that plays back the microphone. But instead of doing that you could actually try a script that @xzodia created a while ago. I'm pretty sure it gives a lot better results.

    Here it is:
    Code (CSharp):
    1. using System;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using System.Runtime.InteropServices;
    5. using UnityEngine;
    6. namespace pmjo.NextGenRecorder
    7. {
    8.     [AddComponentMenu("Next Gen Recorder/Microphone Recorder")]
    9.     public class MicrophoneRecorder : Recorder.AudioRecorderBase
    10.     {
    11.         int SampleRate = 16000;
    12.         const int RecordingLength = 10;
    13.         AudioSource audioSource;
    14.         string micName;
    15.         private void Awake()
    16.         {
    17.             Recorder.RecordingStarted += Recorder_RecordingStarted;
    18.             Recorder.RecordingStopped += Recorder_RecordingStopped;
    19.         }
    20.         private void OnDestroy()
    21.         {
    22.             if (gCHandle.IsAllocated)
    23.                 gCHandle.Free();
    24.             if (ListenCoroutine != null) StopCoroutine(ListenCoroutine);
    25.             Recorder.RecordingStarted -= Recorder_RecordingStarted;
    26.             Recorder.RecordingStopped -= Recorder_RecordingStopped;
    27.         }
    28.         private void Recorder_RecordingStopped(long sessionId)
    29.         {
    30.             Microphone.End(micName);
    31.         }
    32.         private void Recorder_RecordingStarted(long sessionId)
    33.         {
    34.             AudioConfiguration configuration = AudioSettings.GetConfiguration();
    35.             if (configuration.sampleRate < 8000)
    36.             {
    37.                 Debug.Log("Only sample rate >= 8000 is supported");
    38.                 return;
    39.             }
    40.             else
    41.             {
    42.                 SampleRate = configuration.sampleRate;
    43.             }
    44.             if (Microphone.devices.Length == 0)
    45.             {
    46.                 Debug.LogError("No Microphone Available");
    47.                 return;
    48.             }
    49.             if (audioSource == null)
    50.                 audioSource = gameObject.AddComponent<AudioSource>();
    51.             micName = Microphone.devices[0];
    52.             Debug.Log("Starting Microphone: " + micName);
    53.             audioSource.clip = Microphone.Start(micName, true, RecordingLength, SampleRate);
    54.             audioSource.loop = true;
    55.             audioSource.mute = true;
    56.             while (!(Microphone.GetPosition(micName) > 0)) { }
    57.             audioSource.Play();
    58.             Debug.Log("Microphone Initialized");
    59.             ListenCoroutine = StartCoroutine(Listen(audioSource.clip.channels, 2));
    60.         }
    61.         Coroutine ListenCoroutine = null;
    62.         GCHandle gCHandle;
    63.         int readSize;
    64.         bool AwaitingFilterRead;
    65.         IEnumerator Listen(int readChannels, int writeChannels)
    66.         {
    67.             int MaxSamples = SampleRate * RecordingLength;
    68.             float[] fReadData = new float[MaxSamples * readChannels];
    69.             float[] fWriteData = new float[MaxSamples * writeChannels];
    70.             int unwrappedReadHead = 0;
    71.             int readHead = 0;
    72.             readSize = 0;
    73.             gCHandle = GCHandle.Alloc(fWriteData, GCHandleType.Pinned);
    74.             if (gCHandle.IsAllocated == false)
    75.             {
    76.                 Debug.LogError("Failed to allocated GC Handle");
    77.                 yield break;
    78.             }
    79.             while (true)
    80.             {
    81.                 yield return null;
    82.                 if (AwaitingFilterRead) continue;
    83.                 int writeHead = Microphone.GetPosition(micName);
    84.                 if (writeHead < unwrappedReadHead) writeHead += MaxSamples;
    85.                 int positionDelta = writeHead - unwrappedReadHead;
    86.                 if (positionDelta == 0) continue;
    87.                 audioSource.clip.GetData(fReadData, readHead);
    88.                 for (int wc = 0; wc < writeChannels; wc++)
    89.                 {
    90.                     var rc = wc % readChannels;
    91.                     for (int i = 0; i < positionDelta; i++)
    92.                         fWriteData[(i * writeChannels) + wc] = fReadData[(i * readChannels) + rc];
    93.                 }
    94.                 unwrappedReadHead = readHead + positionDelta;
    95.                 readHead = unwrappedReadHead % audioSource.clip.samples;
    96.                 readSize = positionDelta * writeChannels;
    97.                 AwaitingFilterRead = true;
    98.             }
    99.         }
    100.         private void OnAudioFilterRead(float[] d, int c)
    101.         {
    102.             if (readSize > 0)
    103.             {
    104.                 base.AppendInterleavedAudio(gCHandle.AddrOfPinnedObject(), readSize, c, SampleRate);
    105.                 readSize = 0;
    106.                 AwaitingFilterRead = false;
    107.             }
    108.         }
    109.     }
    110. }
    Thanks for this script @xzodia :)
     
    ArthurWammes likes this.
  6. xzodia

    xzodia

    Joined:
    Jan 24, 2013
    Posts:
    50
    @pmjo No problem, one thing I have noticed since writing this though, if the user needs to grant microphone permissions at the start of the recording, the video will continue to record in the background. I haven't gotten around to fixing this yet, but its just a case of checking the permissions before starting the recording.
     
  7. auroraDev

    auroraDev

    Joined:
    Nov 22, 2017
    Posts:
    12
    I encountered a problem in the GetPosition Loop after I used the Native iOS Video Recording view.
    Do you know why??

    My Device is iphoneX
     
  8. pmjo

    pmjo

    Unity Technologies

    Joined:
    Sep 17, 2013
    Posts:
    245
    I haven't personally been playing much with this. Sounds like something goes to a bad state. Maybe you should try to close the microphone when the app goes to background by calling End. Like @xzodia mentioned, you also need to implement microphone permission checks to use the above script. It would also be a good idea not to loop forever and timeout if Microphone.GetPosition part loops too long and maybe try to restart or give up after that.

    https://docs.unity3d.com/ScriptReference/Application.RequestUserAuthorization.html