NatMic - Native Microphone API

Hey @Lanre is any progress here with NatMic ? Can't release the app with these bugs.

 

Any ETA for 1.3.1? We are getting ready to send out a product. Is the ArgumentException: GCHandle value belongs to a different domain a concern that could cause any issues? Or will the error not cause any issues aside from appearing in logs?

 

Build is ready. Email me with your invoice number.

 

Emailed you the build.

 

This bug has been fixed. The only reason NatMic 1.3.1 isn't being released yet is cos we're trying to reimplement the MixerDevice. Email me with your invoice number and I'll share the build with you.

 

Hey Lanre -
Curious what your timeline is around adding AAC support (which I assume is whats meant by M4A support in the original post). I'm in a situation where I need to record audio to a file using natmic - then later record video to a file using natcorder - then use ffmpeg to splice those streams together. Since WAV isn't supported by mp4 I have to go through a (rather long) conversion process to splice everything together that ends up being a poor user experience.

I'll be able to deal with it in the meantime - but was was hoping to get official support at some point. Thanks!

 

We won't be adding AAC support. It'll be going farther out of what NatMic's core functionality is. The simplest thing for you to do is to use NatCorder to record a black video with the audio you want (so you get a video file with an AAC audio track); then use FFmpeg to extract said track.

 

I am very much looking forward to the update of NatMic!

 

Emailed you. Thanks!

 

Hi!
We have same exception withNatmic:
"ArgumentException: GCHandle value belongs to a different domain"

 

Just wanted to follow up here and say that the latest build Lanre is sharing privately fixes one of my two issues, which was AirPods support, for me, in the latest, AirPods work. My hang issue still remains, I have contacted Lanre via email regarding this - but just posting here to keep everything in sync. Just to iterate, if I can do anything to help track down the issue please let me know (can I get xcode to do a crash dump when it is hanging and provide you that perhaps? I am not so familiar with xcode/ios/apple but I presume it is possible to do this somehow)?

 

Okay, I found out how to create corefiles, so have created one when it hangs on iOS and sent it over to Lanre. For anyone else interesting, the commands you can issue, via Xcode to create a core dump are:

Code (CSharp):

1. process save-core /tmp/corefile

For me it took a long time to create the coredump (1.9GB) as in probably well over an hour (I had left it going for 30 minutes not sure if it was hanging, before noticing the file size was slowly going up, so I left it overnight and it was done when I got in this morning).

 

Sorry for the late response. Checking it out now.

 

Hi I update to 1.3.1
but i also have this problem
---------------------------------------
when I call audioDevice.StopRecording() throw exception

AndroidJavaException:
java.lang.NullPointerException: Attempt to invoke virtual method 'void java.lang.Thread.interrupt()' on a null object reference
--------------------------------------

Run Environment:
Unity 2018.3.0f2
Android 8.0.0
SAMSUNG Galaxy A7

 

Can you share your code? You need to make sure that you cache a reference to the audio device you are recording from.

 

This is my code

Code (CSharp):

1.  
2.     private MP4Recorder videoRecorder;
3.     private RealtimeClock recordingClock;
4.     private CameraInput cameraInput;
5.     [SerializeField] private Camera RecordCamera;
6.     private IAudioDevice audioDevice;
7.  
8.     public void StartRecording()
9.     {
10.         var sampleRate = 44100;
11.         var channelCount = 2;
12.         // Start recording from the main camera
13.         recordingClock = new RealtimeClock();
14.         videoRecorder = new MP4Recorder(videoWidth, videoHeight, 60, sampleRate, channelCount, RecordingCallback);
15.         cameraInput = new CameraInput(videoRecorder, recordingClock, RecordCamera);
16.         // Start the microphone
17.         audioDevice = AudioDevice.GetDevices()[0];
18.         audioDevice.StartRecording(sampleRate, channelCount, this);
19.     }
20.  
21.     public void StopRecording()
22.     {
23.         // Stop the microphone
24.         audioDevice.StopRecording();
25.         // Stop recording
26.         cameraInput.Dispose();
27.         videoRecorder.Dispose();
28.         cameraInput = null;
29.         videoRecorder = null;
30.     }

 

Hi I update to 1.3.1
but i also have this problem
-
DllNotFoundException: NatMic
NatMic.Internal.AudioDeviceiOS.GetDevices () (at Assets/NatMic/Plugins/Managed/Internal/AudioDeviceiOS.cs:21)
NatMic.AudioDevice.GetDevices () (at Assets/NatMic/Plugins/Managed/AudioDevice.cs:30)
NatMic.Examples.VoiceRecorder.Start () (at Assets/NatMic/Examples/VoiceRecorder/VoiceRecorder.cs:20)
-
Run Environment:
Unity 2018.3.14f1 (64-bit)
win10

 

Can you try deleting and reimporting NatMic from the Asset Store? NatMic.dll should be in the NatMic > Plugins > Windows directory, and in the inspector, it should be set to apply to the Editor and Windows Standalone builds.

 

I downloaded it from the asset store and it is a new project.

 

Hi @Lanre
Running the VoiceRecorder case will give you an error, and friends around me have tried it.

 

Hello,

NatMic - Microphone API issue

I got error DllNotFoundException : NatMic, this error is for window 10 Home Edition, for Win 10 Pro 64 bit working fine.
Check Image for full error. Also attached the error log.

 

Can you run `msinfo32` using the Windows run box and share screenshots?

 

HI!

I also have found (on another computer, not mine) a "DLL Not Found" error for NatMic.
The problem I think is with the "GetDevices" function (here an extract from the error log)

DllNotFoundException: Unable to load DLL 'NatMic': The specified module could not be found.
at NatMic.Internal.AudioDeviceiOS.GetDevices () [0x00000] in <00000000000000000000000000000000>:0

Unluckily I don't know how to reproduce it on my computer (here it works fine)! :/

 

Can you try installing the latest Visual C++ redistributable (vcredist)? I'm not sure why this happens.

 

Hello
when I call audioDevice.StopRecording() throw exception

AndroidJavaException:
java.lang.NullPointerException: Attempt to invoke virtual method 'void java.lang.Thread.interrupt()' on a null object reference

Run Environment:
Unity 2018.3.0f2
Android 8.0.0
SAMSUNG Galaxy A7

My Code

Code (CSharp):

1. private MP4Recorder videoRecorder;
2.     private RealtimeClock recordingClock;
3.     private CameraInput cameraInput;
4.     [SerializeField] private Camera RecordCamera;
5.     private IAudioDevice audioDevice;
6.  
7.     public void StartRecording()
8.     {
9.         var sampleRate = 44100;
10.         var channelCount = 2;
11.         // Start recording from the main camera
12.         recordingClock = new RealtimeClock();
13.         videoRecorder = new MP4Recorder(videoWidth, videoHeight, 60, sampleRate, channelCount, RecordingCallback);
14.         cameraInput = new CameraInput(videoRecorder, recordingClock, RecordCamera);
15.         // Start the microphone
16.         audioDevice = AudioDevice.GetDevices()[0];
17.         audioDevice.StartRecording(sampleRate, channelCount, this);
18.     }
19.  
20.     public void StopRecording()
21.     {
22.         // Stop the microphone
23.         audioDevice.StopRecording();
24.         // Stop recording
25.         cameraInput.Dispose();
26.         videoRecorder.Dispose();
27.         cameraInput = null;
28.         videoRecorder = null;
29.     }

 

I hope to use both NatMic and Phton Voice Chat. But when using a photon voice, recording with NatMic is a problem.

Is there any way I can use it with Microphone.Start () at the same time?

using UnityEngine;
using System.Collections;
using System;

// Wraps UnityEngine.Microphone with Voice.IAudioStream interface.
public class MicWrapper : ExitGames.Client.Photon.Voice.IAudioReader<float>
{
private AudioClip mic;
private string device;

public MicWrapper(string device, int suggestedFrequency)
{
if (Microphone.devices.Length < 1)
{
return;
}
this.device = device;
int minFreq;
int maxFreq;
Microphone.GetDeviceCaps(device, out minFreq, out maxFreq);
var frequency = suggestedFrequency;
// minFreq = maxFreq = 44100; // test like android client
if (suggestedFrequency < minFreq || maxFreq != 0 && suggestedFrequency > maxFreq)
{
Debug.LogWarningFormat("[PV] MicWrapper does not support suggested frequency {0} (min: {1}, max: {2}). Setting to {2}",
suggestedFrequency, minFreq, maxFreq);
frequency = maxFreq;
}
this.mic = Microphone.Start(device, true, 1, frequency);
}


}
}

 

This means that the microphone didn't start in the first place. Check if the mic is recording before trying to stop recording. We will change the signature of StartRecording to return a boolean indicating whether recording was in fact started.

 

Unfortunately this is not possible. Only one service can get exclusive access to the mic at any time. This is more of a hardware/OS-level restriction.

 

Using Window 10 trying to get the plugin to work and getting error using the sample scene.

Plugins: Failed to load 'Assets/NatMic/Plugins/Windows/NatMic.dll' because one or more of its dependencies could not be loaded.
NatMic.Internal.AudioDeviceiOS:GetDevices() (at Assets/NatMic/Plugins/Managed/Internal/AudioDeviceiOS.cs:21)
NatMic.AudioDevice:GetDevices() (at Assets/NatMic/Plugins/Managed/AudioDevice.cs:30)
NatMic.Examples.VoiceRecorder:Start() (at Assets/NatMic/Examples/VoiceRecorder/VoiceRecorder.cs:20)

Looking at the DLL dependencies it looks like there are 2 that are debug DLLs

MSVCP140D.dll
VCRUNTIME140D.dll

These are part of the 2015 Visual C++ redistributable (I'm using 2017) but the version shouldn't be an issue. I think it is due to the debug DLLs being referenced. Can you check to make sure you compiled it in release mode or give me some guidance on how to resolve this issue.

Thanks

======================================================
Dump of file NatMic.dll

File Type: DLL

Image has the following dependencies:

KERNEL32.dll
ole32.dll
MSVCP140D.dll
MF.dll
MFPlat.DLL
MFReadWrite.dll
VCRUNTIME140D.dll
ucrtbased.dll

Summary

1000 .00cfg
1000 .data
2000 .idata
1000 .msvcjmc
3000 .pdata
13000 .rdata
1000 .reloc
1000 .rsrc
11000 .text
10000 .textbss

 

I got your email. Forgetting to build in release mode when packaging 1.3.1 was oversight on my part. Update coming in a few days.

 

I'm aware this is not due to NatMic, but I just thought I'd mention this here as a potential issue.

I'm having problems with the ClipRecorder. The AudioClip that is created and returned to the callback method upon Dispose contains no sample data after AudioClip.SetData() has been invoked, when ran on Android.
I'm not sure if this is a known issue, or if there's a workaround.
Using Unity 2019.1.8f1.

 

I wasn't aware of this. You could file a bug report with Unity.

 

Having looked into the issue a bit more, it all seems to stem from the creation of the AudioClip. The number of channels requested results in 0 actually being created for the AudioClip, and thus the AudioClip is unable to have it's data set.

Code (CSharp):

1. Debug.Log("AudioClip.SetData failed; AudioClip " + GetName() + " contains no data");

is invoked from the AudioClip class.
This is for both iOS and Android it'd seem.

I've managed to temporarily work around it, but I just wanted to inform you that this may be an issue for some users.
Hopefully the Unity team are aware and have this solved soon.

 

@Lanre is there a way for us to obtain a pre-release build of Natmic with the latest fixes? Or at least do you have a clear timeline now for the fixes to be released? Many thanks.

 

The update has been submitted. It should be live on Monday or Tuesday.

 

Does NatMic work at all in the Windows Unity editor for 1.3.1? I just purchased this, added it to my "selfie recording" feature, only to have everything grind to a snails pace, and hanging on StopRecording() of the audio device. First tried using the MixerDevice to mix the game audio with the microphone input, just tried it out now with only the microphone input.

Works fine without using NatMic. I haven't tested on Android or iPhone yet. Just trying to avoid the feedback when I pipe the mic into the scene during recording.

My code:

Code (CSharp):

1.     void StartRecording()
2.     {
3.         if (GameManager.gameManager.RecordAction)
4.         {
5.             Debug.Log($"====================================STARTING RECORDING==={Camera.main.pixelWidth} {Camera.main.pixelHeight}============================");
6.             recordingClock = new RealtimeClock();
7.             if (GameManager.gameManager.RecordSelfie)
8.             {
9.                 var microphone = AudioDevice.GetDevices()[0];
10. #if UNITY_EDITOR              
11.                 listener = microphone;
12. #else
13.                 var gameAudio = new VirtualDevice(Camera.main.GetComponent<AudioListener>());
14.                 listener = new MixerDevice(gameAudio, microphone);
15. #endif              
16.             }          
17.             else{
18.                 listener = new VirtualDevice(Camera.main.GetComponent<AudioListener>());
19.             }
20.             if (listener != null)
21.             {
22.                 mediaRecorder = new MP4Recorder(
23.                     Camera.main.pixelWidth, Camera.main.pixelHeight,
24.                     30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, onRecordingComplete);
25.                 listener.StartRecording(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, this);
26.             }
27.             else
28.             {
29.                 mediaRecorder = new MP4Recorder(
30.                     Camera.main.pixelWidth, Camera.main.pixelHeight,
31.                     30, 0, 0, onRecordingComplete);
32.                 audioInput = null;
33.             }
34.             cameraInput = new CameraInput(mediaRecorder, recordingClock, Camera.main);
35.         }
36.     }
37.  
38.  
39.     void StopRecording()
40.     {
41.         Debug.Log("====================================ENDING RECORDING===============================");
42.         // Destroy the recording inputs
43.         audioInput?.Dispose();
44.         cameraInput?.Dispose();
45.         cameraInput = null;
46.         audioInput = null;
47.         if (listener != null && listener.IsRecording)
48.         {
49.             listener.StopRecording();
50.             listener = null;
51.         }
52.         // Stop recording
53.         mediaRecorder?.Dispose();
54.         mediaRecorder = null;      
55.     }
56.  

 

Got it. Works great!

 

Is the editor slowing down when you are recording the microphone? Can you share your OnSampleBuffer method?

 

Code (CSharp):

1.     public void OnSampleBuffer (float[] sampleBuffer, int sampleRate, int channelCount, long timestamp) {
2.         // Send sample buffers directly to the video recorder for recording
3.         mediaRecorder.CommitSamples(sampleBuffer, recordingClock.Timestamp);
4.     }
5.  

Right out of the example. I see a new version of NatMic is up, but I can't help but wonder if this is an issue with the MixerDevice code.

UPDATE: I did get this working with the Unity Microphone as a source, after the update. It's "good enough" that I can move forward (Just looking for user to comment during action, while keeping game audio rolling without microphone feedback). I use an AudioSource piped into a muted AudioMixer for the microphone, which kills any feedback, but gives me input for mixing.

NEW UPDATE: Sigh.... code works in Windows Editor, but NatCorder isn't recording any sound on Android (Moto G7, Android Version 9). Back to square one....

 

We're in the process of removing the MixerDevice implementation because we haven't been able to synchronize audio from the microphone with audio from Unity. If you want to record mic + game audio, a quick and dirty way would be to record from a microphone that does not have echo cancellation, so it hears the outside world and the game audio.

 

@Lanre FYI building for Android appears to be an issue with 1.3.2:

Assets/ThirdParty/NatMic/Plugins/Managed/Internal/AudioDeviceBridge.cs(56,28): error CS0161: 'AudioDeviceBridge.StartRecording(IntPtr, int, int, AudioDeviceBridge.SampleBufferCallback, IntPtr)': not all code paths return a value

FYI, the workaround is to change:

Code (CSharp):

1.         public static bool StartRecording (this IntPtr device, int sampleRate, int channelCount, SampleBufferCallback callback, IntPtr context) {}
2.  

To:

Code (CSharp):

1.         public static bool StartRecording (this IntPtr device, int sampleRate, int channelCount, SampleBufferCallback callback, IntPtr context) {return false;}
2.  

 

This has been fixed. Simply re-download NatMic from the Asset Store.

 

When the recording and video are played together, the video sound will become smaller. How can I solve it?
and I used this code also useless.
AVAudioSession *session = [AVAudioSession sharedInstance];
NSError *setCategoryError = nil;
[session setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker error:&setCategoryError];

 

Hello
My natmic can not recording on android

Environment:

Unity:2019.2.0f1
Device: Samsung Galaxy A7
Android: 8.0.0

Code (CSharp):

1.  
2. audioDevice = AudioDevice.GetDevices()[0];
3. audioDevice.StartRecording(44100, 2, this);
4. Debug.Log(audioDevice == null);
5. Debug.Log(audioDevice.IsRecording);
6.  

audioDevice is not null
but audioDevice.IsRecording always return false

 

NatMic already sets the category with the option to default to speaker. I don't think iOS will allow changing this.

 

Can you share the logs from logcat in a .txt file? Also, you might want to change the channel count to 1 as some microphones don't support stereo.

 

Thank you for your reply

I find the question is I didn't get mic permission.

 

Hello,
Just purchased NatMic, and it's working great with NatCorder to record audio. Thank you.

- I was wondering, is there any way to get the levels from an AudioDevice?

This would be in order to update a microphone monitor for the user, so that they can confirm that their selected input is in fact receiving audio.


The implementation for the Unity native Microphone class seems to be provided here:
https://forum.unity.com/threads/check-current-microphone-input-volume.133501/

I'd much rather not have to instantiate a Unity native Microphone in addition to the NatMic AudioDevice, just to update an audio monitor.... any advice?

Thank you

 

Hi,

I think the latest release of NatMic and NatCorder is causing my iOS Unity Cloud Build to fail, because their latest version of Xcode is only 10.2, is it possible for me to get an older version of NatMic?

Thank you.

 

Is the category AVAudioSessionCategoryPlayAndRecord? I have purchased a plugin for another account. Can you send me a copy of the iOS source code? My project is very urgent because of this problem. I just started using unity mic api also had that problem, and I solved the problem by set that category and option. but when I use natmic, that method is useless. I need help, please.