NatCorder - Video Recording API

Thanks for the quick reply @Lanre .

Good to know. I can just change my approach to fit within the limitations then.

All the best!

Eric

 

Hi! I've been using NatCorder for a few months and loving it! But recently, after taking a break from recording for a while, I discovered that my videos had an ugly dark green/yellow tint. Any idea why this might be happening?

I have tried resetting all my monitor color profiles (since Google suggested they might be involved), and I deleted some of the other assets I downloaded recently (including one that did its own video export, and which I worried was interfering somehow), but to no avail. I'd very much appreciate any suggestions!

 

Hey there! Are you by any chance recording on Windows? If so, is your project using Linear lighting as opposed to Gamma lighting? NatCorder currently only supports gamma rendering. We are looking at adding support for linear lighting and the new render pipeline infrastructure.

 

I just purchased NatCorder today and found it fairly easy to use and integrate, but I've got a problem and a question.

First, the problem. When I attempt to build an Android apk, I get the following:


I am using Unity 2018.2.11f1, Windows 64-bit.

Now, the question. I am attempting to record an incoming stream. The video is coming through fine, but I cannot figure out how to get the audio to copy. I am using RenderHeads AV Video Pro for my player.

Thanks.

 

Any timelines on the linear support? Even something rough would be useful. It's unlikely we can transition to a purely gamma workflow.

 

hi @Lanre, i bought NatCorder 2 weeks ago. in my project i am using webrtc video chat. everything is fine about video but no sound from audio listener.

i am missing that if am i write any code about recording sound on update or system write it down on OnAudioFilterRead. where should i commitsamples?

Code (CSharp):

1. public void StartRecording(){
2.         var audioFormat = recordMicrophone ? AudioFormat.Unity : AudioFormat.None;
3.  
4.         if(recordSound){
5.             audioFormat = new AudioFormat (AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode);
6.         }
7.  
8.         var videoFormat = new VideoFormat(960, 540);
9.         clock = new RealtimeClock();
10.  
11.         NatCorder.StartRecording (Container.MP4, videoFormat, audioFormat, OnSessionEnd);
12.         videoRecorder = CameraRecorder.Create(myCamera, clock);
13.  
14.         // Start microphone and create audio recorder
15.         if (recordMicrophone) {
16.             StartMicrophone();
17.             audioRecorder = AudioRecorder.Create(microphoneSource, true, clock);
18.         }
19.  
20.         if (recordSound) {
21.             audioRecorder = AudioRecorder.Create (sceneListener, clock);
22.         }
23.  
24.     }

Code (CSharp):

1. void Update()
2.     {
3.         if (NatCorder.IsRecording){
4.             var frame = NatCorder.AcquireFrame ();
5.             myCamera.targetTexture = frame;
6.             myCamera.Render ();
7.             NatCorder.CommitFrame (frame, clock.CurrentTimestamp);
8.         }
9.     }

 

Ah, yes, that's probably it. I don't remember switching over to Linear (I thought I was using that the whole time...), but switching back to Gamma fixed it.

 

Does microphone record work on iOS yet?

 

Replay

Is there a way to record UI or select UI?

 

NatCorder requires Android API level 18 and over. You can set this in Player Settings.

I am not familiar with AVPro's API's, but you need a way to get the audio data from AVPro and send it to NatCorder for encoding. Check out this article for more info.

 

I can try to have a working prototype up by next week. Just check my progress on the NatCorder Extensions repo.

 

If you are playing the WebRTC audio through an AudioSource, then you can create an AudioRecorder for that AudioSource which will extract the audio data. If you have access to the raw uncompressed audio data (PCM sample buffers), then send it to NatCorder for encoding with the CommitSamples function.

EDIT: What platform are you running on?

 

Great!

 

I'm not sure I understand what you're asking. You have always been able to use Unity's Microphone API to record mic audio on every supported platform. Unity's Mic API has glitches though.

 

You can record a UI canvas; simply make sure that the render mode of the canvas is not Overlay.

 

Hmm, so this would necessitate setting the render mode to a specific camera? Which camera would be preferable? How would this work in a multiple camera setup?

 

Yes - just trying to figure out how to use device microphone with this plugin. What should microphoneSource and such be set up as? (if using ReplayCam)

 

Also, just checking if ReplayCam example is supposed to record microphone audio as well on iOS? Can't seem to get it to do that...

 

NatCorder's camera recorder records from one camera. For multiple cameras, you would need to create a multiple-camera recorder.

The ReplayCam example already has this setup. The microphone clip is played through and AudioSource, then an audio recorder is created to record whatever comes out of that audio source, which in this case will be the microphone audio.

Yes. Check that you have enabled the app's microphone permissions. Also make sure that you aren't encountering this Unity bug.

 

Hi Lanre,

I installed the latest version of NatCam and since deploying this I've had the following errors, unfortunately I don't have the logs:

java.lang.Error: FATAL EXCEPTION [NatCorder Video Encoding Thread] Unity version : 2018.2.13f1 Device model : HUAWEI CAM-L03 Device fingerprint: HUAWEI/CAM-L03/HWCAM-H:6.0/HUAWEICAM-L03/C110B155:user/release-keys Caused by
at android.media.MediaCodec.native_dequeueOutputBuffer (Native Method)
at android.media.MediaCodec.dequeueOutputBuffer (MediaCodec.java:2406)
at com.yusufolokoba.natcorder.mp4.Encoder.drainEncoder (Encoder.java:79)
at com.yusufolokoba.natcorder.mp4.Encoder.run (Encoder.java:136)
at java.lang.Thread.run (Thread.java:833)


java.lang.Error: FATAL EXCEPTION [NatCorder Video Encoding Thread] Unity version : 2018.2.13f1 Device model : HMD Global Nokia 1 Device fingerprint: Nokia/Frontier_00WW/FRT:8.1.0/O11019/00WW_0_46C:user/release-keys Caused by
at android.media.MediaMuxer.nativeStop (Native Method)
at android.media.MediaMuxer.stop (MediaMuxer.java:457)
at com.yusufolokoba.natcorder.mp4.MP4Recorder.encoderFinished (MP4Recorder.java:110)
at com.yusufolokoba.natcorder.mp4.Encoder.run (Encoder.java:132)
at java.lang.Thread.run (Thread.java:764)


java.lang.Error: FATAL EXCEPTION [NatCorder Video Encoding Thread] Unity version : 2018.2.13f1 Device model : Xiaomi Redmi Note 3 Device fingerprint: Xiaomi/kenzo/kenzo:6.0.1/MMB29M/V8.5.1.0.MHOMIED:user/release-keys Caused by
at android.media.MediaMuxer.nativeStop (Native Method)
at android.media.MediaMuxer.stop (MediaMuxer.java:239)
at com.yusufolokoba.natcorder.mp4.MP4Recorder.encoderFinished (MP4Recorder.java:110)
at com.yusufolokoba.natcorder.mp4.Encoder.run (Encoder.java:132)
at java.lang.Thread.run (Thread.java:818)

 

Are you recording with audio? And just to confirm, you are using the latest version of NatCorder (1.3f2)? Can you share your recording code?

 

Yes I'm recording with audio it's on 1.3f2:



This is the code:

Code (CSharp):

1.         public void StartRecording()
2.         {
3.             // First make sure recording microphone is only on MP4
4.             recordMicrophone &= container == Container.MP4;
5.             // Create recording configurations // Clamp video width to 720
6.             //            var width = 720;
7.             //            var height = width * Screen.height / Screen.width;
8.             var width = Screen.width / 2;
9.             var height = Screen.height / 2;
10.  
11.             var framerate = container == Container.GIF ? 10 : 30;
12.             var videoFormat = new VideoFormat(width, (int)height, framerate);
13.             var audioFormat = recordMicrophone ? AudioFormat.Unity : AudioFormat.None;
14.             // Start recording
15.             NatCorder.StartRecording(container, videoFormat, audioFormat, OnShare);
16.             //videoRecorder = CameraRecorder.Create(Camera.main);
17.             videoRecorder = CameraRecorder.Create(_camera);
18.  
19.             // If recording GIF, skip a few frames to give a real GIF look
20.             if (container == Container.GIF)
21.                 videoRecorder.recordEveryNthFrame = 5;
22.             // Start microphone and create audio recorder
23.             if (recordMicrophone)
24.             {
25.                 StartMicrophone();
26.                 audioRecorder = AudioRecorder.Create(microphoneSource, true);
27.             }
28.         }
29.  
30.         private void StartMicrophone()
31.         {
32. #if !UNITY_WEBGL || UNITY_EDITOR // No `Microphone` API on WebGL :(
33.             // Create a microphone clip
34.             microphoneSource.clip = Microphone.Start(null, true, 60, 48000);
35.             while (Microphone.GetPosition(null) <= 0);
36.             // Play through audio source
37.             microphoneSource.timeSamples = Microphone.GetPosition(null);
38.             microphoneSource.loop = true;
39.             microphoneSource.Play();
40. #endif
41.         }
42.  
43.         public void StopRecording()
44.         {
45.             // Stop the microphone if we used it for recording
46.             if (recordMicrophone)
47.             {
48.                 Microphone.End(null);
49.                 microphoneSource.Stop();
50.                 audioRecorder.Dispose();
51.             }
52.             // Stop the recording
53.             videoRecorder.Dispose();
54.             NatCorder.StopRecording();
55.         }
56.  
57.         void OnShare(string path)
58.         {
59.             if (OnVideoRecorded != null)
60.                 OnVideoRecorded(path);
61.         }

 

The code looks good. Can you get the full logs from any of these devices? I'll need to see the logs to know what exactly is happening.

 

HI. I am trying to record microphone audio along with the game music. The in game music is recorded without any issue. When I try to record through the microphone, I play the audio clip so that the output can go the AudioListener. So When I speak into the microphone I am able to hear the microphone audio which I don't want. I just want to record the audio through the Microphone without playing it in the game
HI. I am trying to record microphone audio along with the game music. The in game music is recorded without any issue. When I try to record through the microphone, I play the audio clip so that the output can go the AudioListener. So When I speak into the microphone I am able to hear the microphone audio which I don't want. I just want to record the audio through the Microphone without playing it in the game

 

Hii ... i still have the same isssue ..... Everything is working fine in pixel 2 ....but the audio is not coming when i start doing the recording . Kindly help me out .Thanks in AdVANCE

 

This is expected to happen. Unity's microphone API doesn't do adaptive echo cancellation (AEC), so you will get feedback in the audio output (Larsen effect). Our microphone API, NatMic, is not affected by this issue because it does not send the microphone input to the speakers, hence avoiding the feedback loop.

 

Can you attach the full logs from logcat in a .txt file? Thanks.

 

Code (CSharp):

1. var microphoneFormat = Format.Default;
2.         NatMic.StartRecording(microphoneFormat, OnSampleBuffer);
3.         // Start recording
4.         recordingClock = new RealtimeClock();
5.         var audioFormat = new AudioFormat(microphoneFormat.sampleRate, microphoneFormat.channelCount);
6.         NatCorder.StartRecording(Container.MP4, VideoFormat.Screen, audioFormat, OnRecording);

If i run this code for video recording... neither the video is recorded nor the audio comes ... And if i use this code

Code (CSharp):

1. var audioFormat = AudioFormat.None

Then the video is recored but the audio doesnt comes .

And we are using google pixel 2 (OS 9)... and no permission popup comes for these.

 

Are you committing NatMic's audio sample buffers to NatCorder for encoding? Can you share your full code?

 

The same code is working on Samsung Galaxy S6, but video is not recording on Pixel unless i set audioSource to none.
Not possible to share whole code.
i am sharing my script. Please have a look. You can also test your example scene "ReplayCam", Its also working same.

 

Samsung Galaxy S6 have OS 7 & Everything is working fine on S6. My IOS version is also working great. Only the ARCore part (which is on Pixel) have an audio issue.

 

I see. Can you upload the full logs from logcat in a .txt file?

 

Apparently when using ARCore, the microphone permissions dialog is not shown. You have to explicitly request it.

 

I gave permission manually but it's not working.
Right now I am unable to share logcat. Is there other way to find out the problem ?

 

I'll need to see the logs to know what's going on; there is no other way.

 

Hey !!

Actually I am using the updated version of natcorder pluginn.... but I got stuck in the mid .... All the things are working fine , but when I record the video and immediately after stopping the video ... I want the recorded video to be played In a cropped format ... Earlier it was working fine ... the video was playing in cropped format .. but after updating the natcorder plugin ... the video is not been played in the cropped format ( Now the recorded video is been playing on full screen .which I don't need .. I want it to it to be played in cropped manner .

So Kindly help me out to overcome this issue .
Thanks in Advance

 

I will share my logcat file tomorrow, Today I don't have my DataCable with me.
Thanks for your help.

 

In previous versions of NatCorder, we had a shader that performed aspect fitting on the image, which meant that black bars will be placed on the correct sides of the video so that there will be no stretching in the video. We removed this from NatCorder because we are moving it to a separate package. I am currently reimplementing it in the NatCorder-Extensions project.

 

Okay! Thanks for the support .Really appreciated your quick response.

 

Hello, i have problem.My video are flipped

this CODE

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.UI;
5. using NatCorderU.Core;
6.  
7. public class Recorder : MonoBehaviour
8. {
9.     private WebCamTexture _cameraTexture;
10.     private bool _isRecording;
11.  
12.     public Text RecordTextfield;
13.  
14.  
15.     public void Record()
16.     {
17.         if (_isRecording)
18.         {
19.             StopRecord();
20.         }
21.         else
22.         {
23.             StartRecord();
24.         }
25.     }
26.  
27.     private void StartRecord()
28.     {
29.         //_cameraTexture = new WebCamTexture();
30.         //_cameraTexture.Play();
31.         Configuration configuration = new Configuration(Screen.width,Screen.height);
32.         NatCorder.StartRecording(configuration, OnVideo);
33.         _isRecording = true;
34.     }
35.  
36.     private void OnVideo(string path)
37.     {
38.         Debug.Log(path);
39.     }
40.  
41.     private void StopRecord()
42.     {
43.         NatCorder.StopRecording();
44.         _isRecording = false;
45.     }
46.  
47.     void Update()
48.     {
49.         if (NatCorder.IsRecording)
50.         {
51.             // Acquire an encoder frame from NatCorder
52.             var frame = NatCorder.AcquireFrame();
53.             // Blit the current camera preview frame to the encoder frame
54.             Graphics.Blit(_cameraTexture,frame);
55.             // Commit the frame to NatCorder for encoding
56.             NatCorder.CommitFrame(frame);
57.         }
58.  
59.         RecordTextfield.text = _isRecording ? "Stop" : "Record";
60.     }
61. }
62.  

 

Are you recording on Windows? If so, is your app using DirectX 9, 10, 11, or 12?

 

Windows, Direct3D11

 

Ah I just realized that you are using an older version of NatCorder. Please upgrade to NatCorder 1.3f2 from the Asset Store. Delete your current version of NatCorder from your project before importing the updated version.

 

and i have one more problem.In my windows build i doesn`t have a path and video

Code (CSharp):

1.   private void StartRecord()
2.     {
3.         //_cameraTexture = new WebCamTexture(Screen.width,Screen.height);
4.         //_cameraTexture.Play();
5.         //Configuration configuration = new Configuration(Screen.width,Screen.height);
6.         NatCorder.StartRecording(Configuration.Screen, OnVideo);
7.         _isRecording = true;
8.     }
9.  
10.     private void OnVideo(string path)
11.     {
12.         Debug.Log("video save at " + path);
13.     }
14.  
15.     private void StopRecord()
16.     {
17.         NatCorder.StopRecording();
18.         _isRecording = false;
19.     }

 

App Crashes Sometimes on calling StopRecording with the following error
OPENGL NATIVE PLUG-IN ERROR: GL_OUT_OF_MEMORY: Not enough memory left to execute command

 

I mentioned this right before you posted this response, but please upgrade to NatCorder 1.3f2.

 

What device does this happen on? And what is your recording resolution?

 

Hi there,


What is the correct way to record both game audio and microphone?

Using latest NatCorder and NatMic.

If I try to record game audio with microphone both sources are distorted when played back (most probably because the sample rates are not the same)

Recording from only one source works totally fine.

Code (csharp):

1.  
2. public void StartRecording()
3.     {
4.         recordingTextRoot.gameObject.SetActive(true);
5.  
6.         recordingClock = new RealtimeClock();
7.  
8.         // Start the microphone with NatMic
9.         var microphoneFormat = Format.Default;
10.         NatMic.StartRecording(microphoneFormat, OnSampleBuffer);
11.         // Start recording with NatCorder
12.         recordingClock = new RealtimeClock();
13.         var audioFormat = new AudioFormat(microphoneFormat.sampleRate, microphoneFormat.channelCount);
14.  
15.         NatCorder.StartRecording(Container.MP4, VideoFormat.Screen, audioFormat, OnReplay);
16.         sceneAudioRecorder = AudioRecorder.Create(sceneListener, recordingClock);
17.         videoRecorder = CameraRecorder.Create(Camera.main, recordingClock);
18.     }
19.  
20.     // Invoked by NatMic on new microphone events
21.     void OnSampleBuffer(AudioEvent audioEvent, float[] sampleBuffer, long timestamp, Format format)
22.     {
23.         // Send sample buffers directly to NatCorder for recording
24.         if (audioEvent == AudioEvent.OnSampleBuffer && NatCorder.IsRecording)
25.             NatCorder.CommitSamples(sampleBuffer, recordingClock.CurrentTimestamp);
26.     }
27.  
28.     public void StopRecording()
29.     {
30.         // Stop the recording
31.         NatMic.StopRecording();
32.         NatCorder.StopRecording();
33.         videoRecorder.Dispose();
34.         sceneAudioRecorder.Dispose();
35.  
36.     }
37.  

 

A few things need to change. The first is that you should only commit audio from one place. You are committing audio from both NatMic's delegate and from the AudioRecorder you are creating. This results in garbage audio. Instead, you should go through NatMic. NatMic has an overload for StartRecording that takes in an AudioListener. NatMic will automatically mix the audio from the microphone and the audio listener and provide the mixed audio in sample buffer callback.

The second thing is that you want to change the NatMic format from `Format.Default` to `Format.DefaultForMixing`. This will cause NatMic to match the microphone sample rate and channel count to that of Unity's audio engine.

With these changes, you will not need to use an AudioRecorder at all.

 

Thank you for the reply.

Another main issue is that when creating a video file using NatMic and saving it to the camera roll using nat share
The video cannot be played back on the device (it is ok on the pc) so we had to revert to the old microphone recording system.

Another thing, I don't know how much control you have over this but you should optimize the memory usage when recording. trying to record the screen with original sizes will cause most of our devices to crash due to out of memory exceptions, my quick fix for the moment is to record at half resolution.