NatCorder - Video Recording API

Thanks.
Weird that it works flawlessly on Android. So pretty much I should stop recording and save the video on lose focus?

 

Some Android devices might not be as strict in enforcing this constraint, but you can probably get it to break by going to a different app that uses the encoder (like recording a video in Snapchat). In any case, you definitely want to stop recording before the app loses focus.

 

I come back with the same news, in order to test I used your example scene HotMic, here is the log

Spoiler: Log

NatSuite.Examples.<Start>d__2:MoveNext() (at Assets\Examples\HotMic\HotMic.cs:31)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(<Start>d__2&)
NatSuite.Examples.HotMic:Start()
(Filename: Assets/Examples/HotMic/HotMic.cs Line: 31)
Starting recording with format: 44100Hz with channel count 1
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
NatSuite.Examples.HotMic:StartRecording() (at Assets\Examples\HotMic\HotMic.cs:37)
UnityEngine.Events.InvokableCall:Invoke()
UnityEngine.Events.UnityEvent:Invoke()
NatSuite.Examples.Components.<Countdown>d__10:MoveNext() (at Assets\Examples\HotMic\MicrophoneButton.cs:43)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
(Filename: Assets/Examples/HotMic/HotMic.cs Line: 37)
Recorded audio to path: F:\Unity\Projects\zettlo\recording_2020_07_21_17_16_16_164.wav
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
NatSuite.Examples.<StopRecording>d__4:MoveNext() (at Assets\Examples\HotMic\HotMic.cs:49)
System.Runtime.CompilerServices.AsyncVoidMethodBuilder:Start(<StopRecording>d__4&)
NatSuite.Examples.HotMic:StopRecording()
UnityEngine.Events.InvokableCall:Invoke()
UnityEngine.Events.UnityEvent:Invoke()
NatSuite.Examples.Components.<Countdown>d__10:MoveNext() (at Assets\Examples\HotMic\MicrophoneButton.cs:54)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
(Filename: Assets/Examples/HotMic/HotMic.cs Line: 49)

As for the code, I am using your examples and your github examples, none of them work for me on WIndows or Android

 

Thanx, update fix it. buuut)) now on one my test phone i get 1 recorded video file mp4 format, and this is how it should be, and on my 2-nd test phone i get 2 recorded video files. mp4 (saved in gallery) and 3gp.(saved in Video folder). What i doing wrong? Thanx

 

I'm confused. Are you saying that the recorded WAV file is completely silent on Windows and Android? Can you be more descriptive with the issue you are facing, specifically how NatCorder factors into any of this (the logs have no mention of NatCorder). Also when sharing logs, specifically on Android, always share the full (unfiltered) logs from app start to close in a .txt attachment.

 

If you are using NatShare, then it seems that Android (incorrectly) stores any saved videos to the camera roll as a .3gp file. This should not cause any problems whatsoever.

 

hi, there is a problem with the screen recording distribution function not working. NatCorder + NatShare. I can't understand what the problem is.
unity 2019.3
Test apk samsung s8+ android 9

 

I'll need more descriptive information on this. I have no clue what you could be referring to. If you're facing issues on Android, always provide the full logs from logcat (in a .txt attachment) along with your detailed description of what the issue is. Also, your code needs to be updated to use the latest version of NatShare.

 

[Update - seems working now, made some changes by looking at examples]

Hi @Lanre. I have updated natcorder version. but now have some other issues.
application some times freezing in Finishwriting task, then crashing.

Code (CSharp):

1. public void StopRecording()
2.     {
3.             audioInput.Dispose();
4.             cameraInput.Dispose();
5.             try
6.             {
7.                 string path = videoRecorder.FinishWriting().Result;
8.                 .....
9.              }
10.              catch(Exception e){}

Catch block not working as well. any idea? i can provide some other information too, just ask.

 

also one more thing with new version of Natcorder.
When im using vuforia's camera, recorded video is smooth, but when im using ArFoundation camera, then recorded video is not smooth, its like low fps. what can affect to it?

 

I'm not sure on the rendering differences between Vuforia and ARFoundation. I recommend profiling your app to see.

 

Thanks for seeing to this!

 

I can use 2 camera for recording, can i make recording from texture and 1 camera at same time?

 

This isn't a trivial thing to do, because you will have to write custom code to blend the camera view with the texture. This is out of scope for NatCorder.

 

@Lanre thanks for replies.
I have one more problem. Android work fine, but ios not.
When im starting recording, app fully freezing. no errors/warning in console
It just prints last few logs: Natcorder starts recording (something like this and parameters. )
and Natmic started recording(something like this and parameters)

I have no clue why it is happening.

 

What version of Unity are you using?

 

Hello,

I've purchased the plugin and it looks great! Is there is a way to skip UI during recording the video or making screenshot? Is there any code example?

Btw I use AR Camera from Vuforia AR SDK.

Thanks.

 

You can either set the UI canvas to render with the Overlay mode, or create a secondary game camera through which the UI canvas will render (then record only the primary camera).

 

2018.4.23f1
I wrote Debug.Log on every line in StartRecording function.
Console writes them all, so wherever it freezes, it is outside of this function.

[UPDATE] @Lanre news about the situation. When im starting recording screen is freezing.
When im minimizing app(i stoping recording that moment), then opening again. i can see recorded video preview. So recording is working. but it freezing app frame. )

 

Thanks Martin, your helpfulness is amazing!

 

This is a known issue in Unity 2018, coming from Unity. Unity 2019.2 fixes the issue.

 

hmm. but why it is works on Android.
So there is no way to fix it in Unity 2018?

 

The bug solely affects iOS. And unfortunately there isn't any workaround. I don't think a Unity issue was opened for it, but Unity will freeze the screen when rendering to a RenderTexture (which is needed by CameraInput to work).

 

Hi,
I'm looking to implement video recording in an Oculus Go application we've developed. The main bottleneck here is performance. Most recording solutions I've tried so far basically ruined the framerate. Is there any demo scene/package I can give a try before purchasing NatCorder?

 

PM me your email address and I'll send you an APK through WeTransfer.

 

Hi Lanre. i dont know remeber you or not. I had problem with mp4recorder which caused a crash(line 333)
You said that i need to use new version. Im now using new version. but i got few crashes as well


Fatal Exception: java.lang.Error
FATAL EXCEPTION [MP4Recorder Video Encoding Thread] Unity version : 2018.4.23f1 Device model : HMD Global Nokia 2.1 Device fingerprint: Nokia/Dynamo2_00WW/E2M:9/PKQ1.181105.001/00WW_1_13C:user/release-keys


Caused by java.lang.IllegalStateException

api.natsuite.natcorder.MP4Recorder$6.run (MP4Recorder.java:192)

im stil using Unity 2018

 

Thanks, I sent you a message.

 

Can you share the full logs from app start to crash in a .txt attachment?

 

Is there a way to set the orientation of the recorder?
For example, I want to record video in portrait mode but output video in landscape mode.
The reason is that I don't want to change ScreenOrientation because rotating UI is slow, but I want user to be able to record video in both mode base on Input.deviceOrientation

 

I'm not sure if you mean rotating the image, or simply recording in a wide aspect. The former is not entirely trivial, and would require you writing a custom `CameraInput` script. For the latter simply create a recorder with a wide aspect (width > height) then use a `CameraInput` to record the game camera. The `CameraInput` script automatically resizes the camera view to match the recorder's resolution.

 

Hello,

I'd like to know how to maximize the option for a well recorded sound. Currently the sound looks like a mp3 from the early 90's like if it was in 96kbps.

The record code look like :

Code (CSharp):

1. GetComponentInChildren<TMPro.TextMeshProUGUI>().text = "stop recording";
2.  
3.     Debug.Log("width: " + Screen.width + " height: " + Screen.height);
4.  
5.     recorder = new MP4Recorder(Screen.width, Screen.height, 30, 48000, 2, 6_000_000, 2);
6.     cameraInput = new CameraInput(recorder, clock, Camera.main);
7.     audioInput = new AudioInput(recorder, clock, Camera.main.GetComponent<AudioListener>());

And then later :

Code (CSharp):

1. async void StopRecording () {
2.     isRecording = false;
3.  
4.     audioInput.Dispose();
5.     cameraInput.Dispose();
6.     path = await recorder.FinishWriting();
7.  
8.     ShareRecording();
9.  
10.     GetComponent<Image>().DOFade( 1f, .333f);
11.     GetComponentInChildren<TMPro.TextMeshProUGUI>().text = "record";
12.   }

We also noticed a long time between the moment we await and the display of the share. The app look a bit frozen.

Thanks for your help

 

Overwhelming chances are that the audio compression is not what you hear as a degraded sound quality. Check the audio settings of your project along with the audio clips you are playing in your scene. And 96kbps is the bitrate that the iOS camera app uses for audio (you can verify this by recording a video and inspecting the file), so it's by no means low.

FinishWriting is asynchronous, so it will take some time before the encoder finalizes the recording. This is no different from the iOS native screen recorder, which takes a brief while before providing the recorded video. The call is entirely async, so it will not block your app.

 

can you post the limk to the new version of natcorder + natmic on asset store, i cant find it

 

anybody knows how to fix this error

WebCamTexture class is used but Camera Usage Description is empty. App will not work on iOS 10+.

Microphone class is used but Microphone Usage Description is empty. App will not work on iOS 10+.
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr, Boolean&) (at /Users/builduser/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:189)

 

NatMic has been deprecated, replaced by NatDevice. You can grab the NatDevice upgrade here, and grab NatCorder here. As for the warning, specify a camera usage description in Player Settings.

 

Hi Lanre.
We need the video to be shot with an aspect ratio of 4: 3 and a screen resolution of 800x600
But when shooting, the picture is compressed (flattened).
Tell me how to fix it!

 

Are you using an AR framework? These are the prominent cases where you can't just record at any resolution you want.

 

We use AR. We do on Vuforia

 

Yes so pretty much all of the AR frameworks render the game camera by overriding the normal behaviour you would typically expect. As a result, you will always have to record at the screen's aspect ratio. Using a resolution with a different aspect will cause stretching.

 

What in the code do i need to change for the new one? I'm using NAT-CORDER + NAT+MIC both

 

I think I'm experiencing this same memory leak (I think) with the latest version of NatCorder. In my project, I've boiled it down to this:

Code (CSharp):

1.    
2. void OnAudioFilterRead(float[] sampleBuffer, int channelCount)
3. {
4.     if( videoRecorder != null )
5.     {
6.         videoRecorder.CommitSamples(sampleBuffer, recordingClock.timestamp);
7.     }
8. }
9.  

If I comment out the CommitSamples line, I don't see a memory leak when I stop and start recording. If I keep the CommitSamples line in, I see a memory leak each time I stop recording (roughly 10-15mb). I have a similar graph like bhtroung93's in iOS overtime (attached). Peaks are recording but notice the lows are slowly rising.

Any thoughts on this? Seems like something with committing sound samples to the video recorder is not getting cleaned up.

 

And to clarify, this only seems to happen if you use an AudioListener as bhtroung93 suggested. Note the code example I sent over was just my custom version of AudioInput. I also tried the built in AudioInput script and see the same issue. If I use AudioListener, I get the leak.

 

You can follow the integration example.

 

Can you share your full recording code, along with the logs from Xcode? And does this only happen when recording an `AudioListener`, or does it also happen when recording an `AudioSource`? What version of Unity are you using?

 

Audio Listener only. AudioSource seems to work fine and does not leak. The Audio Listener use case is that our experience has a number of AudioSources playing and we want to screen record all of the audio (and not just one or the mic). I've modified the ReplayCam example to show the leak. Simply takes an AudioListener in (set via Inspector) and passes it to AudioInput. This shows the leak on iOS. See attached screenshot of memory graph. Each time I stop recording you'll see a flat area, but notice the trend is going up.

Unity version 2019.4.3f1
NatCorder versions: latest

Code (CSharp):

1. /*
2. *   NatCorder
3. *   Copyright (c) 2020 Yusuf Olokoba
4. */
5.  
6. namespace NatSuite.Examples {
7.  
8.     using UnityEngine;
9.     using System.Collections;
10.     using Recorders;
11.     using Recorders.Clocks;
12.     using Recorders.Inputs;
13.    
14.     public class ReplayCam : MonoBehaviour {
15.  
16.         [Header(@"Recording")]
17.         public int videoWidth = 1280;
18.         public int videoHeight = 720;
19.         public AudioListener audioListener;
20.  
21.         private IMediaRecorder recorder;
22.         private CameraInput cameraInput;
23.         private AudioInput audioInput;
24.  
25.         private void OnDestroy () {
26.             Microphone.End(null);
27.         }
28.  
29.         public void StartRecording () {
30.             // Start recording
31.             var frameRate = 30;
32.             var sampleRate = AudioSettings.outputSampleRate;
33.             var channelCount = (int)AudioSettings.speakerMode;
34.             var clock = new RealtimeClock();
35.             recorder = new MP4Recorder(videoWidth, videoHeight, frameRate, sampleRate, channelCount);
36.             // Create recording inputs
37.             cameraInput = new CameraInput(recorder, clock, Camera.main);
38.             audioInput = new AudioInput(recorder, clock, audioListener);
39.  
40.         }
41.  
42.         public async void StopRecording () {
43.  
44.             // Stop recording
45.             audioInput?.Dispose();
46.             cameraInput.Dispose();
47.             var path = await recorder.FinishWriting();
48.             // Playback recording
49.             Debug.Log($"Saved recording to: {path}");
50.             Handheld.PlayFullScreenMovie($"file://{path}");
51.  
52.         }
53.     }
54. }

 

Hi Lanre,

Is it possible to get only 1 frame of the video (the start) and save this frame to be shown?

 

Currently this is my code to record a video with audio:

Code (CSharp):

1.  public async void SetMediaDevices()
2.     {
3.         RKLog.Log("Set Devices.");
4.         MediaDeviceQuery audioDeviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
5.         MediaDeviceQuery cameraDeviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.FrontFacing);
6.  
7.         _audioDevice = (IAudioDevice)audioDeviceQuery.currentDevice;
8.         _cameraDevice = (ICameraDevice)cameraDeviceQuery.currentDevice;
9.  
10.         _cameraTexture = await _cameraDevice.StartRunning();
11.         _rawImage.texture = _cameraTexture;
12.  
13.         // Scale preview panel for portrait only.
14.         _aspectFitter.aspectRatio = (float)_cameraTexture.width / (float)_cameraTexture.height;
15.     }
16.  
17.     public void StartRecording()
18.     {
19.         int sampleRate = recordMicrophone ? _audioDevice.sampleRate : 0;
20.         int channelCount = recordMicrophone ? _audioDevice.channelCount : 0;
21.  
22.         // Start recording.
23.         _clock = new RealtimeClock();
24.         _videoRecorder = new MP4Recorder(_cameraTexture.width, _cameraTexture.height, Config.videoFPS, sampleRate, channelCount);
25.         _cameraInput = new CameraInput(_videoRecorder, _clock, Camera.main);
26.  
27.         _audioDevice.StartRunning((sampleBuffer, timestamp) => _videoRecorder.CommitSamples(sampleBuffer, _clock.timestamp));
28.     }
29.  
30.     public async void StopRecording()
31.     {
32.         _cameraInput.Dispose();
33.         _audioDevice.StopRunning();
34.         string path = await _videoRecorder.FinishWriting();
35.  
36.         // Playback recording.
37.         Debug.Log($"Saved recording to: {path}");
38.         string prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
39.         Handheld.PlayFullScreenMovie($"{prefix}{path}");
40.     }

 

Thanks for the in-depth info. I'm gonna try to reproduce it and find a fix.

 

NatCorder won't be able to handle this; and it wouldn't need to. You can simply capture a screenshot.