NatCorder - Video Recording API

I don't have code for this. Check Unity forums/answers, StackOverflow, or Google.

 

Frame processing rate has fallen below pre-set threshold

My app, dev with Unity 2018.4.11 - Vuforia 8.3.8, is crashing when I hit record button. I'm using XCode 11.1

 

Can you share the logs and a screenshot of Xcode when the app crashes?

 

Hi, Mr. Lanre.
In attachment the entire log in .txt and the screenshot. The app is freezing when we hit the share button.
Thanks for your help.
Regards

 

Hello any one facing this kind of issue in ios Unity using nat coder Recording ?

I am trying to record video with mic Audio using Nat coder Record ,
Unity version : 2018.4.8
Nat coder : 1.6.3

issue : first few time audio record very good with video but after that audio not recording properly.

i had try option with prepare ios for recoding True , force ios when recording true

 

I was getting the same "Main Thread Checker: UI API called on a background thread" error when using NatShare. Don't remember seeing that in earlier builds of my app so not sure what could have changed. Had to switch over to a different share library in the meantime

 

This is a NatShare bug, and it has been fixed in the PR I'm working on.

 

Thanks! I'll be waiting. Please let me know as soon it available.

 

Will the new version of NatCorder be up soon? The Unity Editor shutting down when the recording is stopped is driving me nuts.
At least it seems to be working great on Android, and I also was able to record nice video on iOS.

 

Ok - I'll try! Here's a vote to include within the plugin though.

 

You can always email me with your invoice number for a build.

 

Hi Lanre,

I have three major issues in my iOS build.

1. I used the same RealtimeClock for both CameraInput and AudioInput, the code looks like:

Code (CSharp):

1. _clockReplay = new RealtimeClock();
2. _recorderReplay = new MP4Recorder(1280, 720, 30, 44100, 2, SaveReplay, 3600000);
3. _cameraInputReplay = new CameraInput(_recorderReplay, _clockReplay, _cameraSelected);
4. _audioInputReplay = new AudioInput(_recorderReplay, _clockReplay, _audioListener);

It records normally in Unity Editor. But in the iOS build, the audio playback speed is way faster than the picture. Please check the two links. One is recorded in the Editor and the other one is exported from iOS.

2. The output from iOS does not follow the parameters I set, e.g., in my code, I set the audio channels to 2, but the actual recorded soundtrack will be 1. Please check the attached screenshot for the different outputs with the same code.

3. In the iOS build, if I stop recording when the RealtimeClock is set to paused(

_clockReplay.Paused = true;

), the app crashes.

My environments:
Unity: 2018.4.11f1
NatCorder: 1.6.3
iOS: 13.1.2

 

I have updated the plugin to the latest available and refactored my code to suit. It keeps crashing in the editor any time i attempt to record a video now unfortunately! This is happening on editor versions 2018.4.6f1 and 2019.2.9f1. Any help is much appreciated!

 

And for the Android build, I have tested on a Sony Z5 Premium running Android 7.1.1 and a Xiaomi 8SE with Android 8.0. Audio recorded by the Sony Z5 Premium is perfectly synchronized with the picture, but the Xiaomi 8SE has the same issue as iPhone - audio is way faster than the picture.

In an Android device, stopping the recorder when RealtimeClock is paused does not crash the app, but I got the error message as follow:

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object.
2.  
3. UnityEngine.GlobalJavaObjectRef.op_Implicit (UnityEngine.GlobalJavaObjectRef obj) (at <00000000000000000000000000000000>:0)
4. UnityEngine.AndroidJavaObject._Call (System.String methodName, System.Object[] args) (at <00000000000000000000000000000000>:0)
5. NatCorder.Internal.Readback.GLESReadback.Dispose () (at <00000000000000000000000000000000>:0)

 

You are passing in 44100 and 1 as the sample rate and channel count respectively, but there is no guarantee that this is the sample rate and channel count of the audio data you are recording (because you are recording Unity audio). Refer to Unity's AudioSettings class for the correct values to pass in.

This can't be possible. NatCorder passes the values of sample rate and channel count in verbatim.

Can you upload a crash log and a screenshot of the Xcode window when the app crashes?

 

Fix coming to the Asset Store soon.

 

Can you share your pause and stop recording code?

 

Getting black screen issue in natcorder 1.6.2 plugin in moto g6 plus

 

Referring to Unity's AudioSettings class works. Issue 1 solved! Appreciate your timely help.

After further digging, I believe Issue 2 has nothing to do with NatCorder. The video is compressed by the app that transfers files from my iOS device to PC, causing the parameters to change.

For issue 3, my new code solved the problem. It looks like:

Code (CSharp):

1.    void StartRecorder()
2.     {
3.         isRecording = true;
4.         clockReplay = new RealtimeClock();
5.         recorderReplay = new MP4Recorder(1280, 720, 30, AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode, SaveReplay, 3600000);
6.         cameraInputReplay = new CameraInput(recorderReplay, clockReplay, cameraSelected);
7.         audioInputReplay = new AudioInput(recorderReplay, clockReplay, audioListener);
8.     }
9.  
10.     void StopRecorder()
11.     {
12.         if (!clockReplay.Paused)
13.         {
14.             audioInputReplay.Dispose();
15.             cameraInputReplay.Dispose();
16.         }
17.         recorderReplay.Dispose();
18.         isRecording = false;
19.     }
20.  
21.     void PauseRecorder()
22.     {
23.         //clockReplay.Pause();    //NatCorder1.5.1
24.         clockReplay.Paused = true;
25.         audioInputReplay.Dispose();
26.         cameraInputReplay.Dispose();
27.     }
28.  
29.     void ResumeRecorder()
30.     {
31.         //clockReplay.Resume();    //NatCorder1.5.1
32.         clockReplay.Paused = false;
33.         cameraInputReplay = new CameraInput(recorderReplay, clockReplay, cameraSelected);
34.         audioInputReplay = new AudioInput(recorderReplay, clockReplay, audioListener);
35.     }
36.  

In my old code, the AudioInput and CameraInput are disposed without checking if they have been disposed in the PauseRecorder method. The old code:

Code (CSharp):

1. void StopRecorder()
2.     {
3.         audioInputReplay.Dispose();
4.         cameraInputReplay.Dispose();
5.         recorderReplay.Dispose();
6.         isRecording = false;
7.     }

But the old code won't crash the app in NatCorder 1.5.X.
Attached is the screenshot of the XCode window when the app crashes.

 

Can you upload the unfiltered logs from logcat in a .txt file? Make sure that your app is not rendering with Vulkan (Player Settings > Auto Graphics API).

 

Natcorder 1.6.3. I found that my app will crash sometimes when the recording stopped, and if I remove Vulkan from Graphis APIs list( with OpenGles2 left), the game will not crash any more. Is there any other suggestions?

 

Hi @Lanre

I record my AR scene using your ReplayCam example. I updated to the latest version and I see a couple of black frames at the end of the recorded videos. What could be the reason?

 

How can I know if MP4Recorder is available on current device?

 

NatCorder doesn't support Vulkan; only OpenGL ES 3.

 

Not sure why this would happen. What platform/device are you running on? Can you share one such video?

 

Every platform that NatCorder supports (Android, iOS, macOS, Windows, WebGL) support the MP4Recorder.

 

Hi, is ArmV7 or 7s supported ?

 

I presume you are asking about iOS. If so, then no. Only arm64 is supported.

 

Hi Lanre,

After days of banging my head with Google Chrome support about the issue, they finally allowed me to raise it on the Chromium Bugs repository. Now they seem to have found the problem with the video and said:

Fails audio decoding; at least one of the initial audio packets is corrupt. You can check this with ffmpeg by running

ffmpeg -err_detect explode -i <file> -acodec pcm_f32le -vn -f wav -y /dev/null​

And you can see my bug report for Chrome browser here - https://bugs.chromium.org/p/chromium/issues/detail?id=1016081#c3

Now that we know that the issue is with some of the initial audio packets being corrupt, can you fix this?

 

Hi Lanre, thanks for creating this awesome asset.
I do have one important question before purchasing... is it work on Android 10?

 

Hello,
I've searched through the documentation, the Medium articles, and this forum thread, and I'm a bit lost still.
My ultimate goal here is to record audio from game objects with audio sources in a scene, as well as microphone audio, and include that in a video capture.
In my quest to do this, I've tried starting small first, and attempted to record 1 video that includes audio with the Replay example. So far, I've managed to get a video with the audio from regular game objects by creating the AudioInput with the audioListener as a parameter. When using any audio sources as a parameter, I've gotten videos with no audio. Following the audio workflows article has gotten it to crash Unity when stopping recording. I'm curious to know if that article is just outdated, and there's something else I'm missing here. But through the documentation, examples, and forum thread, I haven't figured out what I need to do to include recorded microphone audio in the video capture.

According to this, it seems like I'm able to do what I describe, I'm just not figuring it out. Even a point in the right direction would help out, thanks.

 

Updated NatCorder from 1.6.3 to 1.6.4 and got the latest NatMic, everything works in editor (start and stop recording without Unity crashing), but on device (iPhone X with iOS 13.1.3) recording video freezes my entire UI. Video does get recorded tho.

Not sure yet if this is an error on my part, I'll have more time in the next day or two to investigate, just posting here for now to see if anyone else is getting the same.

 

Hi, just purchased NatRecorder 1.6.4 -

The ReplayCam Example script is completely crashing the Editor for me on StopRecording() in Unity 2019.2.9f1 on MacOS Catalina 10.15.

Unity is giving me the option to report a crash bug to them, but I'm not getting normal MacOS crash log, that I can see.


I checked that my Player Settings are set to use Metal.

Seems like this could still be the issue people were experiencing on the last page of the forum, with version 1.6.3?


Would really like to get this working...
Thanks,
Peter

 

NetCorder Video recording not work in 64bit apk in pie device it can be crash after half video capture

 

After searching around some more, I've found the example of using NatCorder + NatMic together is a maybe possibly better solution to what I'm doing, but the example provided is using a very outdated version of NatCorder. Now I'd be willing to go and get NatMic to just get this functionality done and over with, $50 is worth not having the headache of building this from scratch myself, which I don't have the time to do that anyway. But if the only documentation is obsolete and can't be used, that's more wasted money, and more wasted time on top of that trying to figure out if it still works together.
I don't doubt that these assets are good, I just can't seem to figure out how they're supposed to work. So a suggestion on where to start and how would be awesome. Thanks.

 

I got your email. I'll check this out this weekend.

 

Yup. There's nothing different about Android 10.

 

The challenge with doing this is that you'll need some way to mix audio from the game with audio from the microphone. We tried rolling out our own implementation in NatMic but it didn't work properly due to timestamp synchronization issues, so we're removing the functionality.

Make sure that the audio source does not bypass effects or listener effects. This is mentioned in the README.

Fixed in 1.6.4. Grab from the Asset Store.

The easiest thing to do is to record only the microphone. The expectation is that the microphone hears game audio, in addition to audio from the outside world, hence achieving your goal.

 

Not sure what this might be. Are you using a custom rendering setup (AR, postfx, and so on)?

 

1.6.4 fixed the crashing bug when stop recording is called. Click 'preview' in the bug report window that pops up then upload the crash logs.

 

Can you upload the full logs from logcat in a .txt?

 

Thanks - even though I just purchased yesterday, maybe I still somehow got 1.6.3, because reimporting the Asset fixed the crash on StopRecording() for me. So good to go there, and awesome to see a first recording in the example!

- However, whenever I try to Build and Run the ReplayCam example, nothing seems to be happening after the recording finishes - no file is saved. Not sure how to debug this, since it now works perfectly in the Editor. Is there an additional Privacy request I need to be making in actual builds on MacOS, in order for the onReplay function to save the file?


For anyone else who didn't immediately figure this out:


The relevant function in Replay Cam is the OnReplay callback, and here's some code I wrote to save the capture to the user's choice of location on desktop platforms.
It only opens the file in the Editor after recording, because that's what's written at the end of the original Asset function.


private void OnReplay (string path) {

string savePath = "";

if (Application.platform == RuntimePlatform.WindowsPlayer || Application.platform == RuntimePlatform.WindowsEditor || Application.platform == RuntimePlatform.OSXPlayer || Application.platform == RuntimePlatform.OSXEditor)
{
string name = string.Format("SeeMusicVideo_{0}.mp4", DateTime.Now.ToString("yyyy_MM_dd_HH_mm_ss"));

savePath = EditorUtility.SaveFilePanel(
"Save Video",
"",
name,
"mp4");

if (savePath.Length != 0)
{
FileUtil.ReplaceFile(path, savePath);
FileUtil.DeleteFileOrDirectory(path);
}
}


Debug.Log("Saved recording to: "+path);
// Playback the video
#if UNITY_EDITOR
if (savePath != "")
{
EditorUtility.OpenWithDefaultApp(savePath);
}
#elif UNITY_IOS
Handheld.PlayFullScreenMovie("file://" + path);
#elif UNITY_ANDROID
Handheld.PlayFullScreenMovie(path);
#endif
}

 

Can you share the logs in a .txt file?

 

Definitely didn't have bypass effects or listener effects on. Maybe I just did something wrong, I really have no idea without more info.

I believe I have this working almost how I am intending, thanks. I think my volume levels were just off. When creating audioInput with the audioListener parameter, it seems to work as intended, but I had to lower the game volume clip to be able to hear the microphone input. Also I was testing a lot in the editor, I should've expected some things to not function 100% as intended until it's compiled... I did run several tests on the device (iOS), but compiling for iOS every test gets tiresome.

The one thing I'm coming across now is that the volume of the game audio dips down really far when I am recording. I'm not changing the volume levels when starting the recording, I just set the volume of the clip to lower in the editor before the scene starts. So I'm confused as to how the volume is getting lowered further during recording, then it goes back to normal once the recording is done. Would you happen to know if this is a built-in thing with the Unity Microphone API?

EDIT: Answered my own question on this one.
If anyone else runs into what I describe in the above paragraph, it appears this is an iOS thing. going into Project Settings > Other Settings, and then checking the option "Force iOS Speakers when recording" will fix this.

 

I'm confused. Are you saying that it works in the editor but not on a standalone build on macOS? According to this, there should be a Player.log file.

 

Same problem here on iPhone XR ios 13.1.3 on Unity 2018.4.10 but not on 2019.2.10

Another problem I migrated from everyplay, my app was armv7 and ARM64, but with Natcorder I cannot compile for ARMV7 anymore.

When uploading to AppStoreConnect, I receive this warning:

"This bundle is invalid. The key UIRequiredDeviceCapabilities in the Info.plist may not contain values that would prevent this application from running on devices that were supported by previous versions."

That because now I'm requesting ARM64 only, can you provide a stub librairie for armv7 that will allow to compile even if it has no code behind the functions and I will disable recording for 32 bits iPhones

https://developer.apple.com/library/archive/qa/qa1623/_index.html

 

NatCorder is compiled for iOS 11, and iOS 11 dropped support for armv7, and the article suggest that targeting a more recent version of iOS can prevent this issue. So you simply have to increase your app's deployment target to iOS 11.

 

Our app is targeted at kid so we want to have the largest possible range of devices and specially old ones. The video capture is is a feature that we can turn off in code on our side if the requirements are not met. It will allow us to to support more devices, back to iOS9 ArmV7 OpenGL 2, compared to iOS11 + ARM64 + Metal, for us it's very important for our business.

 

Just want to add that I'm getting the app freeze on on 2018.4.4

 

I ended up defining my own "stub" in .mm file for iOS:

Code (CSharp):

1. #if !CGFLOAT_IS_DOUBLE
2. extern "C" {
3.  
4. void NCEncodeFrame(){}
5. void NCEncodeSamples(){}
6. void NCStartRecording(){}
7. void NCStopRecording(){}
8. void NCCreateGIFRecorder(){}
9. void NCCreateHEVCRecorder(){}
10. void NCCreateMP4Recorder(){}
11.  
12. }
13.  
14. #endif
15.  

And check for possible natcorder support in C# :

Code (CSharp):

1.  
2.     public static bool is64bits
3.     {
4.         get
5.         {
6.             return System.IntPtr.Size > 4;
7.         }
8.     }
9.  
10.     public static bool isVideoRecordingSupported
11.     {
12.         get
13.         {
14.  
15. #if UNITY_IPHONE && !UNITY_EDITOR
16.             if (SystemInfo.graphicsDeviceType != GraphicsDeviceType.Metal){
17.                 return false;
18.             }
19.  
20.             if(!is64bits){
21.                 return false;
22.             }
23.          
24.             var ver = UnityEngine.iOS.Device.systemVersion;
25.             string[] v = ver.Split('.');
26.             var major = v[0];
27.             int maji = 9;
28.             int.TryParse(major,out maji);
29.             if(maji < 11 ){
30.                 Debug.Log("Video Recording disabled iOS < 11 Detected version = " + maji);
31.                 return false;
32.             }
33. #endif
34.  
35.             return true;
36.         }
37.     }
38.  
39.