NatCorder - Video Recording API

Hey there. The upgrade to Android 24 is for parity with the other NatSuite Framework components, specifically NatML (upcoming machine learning library) and NatDevice. I believe NatCorder can still run on API level 21+, it just requires complicated build instructions. You have to export the Gradle project with API level 24 from Unity, then in Android Studio, revert the minimum to API level 22 and add a library override for NatCorder (api.natsuite.natcorder).

 

An issue has been opened for this, and I'll be looking into it today or tomorrow.

 

Thanks Lanre! We'll try this out. We actually integrate Unity into a native project vial UAAL. So this might just work for us. Will report back

 

Sounds good, keep me posted.

 

Hi
can I using unity as a library with NatCorder?

 

Hi,

We're getting the following error and wondered if you had any pointers as to how to narrow down the cause of this issue.... The build works on most machines but just not on some... Should we put in a proper support request? (via the email address on github?)

Code (CSharp):

1. DllNotFoundException: NatCorder
2. NatCorder.MP4Recorder..ctor (System.Int32 videoWidth, System.Int32 videoHeight, System.Int32 videoFramerate, System.Int32 audioSampleRate, System.Int32 audioChannelCount, System.Action`1[T] recordingCallback, System.String recordingDirectory, System.String recordingFilename, System.Int32 videoBitrate, System.Int32 videoKeyframeInterval) (at <b718adeea08544b9955e811addc8646e>:0)
3. Inovus.InoScreenRecorder.StartRecording () (at <b718adeea08544b9955e811addc8646e>:0)
4. Inovus.InoScene.TrackingStartSession () (at <b718adeea08544b9955e811addc8646e>:0)
5. System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <437ba245d8404784b9fbab9b439ac908>:0)
6. Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
7. System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <437ba245d8404784b9fbab9b439ac908>:0)
8.  

 

Hm I've never tried this, but I expect it to work. There shouldn't be any difference in how Unity libraries are loaded.

 

It looks like you're using an outdated version of NatCorder. Upgrade to 1.8 on the Asset Store (make sure to delete NatCorder in your project before pulling the new version) and try again. What platform are you running on?

 

How to save the recorded video from particular album(path).Where to change it.

 

You can't set where NatCorder saves the recording, because it always saves to the app's private documents directory. If you want to move it somewhere else, you can use the functions in System.IO. If you'd like to copy the recording to the camera roll, you can use NatShare.

 

We're running Unity 2019.3.15f1 on PC and macOS. Is there anywhere you can point me towards a release note or readme or anything that shows a fix for this issue as the only think I could see was that a similar error occurred on OSX but that was from a really really long time ago. Thanks

 

Here's the changelog. I need to know the full specifications of the system where you saw this error. There was a similar error on Windows that was fixed sometime ago, but wasn't mentioned in the changelog. You'll want to upgrade to the latest version, and on Windows, make sure you install the latest Visual C++ Redistributable.

 

Thanks

 

Hi. I have noticed strange issue.
I have video player, with video(with sound)
when I start recording before playing video, video will play normally during recording.
But when i start recording when video is playing, video got paused(or freeze)

the issue doesn't exist when video has no audio.
Any idea what can be the reason?

 

I'm not sure why this happens. Have you checked the logs for any clues? What device and OS version does this happen on?

 

I'm having a similar issue.

Recorded videos with audio are as if in slow-motion, but if I mute the audio in terms of setting samples/channels to 0, the video is smooth.

 

Sounds like you found the cause of the issue: the app was set to use Mono audio instead of Stereo.

 

Just FYI, we use Unity as a Library with Natcorder and it works for us.

 

Hi Lanre, I was using the
NatCorder Performance Extensions for Android.

I had a quick question on:

Code (CSharp):

1. GLESTextureInput(...);

I have the recording part being generated, and I was wondering if when I choose to stop recording,

should I Dispose() the

GLESTextureInput?

Do I need to Dispose() anything when stopping recording?

Thank you for a very cool asset.

 

Hey there, glad to see you're using NCPX. If you attach the `GLESTextureInput` to a `CameraInput`, then you don't need to dispose. The `CameraInput` disposes its texture input when it gets disposed.

 

Hi, maybe this question was already answered but cannot find it anywhere.

My project is using NGUI, ARFoundation and URP.

Using Natcorder to record the video works fine but the video has the UI (Which I don't want).

ARCamera is Base, NGUI Camera is Overlay, is there a way to record the video of the Base Camera without the Overlayed UI?
Thanks!

 

I'm not familiar with NGUI at all. If you are recording with a CameraInput, have you tried excluding the NGUI camera when you create the camera input?

 

NGUI just creates an extra camera with 2D textures, I set that camera as overlay while the main Camera as Base with the NGUI Camera as overlay.

What do you mean by exclude NGUI? Like disabled it? I did a solution like this but then because the UI disappears, users can't play the game while recording

PS: I use the Camera Input only on the Camera.main
The UI Layer is disabled on the Main Camera, so only the overlaid UI is shown

 

So if you don't have to explicitly record the NGUI camera, then that means there must be some things going on with NGUI that allows it to be rendered on screen. I recommend reaching out to NGUI support on this issue; it has more to do with how their rendering system is architected.

 

Getting a very strange crash on IOS (iPhoneX).
When I record video using an AudioInput from the scene's AudioListener or a specific AudioSource I get a system exception from the recorder saying "recorder failed to finish writing" which is thrown from a concurrent dictionary.
I would also like to point out that without the AudioInput the video is created and saved correctly.

Unity: 2020.1
NatCorder: 1.8.0
Device: iPhoneX

StartRecording:

Code (CSharp):

1.         private void PerformRecordingInitialization()
2.         {
3.             var videoHeight = (int) (RecordingVideoWidth/Camera.main.aspect);
4.             videoHeight = videoHeight >> 1 << 1; // Ensure divisible by 2
5.  
6.             var resolution = new Vector2(RecordingVideoWidth, videoHeight);
7.             _clock = new RealtimeClock();
8.             _recorder = new MP4Recorder(
9.                 (int) resolution.x,
10.                 (int) resolution.y,
11.                 RecordingFramerate,
12.                 RecordAudioSource ? AudioSettings.outputSampleRate : 0,
13.                 RecordAudioSource ? (int) AudioSettings.speakerMode : 0,
14.                 VideoBitRate,
15.                 KeyframeInterval,
16.                 AudioBitRate);
17.  
18.             _cameraInput = new CameraInput(_recorder, _clock, Camera.main) {frameSkip = FrameSkip};
19.             if (Application.platform == RuntimePlatform.Android)
20.                 _cameraInput.textureInput = GenerateTextureInput();
21.  
22.             if (RecordAudioSource)
23.             {
24.                 var sourceAudio = ContentNode.GetComponentInChildren<AudioSource>(true);
25.                 _audioInput = new AudioInput(_recorder, _clock, sourceAudio);
26.             }
27.             else _audioInput = null;
28.  
29.             IsRecording = true;
30.         }

StopRecording:

Code (CSharp):

1.         private async void ShutdownRecording()
2.         {
3.             _audioInput?.Dispose();
4.             _cameraInput?.Dispose();
5.             OutputPath = await _recorder.FinishWriting();
6. #if !UNITY_EDITOR
7.             this.LogMessage($"SavingCameraRollPayload: {OutputPath}");
8.             var payload = new SavePayload();
9.             payload.AddMedia(OutputPath);
10.  
11.             var saved = await payload.Commit();
12.             this.LogMessage($"PayrollSaved: {saved}");
13. #endif
14.         }

 

A hard crash is typically unrelated to the recorder failing to finish writing. For the former, this typically happens if the recorder is used after `FinishWriting` is called. For the latter, it typically happens if your timestamps are not strictly monotonic. How long is a typical recording in your app? If it's long enough, C#'s System.Diagnostics.Stopwatch (which is what the RealtimeClock uses) seems to drift, although this is rare.

 

Typically around 10-15 seconds of recording. I've actually just managed to record correctly. Before I was recording with an audio-bitrate of around 16000 and the default sample-rate from the AudioSettings (which was set to 24000). Increasing my audio-bitrate to 96000 seemed to have solved it. Any idea why?

 

I don't recommend setting the audio bitrate (or video bitrate) if you're not familiar with the standards used in media encoding. a 16kbps bitrate sounds incredibly low. Common values are 64kbps or 96kbps.

 

The annoying part is the reaction delta between IOS and Android devices; but then again write once, deploy everywhere is good for a laugh most days

Thanks for the quick replies by the way. Really awesome.

Is there a way to throw out a better exception for this case? It really seems like a strange race condition.

 

I don't think there is. On iOS, the encoder only provides a vague status code if something goes wrong. Let me know if you run into any other crashes after setting the audio bitrate to 64kbps or 96kbps. If it still happens, then we can look into it more carefully.

 

Hi I am having trouble signing the bundle for deployment to the MacAppStore.
I am getting in XCode a CFBundleIdentifierCollision error.

CFBundleIdentifier Collision. The Info.plist CFBundleIdentifier value 'api.natsuite.NatCorder' of 'XXXXXXXX.app/Contents/PlugIns/NatCorder.bundle' is already in use by another application.

I had this same problem with another plugin in the same project and was able to solve it by adding another suffix to the bundle identifier in the bundles info.plist ( eg. api.natsuite.NatCorder.123456787654 ). This worked for the other plugin, but doesnt for the NatCorder plugin.

Any idea how I can overcome this issue?

I also tried removing the _CodeSignature folder from the xcode project but it also didn't seem to make any difference.

Using v1.8
Unity 2021

In addition, if I change the bundle identifier in the bundles info.plist to ( for example ) com.xxxxxxx.APPNAME.NatCorder
I get the error message:

Invalid Code Signature Identifier. The identifier "api.natsuite.NatCorder" in your code signature for "NatCorder" must match its Bundle Identifier "com.xxxxxxx.APPNAME.NatCorder"

Also tried reinstalling NatCorder, as suggested to Arjun-Gupte, but to no avail.

 

Hi, I am using natcode with a plugin AVPRO.
Seems that on IOS sometimes (after 2 to 6 recordings) when recording i get this error:

unity EXC_BAD_ACCESS (code=1 address=0x110)

It happens when the code execute this line:

var path = await recorder.FinishWriting();

any solution?

Please help me....

 

Hey there, it sounds like you might have multiple NatCorder bundles in your project. Here's what to try:

1. In your project, search for and delete every folder named NatCorder or NatSuite. Make sure that the API is completely removed from your project.
2. Reimport NatCorder into your project from the Asset Store.
3. Build a clean Xcode project from Unity (Replace instead of Append).

Let me know if the issue persists.

 

Can you share your full recording code? This typically happens if you try to call any function on a recorder after `FinishWriting` has been called.

 

Yes I tried that before..
I deleted the entire NatSuite folder , checked for any other files, reimported built new clean
Issue persists

 

Can you check the second answer in this SO question? Perhaps Xcode is incorrectly copying the bundle's Info.plist. You can also try removing the NatCorder bundle from your Xcode project (remove reference, don't move to trash), then add it back and see if that works.

 

Actually I just resolved it. You were right there was script i had modified earlier that referred to an earlier version of a class. It worked somehow but caused the signing issue.
All fixed.

Actually the reason this problem exists is primarily because due to failures in the design of the PackageManager, reimported updates are reverting to the 2018 version ( in my case)

It seems to unpredictably install prior versions and blend them together

 

Ah I see, that's good to know. UPM is promising but still feels very incomplete.

 

Actually this is still a mess though I am fairly certain that at one point it DID pass validation.
I have now completely removed the NatSuite folder and then installed the NatCorder 2021 asset directly from the Asset store cache in the Library folder.
However I am now back to CFBundleIdentifier collision.


And again I have completely removed the folder.
Reinstalled from Asset store ( confirming that this time it is the 2021 version)
Validation fails again with CFBundleIdentifier collision

So frustrating.

 

Tried all these things . They all fail to resolve it.

try removing the NatCorder bundle from your Xcode project (remove reference, don't move to trash), then add it back ( this did work for a seperate plugin (AVProVideo) but would not work with NatCorder)

 

Looks like the issue was coming from the audio engine after all but caused by other factors after we 'paused'

 

Sure

Code (CSharp):

1.  
2.  
3.  public class CameraRecorderReplay1 : MonoBehaviour
4.     {
5.         [Header(@"Recording")]
6.         private int videoWidth = 720;
7.         private int videoHeight = 1280;
8.         public bool recordMicrophone = true;
9.         public Camera CameraRecorded;
10.         private IMediaRecorder recorder;
11.         private CameraInput cameraInput;
12.         private AudioInput audioInput;
13.         private AudioSource microphoneSource;
14.         public static CameraRecorderReplay1 instance;
15.         public AudioMixer audioMixer;
16.         private void Awake()
17.         {
18.             videoWidth = Screen.width;
19.             videoHeight = Screen.height;
20.         }
21.         public void ExternalStartRecording()
22.         {
23.             StartCoroutine(StartRecordingCheck());
24.         }
25.         private IEnumerator Start()
26.         {
27.        
28.                 instance = this;
29.  
30.             yield return new WaitForEndOfFrame();
31.         }
32.      
33.         private IEnumerator InitMicrophone()
34.         {
35.             microphoneSource = gameObject.AddComponent<AudioSource>();
36.             microphoneSource.mute =
37.             microphoneSource.loop = true;
38.             if (Microphone.devices.Length > 0)
39.             {
40.                 microphoneSource.clip = Microphone.Start(Microphone.devices[0], true, 180, AudioSettings.outputSampleRate);
41.                 yield return new WaitUntil(() => Microphone.GetPosition(null) > 0);
42.                 microphoneSource.Play();
43.             }
44.             else
45.             {
46.                 ToastMessage.Instance.SendSpecificMessage("NO MICHROPONE DETECTED");
47.             }
48.          
49.         }
50.         public IEnumerator StartRecordingCheck()
51.         {
52.             Debug.Log("INIZIO");
53.                 if (microphoneSource == null)
54.                 {
55.                     StartCoroutine(InitMicrophone());
56.                     yield return new WaitUntil(() => microphoneSource != null);
57.                 }
58.             StartRecording();
59.         }
60.         public void StartRecording()
61.         {
62.             // Start recording
63.             var frameRate = 30;
64.             var sampleRate = recordMicrophone ? AudioSettings.outputSampleRate : 0;
65.             var channelCount = recordMicrophone ? (int)AudioSettings.speakerMode : 0;
66.                 RealtimeClock clock = new RealtimeClock();
67.                 recorder = new MP4Recorder(videoWidth, videoHeight, frameRate, sampleRate, channelCount);
68.             // Create recording inputs
69.             cameraInput = new CameraInput(recorder, clock, CameraRecorded);
70.      
71.                 audioInput = recordMicrophone ? new AudioInput(recorder, clock, microphoneSource, true) : null;
72.        
73.             // Unmute microphone
74.             if (microphoneSource != null)
75.                 microphoneSource.mute = audioInput == null;
76.          
77.         }
78.         public async void StopRecording(UnityAction<string> callback)
79.         {
80.                 // Mute microphone
81.              
82.                     if (microphoneSource != null)
83.                         microphoneSource.mute = true;
84.                 // Stop recording
85.                 audioInput?.Dispose();
86.                 cameraInput?.Dispose();
87.                     Debug.Log("still ok");
88.                 var path = await recorder.FinishWriting();
89.             Debug.Log("never go here when there is the error");
90.             // Playback recording
91.             Debug.Log($"Saved recording to: {path}");
92.                 StartCoroutine(DeleteMic());
93.                 callback(path);
94.          
95.         }
96.         public IEnumerator DeleteMic()
97.         {
98.                     yield return new WaitForSeconds(0.2f);
99.             microphoneSource.Stop();
100.             yield return new WaitForSeconds(0.1f);
101.             Destroy(microphoneSource);
102.         }
103.     }
104.  
105.  
106.  

Can you help me with this?

 

Hi, I am having an issue with the cast area of the buttons on my UI when recording a video with non 16:9 aspect ratio. For example, with 18:9, 20:9 aspect ratio, like some mobile devices have.

The basic setup to reproduce the issue is to use a main camera (perspective depth -1) and a UI camera (orthographic depth 1, clear flags: don't clear).

The UI canvas with the buttons is set to screen space - camera, with the UI camera on it. The canvas scaler is set to scale with screen size, with the base resolution set as 1280x720, and the Match does not seem to have any influence (it is currently used on 0.5)

Put both cameras on the CameraInput and use a MP4Recorder. When not recording, the cast area of all the components in the UI works perfectly. When recording, it seems to be all messed.

Even when using only the UI camera on the CameraInput the problem persists. By removing it, the problem is gone, but the UI is not recorded obviously.

I am sending the code I use, with the setup functions for video resolution I have tested. Both of the functions lead to the same problem. It can be tested on editor, using both NatCorder and NatShare packages.

Is it something that is messing with the ortographic camera when used on the CameraInput, or am I doing something wrong here?

Any help is appreciated.

Code (CSharp):

1. using NatSuite.Recorders;
2. using NatSuite.Recorders.Clocks;
3. using NatSuite.Recorders.Inputs;
4. using NatSuite.Sharing;
5. using UnityEngine;
6.  
7. public class VideoManager : MonoBehaviour
8. {
9.     public int refVideoMax = 1280;
10.     public int refVideoMin = 720;
11.     //[HideInInspector]
12.     public int finalWidth, finalHeight;
13.     public Camera[] cameras;
14.  
15.     private IMediaRecorder recorder;
16.     private CameraInput cameraInput;
17.  
18.     void Start()
19.     {
20.         SetCurrentResolution();
21.         //SetCurrentResolutionAlt();
22.     }
23.  
24.     private void SetCurrentResolution()
25.     {
26.         if (Screen.width > Screen.height)
27.         {
28.             finalWidth = refVideoMax;
29.             finalHeight = (int)((float)Screen.height / Screen.width * refVideoMax);
30.             finalHeight = finalHeight >> 1 << 1;
31.         }
32.         else if (Screen.height > Screen.width)
33.         {
34.             finalHeight = refVideoMax;
35.             finalWidth = (int)((float)Screen.width / Screen.height * refVideoMax);
36.             finalWidth = finalWidth >> 1 << 1;
37.         }
38.     }
39.  
40.     private void SetCurrentResolutionAlt()
41.     {
42.         var videoHeight = (int)(refVideoMax / cameras[0].aspect);
43.         videoHeight = videoHeight >> 1 << 1; // Ensure divisible by 2
44.         finalWidth = refVideoMin;
45.         finalHeight = videoHeight;
46.     }
47.  
48.     public void StartRecording()
49.     {
50.         // Start recording
51.         var frameRate = 30;
52.         var clock = new RealtimeClock();
53.         recorder = new MP4Recorder(finalWidth, finalHeight, frameRate, 0, 0);
54.         // Create recording input
55.         cameraInput = new CameraInput(recorder, clock, cameras);
56.     }
57.  
58.     public async void StopRecording()
59.     {
60.         // Stop recording
61.         cameraInput.Dispose();
62.         var path = await recorder.FinishWriting();
63.         // Save on gallery
64.         var payload = new SavePayload("TEST ALBUM");
65.         payload.AddMedia(path);
66.         await payload.Commit();
67. #if !UNITY_EDITOR
68.             File.Delete(path);
69. #endif
70.     }
71.  
72. }
73.  

 

I'll try to reproduce this with Xcode and follow up on Discord.

 

Your code looks good. Can you reproduce a crash and share a screenshot of the Xcode window when the crash happens? I'd like to see the call stack when the crash happens.

 

Hey there. I'm not entirely sure what you mean by "cast area". Are you seeing some weird scaling of your UI elements in the recorded video? If so, it sounds like this bug I discovered with Unity's CanvasScaler.

 

I took this photo, it is enough?

 

Thanks for sharing these pictures. They don't show the exact cause of the crash, so can you go to the breakpoints tab in the left pane, then add an exception breakpoint that is triggered on all exceptions. Then try to reproduce the crahs; it should have more information.

Also, when you build from Unity, in Player Settings, make sure that IL2CPP builds in Debug mode, not Release. Then in Build Settings, make sure you run in Xcode as Debug, not Release.

 

All UI elements are drawed normally, but the region where you click/touch is messed up, does not match the visual representation. This is just while recording, when stoped, all seems to work properly.

 

Do you have an illustration of what's going on, just so I understand fully? Is your project using any AR framework, or something else that controls rendering?