NatCorder - Video Recording API

I'm using ARFoundation, which show the front camera stream and on top of that the 3D face mesh. All looks good on iPhone and on iPad (different aspect ratio), no squeezing.

 

Hey,

I noticed that a new update was released 1.7 and I assumed updating it to the latest version will fix the recording issue, but now, I am unable to record the video on Android. Can you please help with the package for 1.6.6 or see how to sort this issue?

If you need any details on this, i would be happy to provide.

 

Do you have more information about the issue on NatCorder 1.7? I'll need specific details in order to be of any kind of assistance. As for the audio, can you share a recording file?

 

I recommend filing a bug report with Unity because this sounds like a bug that shouldn't exist. In the mean time, you could record at the screen's aspect ratio to avoid any squeezing.

 

Sure. Thanks for replying.

I tested used your example "Replay Cam" scene in Android and normally when you press the hold the record button, you will see the red outer circle filing up indicating that the recording is in progress. But in the latest patch (1.7) holding the record button does nothing. In the below link I have placed the before and after videos for our reference.

https://we.tl/t-rUlq3egD0L

Also, please note that this issue is seen only on the Android device, I am yet to check this on iPhone. But when I run the "Replay Cam" scene in the unity editor it works just fine.

For the audio sample recording file, please use the "After" video as it was recorded on my android device.

 

Thanks will do. Will also test in a clean project with only NatCorder to see if it still exists there. But what woud be the best option to record only a partial rectangle of the screen? Like only the bottom part of the screen?

 

Hi, I've updated to the new version as well but after updating my code, I got a null exception:

Code (CSharp):

1. ArgumentException: RenderTextureDesc msaaSamples must be 1, 2, 4, or 8.
2. Parameter name: desc.msaaSamples
3. UnityEngine.RenderTexture.ValidateRenderTextureDesc (UnityEngine.RenderTextureDescriptor desc) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Graphics/Texture.cs:254)
4. UnityEngine.RenderTexture.GetTemporary (UnityEngine.RenderTextureDescriptor desc) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Graphics/Texture.cs:280)
5. NatCorder.Inputs.CameraInput..ctor (NatCorder.IMediaRecorder recorder, NatCorder.Clocks.IClock clock, UnityEngine.Camera[] cameras) (at Assets/NatCorder/Plugins/Managed/Inputs/CameraInput.cs:48)
6. VideoRecoding.OnRecordVideo () (at Assets/Content/Scripts/Media/VideoRecoding.cs:83)
7. EventDelegate.Execute () (at Assets/NGUI/Scripts/Internal/EventDelegate.cs:476)
8. EventDelegate.Execute (System.Collections.Generic.List`1[T] list) (at Assets/NGUI/Scripts/Internal/EventDelegate.cs:644)
9. UIButton.OnClick () (at Assets/NGUI/Scripts/Interaction/UIButton.cs:248)
10. UnityEngine.GameObject:SendMessage(String, Object, SendMessageOptions)
11. UICamera:Notify(GameObject, String, Object) (at Assets/NGUI/Scripts/UI/UICamera.cs:1912)
12. UICamera:ProcessRelease(Boolean, Single) (at Assets/NGUI/Scripts/UI/UICamera.cs:2813)
13. UICamera:ProcessTouch(Boolean, Boolean) (at Assets/NGUI/Scripts/UI/UICamera.cs:2865)
14. UICamera:ProcessMouse() (at Assets/NGUI/Scripts/UI/UICamera.cs:2277)
15. UICamera:ProcessTouches() (at Assets/NGUI/Scripts/UI/UICamera.cs:2391)
16. UICamera:ProcessEvents() (at Assets/NGUI/Scripts/UI/UICamera.cs:2091)
17. UICamera:Update() (at Assets/NGUI/Scripts/UI/UICamera.cs:2050)

Basically happens exactly on this line:
cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);

This was working with the previous version, I didn't change or update anything else.

PS: UICamera is NGUI camera (Got the Main camera and the UICamera) but again, previous version was fine.

 

Hi, Lanre I just bought your Natcorder great work.
So with the help of Natcorder I am able to record the video but now what I want is after the video is recorded I can edit video like adding audio, some UI effects, etc and when I am done editing the video I can save the new video to my gallery.
Can you please tell me how should I do it?

 

Hello,
we are working on a feature where we need to access the Device Cams (Front/Back) and the Mic on Android and iOS and and save the input (both video and audio) to devicedisk in Unity 2018.4.X. Is this possible with this Asset? I saw that the current version is 2019.2 +, but does a version for 2018.4 exist that meets our requirements?
Cheers John

 

In portrait-locked app. I want to record in landscape mode.
What should I do.

App screen,
width : 1080
height : 2272

Mp3
target height : 1080
target width : 2270

 

i'm having the same problem, just opened a new project, imported NatCorder, changed to IL2CPP and .NET 4.X and tried to build.

waiting for the reply.

 

SO MANY BUGS!

i just download the package and build the demo "ReplayCam" and did not word!

unity 2019.2.18f, iOS13.3.1

Code (CSharp):

1. 2020-02-06 10:31:14.875905+0800 tt[3932:1064769] Built from '2019.3/staging' branch, Version '2019.3.0f5 (44796c9d3c2c)', Build type 'Release', Scripting Backend 'il2cpp'
2. -> applicationDidFinishLaunching()
3. -> applicationDidBecomeActive()
4. GfxDevice: creating device client; threaded=1
5. Initializing Metal device caps: Apple A13 GPU
6. Initialize engine version: 2019.3.0f5 (44796c9d3c2c)
7. 2020-02-06 10:31:15.464833+0800 tt[3932:1064769] Unbalanced calls to begin/end appearance transitions for <SplashScreenController: 0x102d3cef0>.
8. UnloadTime: 0.280083 ms
9. Setting up 1 worker threads for Enlighten.
10.   Thread -> id: 16edeb000 -> priority: 1
11. 2020-02-06 10:31:21.656909+0800 tt[3932:1064769] NatCorder: Prepared MP4Recorder video encoder at resolution 720x1280@30.000000Hz with average bitrate 5909760 and keyframe interval 3s
12. ArgumentException: RenderTextureDesc msaaSamples must be 1, 2, 4, or 8.
13. Parameter name: desc.msaaSamples
14.   at UnityEngine.RenderTexture.ValidateRenderTextureDesc (UnityEngine.RenderTextureDescriptor desc) [0x00000] in <00000000000000000000000000000000>:0
15.   at UnityEngine.RenderTexture.GetTemporary (UnityEngine.RenderTextureDescriptor desc) [0x00000] in <00000000000000000000000000000000>:0
16.   at NatCorder.Inputs.CameraInput..ctor (NatCorder.IMediaRecorder recorder, NatCorder.Clocks.IClock clock, UnityEngine.Camera[] cameras) [0x00000] in <00000000000000000000000000000000>:0
17.   at NatCorder.Examples.ReplayCam.StartRecording () [0x00000] in <00000000000000000000000000000000>:0
18.   at UnityEngine.Events.UnityAction.Invoke () [0x00000] in <00000000000000000000000000000000>:0
19.   at UnityEngine.Events.UnityEvent.Invoke () [0x00000] in <00000000000000000000000000000000>:0
20.   at NatCorder.Examples.RecordButton+<Countdown>d__10.MoveNext () [0x00000] in <00000000000000000000000000000000>:0
21.   at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <00000000000000000000000000000000>:0
22. (Filename: currently not available on il2cpp Line: -1)
23.  
24.  

 

I got the same problem!

 

Im experiencing this as well.

 

Hi
I'm trying to use natcorder to record a AR scene but it crashes when it finish writing the video. In Xcode I get the message:

NatCorder: MP4Recorder finishing

and then it crashes. running in the editor makes unity to crash but It creates de video. I tried to dig in this tread but every time a see a similar post the answer is alway "send us the log" I would have liked to see what happened in all this cases , maybe there is a solution to all of us.

The code I'm using is :

Code (CSharp):

1.    
2. public void StartRecording()
3.     {
4.         _clock = new RealtimeClock();
5.         _recorder = new MP4Recorder(_videoWidth, _videoHeight, 30);
6.         _cInput = new CameraInput(_recorder, _clock,Camera.allCameras);
7.         _recording = true;
8.     }
9.  
10.     public void OnSampleBuffer(float[] sampleBuffer, int sampleRate, int channelCount, long timestamp)
11.     {
12.         if (_recording)
13.         {
14.             _recorder.CommitSamples(sampleBuffer, _clock.timestamp);
15.         }
16.     }
17.  
18.     public async void StopRecording()
19.     {
20.         var path = await _recorder.FinishWriting();
21.         _recorder = null;
22.         _recording = false;
23.     }
24.  

I'm using natcorder 1.7.0
unity 2019.2.19
AR Foundation 3.0.1
IOS 13.3
MacOS 10.14.6

Thank you for your help

 

Can you try deleting and reimporting NatCorder from the Asset Store? This sounds like a UI issue (broken references in the scene maybe).

 

This is complicated, because Unity doesn't provide an easy way to do it. You'll need to create a custom CameraInput implementation that crops the frame using a shader.

 

This is because of the anti aliasing setting in CameraInput.cs. Open CameraInput.cs and in the constructor, remove this line:

Code (CSharp):

1. frameDescriptor.msaaSamples = QualitySettings.antiAliasing;

 

Set your `_recording` variable to false before calling `FinishWriting`. Setting it after means that you try to commit audio samples **after** the recorder has shut down, which will almost always result in a crash.

 

Thank you for the reply
I tried putting _recording before FinishWriting and I also checked if _recorder is null just to be sure but Unity continue crashing

 

Do you have an example of that shader for the URP?

 

Hi. Where can I found a basic example using CameraInput and AudioInput for Natcorder 1.7.0?
Thanks

 

Can you share the crash logs?

The ReplayCam example.

 

Nope. You'll have to look at how to create image shaders that do cropping. As I mentioned earlier, it isn't trivial.

 

Yes, I attach the log but the problem is now when a run it in an iPad. It finally runs in Unity editor

I Changed my code to something like this :

Code (CSharp):

1.  
2. private void Start()
3.     {
4.         _videoWidth = 1024;
5.         _videoHeight = 768;
6.     }
7.  
8.     public void StartRecording()
9.     {
10.  
11.         Debug.Log(" Start recording");
12.         _clock = new RealtimeClock();
13.         _recorder = new MP4Recorder(_videoWidth, _videoHeight,30,AudioSettings.outputSampleRate,(int)AudioSettings.speakerMode);
14.         _cInput = new CameraInput(_recorder, _clock, _mainCamera);
15.        _aInput = new AudioInput(_recorder, _clock, _mainCamera.GetComponent<AudioListener>());
16.  
17.  
18.     }
19.  
20.     public void StopRecording()
21.     {
22.  
23.         Debug.Log(" Stop recording");
24.  
25.              if (_cInput != null)
26.                 _cInput.Dispose();
27.  
28.             if (_aInput != null)
29.             {
30.                 _aInput.Dispose();
31.             }
32.        
33.         string path = await _recorder.FinishWriting();
34.  
35.             if (_recorder != null)
36.                 _recorder = null;
37.  
38.            
39.  
40.     }

Thanks again for the support

 

Apparently no frames are being committed before FinishWriting is called. Make sure that your `_mainCamera` variable is not null, and that you are not calling stop recording too soon.

 

Hi, today i bought NatCorder Asset, and my first test was to try to make it work in android build, i´ve set minimum api version to 21, used the ReplayCam sample and tryed it in my devide (samsung galaxy note 8). I didn´t change anything, but im getting this error, in the files there are Mp4 saved with 0 bytes. im using Unity 2018.3.0f3, there is something im missing to correctly run this?

 

The fix is pending approval by Unity. The workaround is simple: enable anti-aliasing in Quality Settings in the Unity Editor.

 

How can i record rendertexture?
i tried convert rendertexture to texture2d with readpixels but its too bad performance
any solution?

 

See AsyncGPUReadback.

 

Thanks Lanre you save my life

 

Hi Lanre,

My users still get crashes:

Samsung Galaxy S8 (dreamlte), Android 9

java.lang.Error: FATAL EXCEPTION [MP4Recorder Video Encoding Thread] Unity version : 2019.2.17f1

Device model : samsung SM-G950F
Caused by
at android.media.MediaCodec.native_dequeueOutputBuffer (Native Method)
at android.media.MediaCodec.dequeueOutputBuffer (MediaCodec.java:2698)
at api.natsuite.natcorder.MP4Recorder$6.run (MP4Recorder.java:192)
at java.lang.Thread.run (Thread.java:764)

Here's my code:

Code (CSharp):

1.  
2.   public async void NatcorderStopRecordingAndroid()
3.     {
4.         // Mute microphone
5.         if (recordMicrophone)
6.         {
7.             microphoneSource.mute = true;
8.         }
9.         // Stop recording
10.         audioInput?.Dispose();
11.         cameraInput.Dispose();
12.         try
13.         {
14.             var path = await videoRecorder.FinishWriting();
15.             OnReplay(path);
16.         }
17.         catch
18.         {
19.             gameController.uiSetup.ShowNotificationRecordingSavedError();
20.  
21.         }
22.     }

 

Can you share your full recording code?

 

Code (CSharp):

1.     public void NatCorderStartRecording()
2.     {
3.         int height;
4.         int width;
5.         if (Screen.width <= 720)
6.         {
7.             height = 720;
8.             width = (int)(height * ((float)Screen.width) / ((float)(Screen.height)));
9.         }
10.         else
11.         {
12.             height = Screen.height;
13.             width = Screen.width;
14.         }
15.         recordingClock = new RealtimeClock();
16.         videoRecorder = new MP4Recorder(
17.     width,
18.     height,
19.     30,
20.      AudioSettings.outputSampleRate,
21.     2
22. );
23.         // Create recording inputs
24.         cameraInput = new CameraInput(videoRecorder, recordingClock, cameraToRecord);
25.         audioInput = new AudioInput(videoRecorder, recordingClock, audioListener);
26.     }
27.  
28.     public async void NatcorderStopRecordingAndroid()
29.     {
30.         // Mute microphone
31.         if (recordMicrophone)
32.         {
33.             microphoneSource.mute = true;
34.         }
35.         // Stop recording
36.         audioInput?.Dispose();
37.         cameraInput.Dispose();
38.         //var path = await videoRecorder.FinishWriting();
39.         //OnReplay(path);
40.         try
41.         {
42.             var path = await videoRecorder.FinishWriting();
43.             OnReplay(path);
44.         }
45.         catch
46.         {
47.             gameController.uiSetup.ShowNotificationRecordingSavedError();
48.             AnalyticsEvent.Custom("Exception thrown when recording.");
49.         }
50.     }

I think I found the issue. The width was set to an odd number not even, on the affected devices. I'll see if it fixes the problem.

 

Solved , I was using a disabled camera. Thank you so much for for help

 

Hello Lenre!

I am Recording Main Camera Input

public void StartRecording()
{
var frameRate = 30;
var sampleRate = recordMicrophone ? AudioSettings.outputSampleRate : 0;
var channelCount = recordMicrophone ? (int)AudioSettings.speakerMode : 0;
var recordingClock = new RealtimeClock();
videoRecorder = new MP4Recorder(
480,
854,
frameRate,
sampleRate,
channelCount,
recordingPath => {
Debug.Log($"Saved recording to: {recordingPath}");
//var prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
//Handheld.PlayFullScreenMovie($"{prefix}{recordingPath}");
}
);
// Create recording inputs
cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
}

Recording Is Working fine but when Android Device Resolution 1440*3120 Video Something looks like.
we want full screen video in all android device.can you help me please ?
Thank You in advance.

 

Your resolution calculation is definitely the problem. I recommend recording at a standard resolution, like 1280x720 or 1920x1080. Android devices are nitpicky about resolutions.

 

This depends on your app's ability to rescale when the screen size changes. You can see this yourself in the Editor; simply change the aspect of the game view. You have to design your app's UI to scale desirably depending on the screen resolution. It has nothing to do with NatCorder.

 

Hey Lanre,
I'm trying to make an app to record some videos to fill a report, this videos must be on portrait mode and I'm trying to do it with webcamtexture, I can show the preview of the camera an record it, but when I look at the file it is flipped as if the recording was made on landscape, do you know what can I do for the final file to be in portrait mode?



Here's what I have in code

Code (CSharp):

1. using System.Collections;
2. using NatCorder;
3. using NatCorder.Clocks;
4. using NatCorder.Inputs;
5. using UnityEngine;
6. using UnityEngine.UI;
7.  
8. public class VideoCamera : MonoBehaviour
9. {
10.     public Image recordButton;
11.     private WebCamTexture webCamTexture;
12.     private WebCamTextureInput webCamTextureInput;
13.     public RawImage rawImage;
14.     public AspectRatioFitter aspectFitter;
15.  
16.     private IMediaRecorder recorder;
17.  
18.     private IEnumerator Start()
19.     {
20.         // Request camera permission
21.         yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
22.         if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
23.             yield break;
24.         // Start the WebCamTexture
25.         webCamTexture = new WebCamTexture(1280, 720, 30);
26.         webCamTexture.Play();
27.         // Display webcam
28.         yield return new WaitUntil(() => webCamTexture.width != 16 && webCamTexture.height != 16); // Workaround for weird bug on macOS
29.         rawImage.texture = webCamTexture;
30.         aspectFitter.aspectRatio = (float)webCamTexture.width / webCamTexture.height;
31.     }
32.  
33.     private void Update()
34.     {
35.         if (webCamTexture == null) return;
36.         var ratio = webCamTexture.width / (float)webCamTexture.height;
37.         aspectFitter.aspectRatio = ratio;
38.  
39.         var scaleY = webCamTexture.videoVerticallyMirrored ? -1f : 1f;
40.         var rectTransform = rawImage.rectTransform;
41.         rectTransform.localScale = new Vector3(1f, scaleY, 1f);
42.  
43.         var orientation = -webCamTexture.videoRotationAngle;
44.         rectTransform.localEulerAngles = new Vector3(0, 0, orientation);
45.     }
46.  
47.     public void StartRecording()
48.     {
49.         // Start recording
50.         recordButton.color = Color.red;
51.         var clock = new RealtimeClock();
52.         recorder = new MP4Recorder(webCamTexture.width, webCamTexture.height, 30);
53.         webCamTextureInput = new WebCamTextureInput(recorder, clock, webCamTexture);
54.     }
55.  
56.     public async void StopRecording()
57.     {
58.         // Stop recording
59.         recordButton.color = Color.white;
60.         webCamTextureInput.Dispose();
61.         var path = await recorder.FinishWriting();
62.         // Playback recording
63.         Debug.Log($"Saved recording to: {path}");
64.         var prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
65.         Handheld.PlayFullScreenMovie($"{prefix}{path}");
66.     }
67. }

 

Hi, there is any way to commitFrame using the last frame render from the main camara without using a renderTexture?

 

On mobile platforms, WebCamTexture produces a preview that is rotated depending on the app orientation (see the WebCamTexture.videoRotationAngle property). We display the WebCamTexture upright in our examples because we use a shader to display it upright, but this has no effect on the actual pixel data, which is what is sent to the recorder to be added to the video. Your best bet is to record the screen instead.

 

You need a RenderTexture to capture what a game camera sees.

 

I'm using Natcorder 1.7.0 and am experiencing a hard crash on Android, only on the Motorola G7 (power). Specifically It's on initialisation of a new MP4Recorder object, and the crash occurs when NatCorder accesses its DLL from NatCorder.Internal.Bridge.CreateMP4Recorder.

All other Android devices I've tested the same .apk on works fine.

 

Can you share the logs from logcat in a .txt file? That's the only way to know what's going on.

 

Hello, I'm using NatCorder and I realised that my shader run twice as fast when I start the recording, do you have any idea of what could be wrong?
This is true with multiple shaders

Code (CSharp):

1.  
2.         // Start recording
3.         recorder = new MP4Recorder(
4.             1024,
5.             768,
6.             30,
7.             0, // audio stuff
8.             0, // audio stuff
9.             OnReplay
10.         );
11.         clock = new RealtimeClock();
12.         cameraInput = new CameraInput(recorder, clock, EffectCamera);
13.  

Shaders are render shader called in OnRenderImage methods

 

CameraInput records the game camera by rendering it to an off-screen RenderTexture, so your camera will be rendered twice (once for the screen and once for recording).

 

Thank you for this quick reply, any quick win to prevent this?

 

It isn't a bug to be prevented, this is required for CameraInput to work.

 

I understood that, I was wondering how I can prevent this, probably using some sort of other setup than CameraInput I guess?

 

Doing this is not trivial, as you will likely have to make your camera render to an off-screen render texture then blit that render texture to the screen. All sorts of issues come up with this approach, so I don't recommend it.