NatCorder - Video Recording API

Hello. When will be the release of v1.6.3? We need the fix for the IOS video problem.

Thank you for the plugin. It's amazing!

 

This probably happens because of aspect fitting. When the recording size has a different aspect ratio from that of the screen, Unity recomputes the viewbox when rendering the camera to the recording render texture. The simplest way to avoid this is to make sure that your game renders properly across different aspect ratios.

 

It has been released.

 

@poshaughnessey We are running into the same issue on iOS. Creating and disposing an MP4Recorder does not seem to help. Have you solved this?

 

It isn't something we can solve. We don't control what resources the encoder allocates when we start recording. Have you tried the 1.6.3 update?

 

Hi, thank you for the response. I have indeed used the 1.6.3 update. But it happened before on 1.6.2 as well. I am not sure how to approach this, as I have already tried to "warm up" the MP4Recorder beforehand, and it doesn't work sadly.

 

Hi Lanre,
I think will buy NatCorder to record video + audio, but I not sure NatCorder can pause/resume in recoding both video & audio, so, it can do that?

 

I have managed to solve this by warming both the MP4Recorder and the microphone as well (I'm using NatMic). Waiting for a second to receive some data, then disposing the recorders, and deleting the file, seems to make it so the next recording is smooth.

 

See this article, under "Pausing Recording".

 

Hi Lanre,
I have been working on updating my app to Android 10 and noticed video recording was throwing a bunch of errors, so I updated from Natcorder 1.6 to 1.63 but now my app is hard crashing 1-2 seconds after recording has started. I am using Unity 2019.2.8f1 and testing on a Google Pixel 2 running Android 10 and a Google Pixel 1 running Android 9, it is crashing on both devices now. I've attached the logcat from a run/crash of the app.

Edit 01 --
After disabling audio recording, the app no longer hard crashes but throws this warning "AndroidVideoMedia: Error opening extractor: -10002".

Edit 02 --
After doing some exhaustive research it looks like this isn't an issue with Natcorder, but an issue with how Android 10 is handling URLs by default...

 

Hi Lanre, thank you for the response!
In the article , I focus on part NatCorder + NatMic because it have the case I want to do "recording video with microphone audio"
so, I need both NatCorder and NatMic to do it, or just need NatCorder to do it?
Thank!

 

Thanks for the 1.6.3 update. Unfortunately, I had to go back to 1.6.2 as it caused Unity to crash on MacOS Mojave after it completed a screen recording.

 

You can use NatCorder with Unity's Microphone API (the ReplayCam example already does this). But if/when you face problems with glitchy audio from Unity, you can switch to NatMic.

 

Can you share the crash logs?

 

Hi Lanre,

right now we have a crash on iOS. I think it's this line:

-[AGXA11FamilyCommandBuffer blitCommandEncoder], line 107: error '<private>'

any idea? For some context, this is our frameloop:

Code (CSharp):

1.         protected override RenderTexture PrepareFrame(RenderTexture framebuffer)
2.         {
3.             // copy frame webcam texture to encoder frame
4.             var webcamTex = _blender.GetWebcamTexture();
5.             var chromakeyMat = _blender.GetOutputMaterial();
6.  
7.             // render webcam onto empty texture, using VideoBlender output with masked video as masking Shader
8.             Graphics.Blit(webcamTex, framebuffer, chromakeyMat);
9.  
10.             // add watermark
11.             Graphics.DrawTexture(new Rect(-130, -275, 66, -10), _watermarkTex, null, -1);
12.  
13.             return framebuffer;
14.         }
15.  

the

_blender

code combines webcam video with chroma-keyed out mask that show a video playing using AVProVideo.

another smaller issue is that (in Editor WIN) the onSaved never fires is audio recording is enabled.

 

Same problem here after update. Also on Android after long ( > 7 sec recording) app freezes.
How to get previous version? That worked at least

 

I've gotten a few reports of crashing when recording is stopped. I have a fix in the works and will hopefully push an update tomorrow. Can you upload the logs from logcat?

 

Sorry, can't because of freezing been detected on client's Pixel 3 device (Android X). My Xiaomi Redmi k20 pro runs well

 

Any idea how to fix this error?

Code (JavaScript):

1. 10/10 17:13:11.550 26363 26476 Error Unity: AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'boolean android.os.Handler.post(java.lang.Runnable)' on a null object reference
2. 10/10 17:13:11.550 26363 26476 Error Unity: java.lang.NullPointerException: Attempt to invoke virtual method 'boolean android.os.Handler.post(java.lang.Runnable)' on a null object reference
3. 10/10 17:13:11.550 26363 26476 Error Unity:     at com.olokobayusuf.natcorder.MP4Recorder.encodeSamples(MP4Recorder.java:161)
4. 10/10 17:13:11.550 26363 26476 Error Unity:   at UnityEngine.AndroidJNISafe.CheckException () [0x00000] in <00000000000000000000000000000000>:0
5. 10/10 17:13:11.550 26363 26476 Error Unity:   at UnityEngine.AndroidJavaObject._Call (System.String methodName, System.Object[] args) [0x00000] in <00000000000000000000000000000000>:0
6. 10/10 17:13:11.550 26363 26476 Error Unity:   at NatCorder.Internal.MediaRecorderAndroid.CommitSamples (System.Single[] sampleBuffer, System.Int64 timestamp) [0x00000] in <00000000000000000000000000000000>:0
7. 10/10 17:13:11.550 26363 26476 Error Unity:
8. 10/10 17:13:11.550 26363 26476 Error Unity: (Filename: currently not available on il2cpp Line: -1)

this is the C# that records audio:

Code (CSharp):

1.         protected virtual void OnAudioFilterRead(float[] data, int channels)
2.         {
3.             if (!_recording || !HasAudio) return;
4.  
5.             AndroidJNI.AttachCurrentThread();
6.             _videoRecorder.CommitSamples(data, _recordingClock.Timestamp);
7.             AndroidJNI.DetachCurrentThread();
8.         }
9.  

 

iOS 13 crash issues on pressing the record button with NatMicCorder.cs even running just the ReplayCam scene...

2019-10-10 22:44:30.709266-0700 polygoncamera[1163:369276] NatCorder: Prepared MP4 video encoder at resolution 720x1280@30.000000Hz with average bitrate 5909760 and keyframe interval 3s

2019-10-10 22:44:30.709976-0700 polygoncamera[1163:369276] NatCorder: Prepared MP4 audio encoder for 2 channels at 24000Hz



UnityGfxDeviceWorker (35)#0 0x00000001a3a82238 in objc_retain ()
#1 0x0000000105ab13fc in -[NCMTLReadback readback:withCompletionHandler:] at /Users/yusuf/Documents/Projects/NatCorder-API/iOS/NatCorder/NCMTLReadback.m:55
#2 0x0000000105ab180c in NCMTLReadbackReadback at /Users/yusuf/Documents/Projects/NatCorder-API/iOS/NatCorder/NCMTLReadback.m:90
#3 0x0000000106aa45e0 in ::Action_Invoke_mC8D676E5DDF967EC5D23DD0E96FB52AA499817FD(Action_t591D2A86165F896B4B800BB5C25CE18672A55579 *, const RuntimeMethod *) ()
#4 0x0000000106560e00 in ::RenderDispatcher_DequeueRender_mC3C2608E5FD8E37C7122E18D4348E3BC9C071221(int32_t, const RuntimeMethod *) at /Users/yosunchang/MyProjects/PolyCam/PolyCam/test3/Classes/Native/Assembly-CSharp1.cpp:24343
#5 0x0000000104fe9dd8 in ::ReversePInvokeWrapper_RenderDispatcher_DequeueRender_mC3C2608E5FD8E37C7122E18D4348E3BC9C071221(int32_t) at /Users/yosunchang/MyProjects/PolyCam/PolyCam/test3/Classes/Native/Assembly-CSharp1.cpp:24266
#6 0x0000000105474b24 in ::InsertCustomMarkerCallback() at /Users/builduser/buildslave/unity/build/Runtime/GfxDevice/GfxDevice.cpp:1993
#7 0x00000001051f9238 in ::RunCommand() ()
#8 0x00000001051fb070 in ::RunExt() at /Users/builduser/buildslave/unity/build/Runtime/GfxDevice/threaded/GfxDeviceWorker.cpp:494
#9 0x00000001051f37b4 in ::RunGfxDeviceWorker() at /Users/builduser/buildslave/unity/build/Runtime/GfxDevice/threaded/GfxDeviceWorker.cpp:457
#10 0x00000001054393f0 in ::RunThreadWrapper() at /Users/builduser/buildslave/unity/build/Runtime/Threads/Thread.cpp:78
#11 0x00000001a3a58d5c in _pthread_start ()

 

I have an issue when use NatCorder with external ffmpeg library https://github.com/tanersener/mobile-ffmpeg to concat video effects on iOS. App crashes when start recording. Is there any conflict on libraries between them?

 

Could you please explain how to correctly record audio with v1.6.3? Whatever I try, I can't get past this error:

Code (CSharp):

1. 10/10 17:13:11.550 26363 26476 Error Unity: java.lang.NullPointerException: Attempt to invoke virtual method 'boolean android.os.Handler.post(java.lang.Runnable)' on a null object reference
2. 10/10 17:13:11.550 26363 26476 Error Unity:     at com.olokobayusuf.natcorder.MP4Recorder.encodeSamples(MP4Recorder.java:161)

 

Can you share your recorder creation code? You are likely passing in an incorrect sample rate and channel count to the MP4Recorder constructor.

 

I suspect that your app is rendering with OpenGL ES. This is not supported. NatCorder requires Metal on iOS and macOS.

 

this is the code I'm using:

Code (CSharp):

1.             _videoRecorder = new MP4Recorder(
2.                 Screen.width,
3.                 Screen.height,
4.                 _fps,
5.                 HasAudio ? AudioSettings.outputSampleRate : 0,
6.                 HasAudio ? (int) AudioSettings.speakerMode : 0,
7.                 OnVideoSaved);

The audio that I need to record differs (loaded dynamically from backend). Is that a problem?
Rephrase.. how do I know what the correct samplerate and channelCount is..?

 

If you are audio recording from Unity, then AudioSettings is the correct place to check. Is your `HasAudio` boolean true when you try to create the recorder? Can you upload the full logs from logcat in a .txt attachment?

 

I too am experiencing Unity crash after completing recording using 1.6.3. Noticed it happens if I use VirtualDevice or MixerDevice, but not AudioDevice.GetDevices()[0]. I have NatMic installed and updated to the latest.
Edit: crash also happens with AudioDevice.GetDevices()[0], but less frequently

Furthermore, I am getting odd behavior, stalls, and / or crashes in editor when setting up an MP4Recorder with AudioSettings.outputSampleRate, and (int)AudioSettings.speakerMode.Using 44100 and 1 for those params fixes the issue.

Also, if anyone upgrading NatCorder sees the following error:
library not found for -lNatRender
Delete your build folder and rebuild. Simply cleaning it doesn't work

Edit: using Unity 2018.4.4

 

What about for Android?

 

Hmm, with NatCorder 1.6.3, I'm seeing a small green strip at the bottom of every video, 1-2 pixels high, spanning the width of the video. Doesn't appear at low resolutions (400x866) but does appear at higher (540x1170, 720x1560).

Any ideas? Thank you!

Unity 2018.4.4, iOS 13.1.2, iPhone X

Edit: This was fixed by ensuring that both my RenderTexture source and video output resolutions had evenly divisible dimensions.

 

We are using natcorder 1.6.3.It works good at most times, but some times the game hang while the recording is stopped. We find the error message in the logs:
10-12 14:21:42.046: W/MessageQueue(14042): Handler (android.os.Handler) {65617e7} sending message to a Handler on a dead thread
10-12 14:21:42.046: W/MessageQueue(14042): java.lang.IllegalStateException: Handler (android.os.Handler) {65617e7} sending message to a Handler on a dead thread
10-12 14:21:42.046: W/MessageQueue(14042): at android.os.MessageQueue.enqueueMessage(MessageQueue.java:543)
10-12 14:21:42.046: W/MessageQueue(14042): at android.os.Handler.enqueueMessage(Handler.java:643)
10-12 14:21:42.046: W/MessageQueue(14042): at android.os.Handler.sendMessageAtTime(Handler.java:612)
10-12 14:21:42.046: W/MessageQueue(14042): at android.os.Handler.sendMessageDelayed(Handler.java:582)
10-12 14:21:42.046: W/MessageQueue(14042): at android.os.Handler.post(Handler.java:338)
10-12 14:21:42.046: W/MessageQueue(14042): at com.olokobayusuf.natcorder.readback.GLESReadback.readback(GLESReadback.java:77)


Please check the logs in attached files and tell us how to prevent this, Thanks!

The recording is very frequently used in our game, and each recording is started after a short time when the previous is stopped.

 

I'm working on a fix for this. I know where it's coming from. Fix should be up by Monday or Tuesday.

 

On Android, NatCorder requires OpenGL ES3. It does not support Vulkan or OpenGL ES 2.

 

Okay so how does one set the project to work for both platforms?

 

Hi Lanre,
I use NatCorder ver 1.6.3, Unity ver 2018.2.0f2, and when call StopRecording in ReplayCam example, unity go crash, but it working fine on iOS device
How to fix it or need setting something?

 

@Lanre how long to wait for the fix? I have been requested by email the previous version package witch worked fine, but you didn't even reply.

 

FYI: We just fixed some crashes and/or 0 byte files by making sure the recorded resolution is an EVEN amount of pixels.
@Lanre, can you confirm or deny this? It seems that the MP4Recorder needs specific dimensions and crashes if encountering ODD number of pixels. Does it need powerOfTwo? or just even?

 

Hi @Lanre, I am facing the Unity editor crash with Natcorder 1.6.3 as well.

Also, I am facing another weird issue with the videos recorded from an android app using Natcorder 1.6.3. The videos recorded with audio input as an audio source don't play in the Google Chrome browser. The videos recorded without audio input works well in all the browsers but the ones recorded with audio input does not play at all in the chrome browser. I am doing the standard code to record videos with audio input as shown in the Replay cam script.

Code (CSharp):

1. public void StartRecording()
2.         {
3.             // Start recording
4.             recordingClock = new RealtimeClock();
5.             videoRecorder = new MP4Recorder(
6.                 videoWidth,
7.                 videoHeight,
8.                 30,
9.                 recordMicrophone ? AudioSettings.outputSampleRate : 0,
10.                 recordMicrophone ? (int)AudioSettings.speakerMode : 0,
11.                 OnReplay
12.             );
13.             // Create recording inputs
14.             cameraInput = new CameraInput(videoRecorder, recordingClock, MainCamera);
15.             if (recordMicrophone)
16.             {
17.                 StartMicrophone();
18.                 audioInput = new AudioInput(videoRecorder, recordingClock, microphoneSource, true);
19.             }
20.         }
21.  
22.         private void StartMicrophone()
23.         {
24. #if !UNITY_WEBGL || UNITY_EDITOR // No `Microphone` API on WebGL :(
25.             // Create a microphone clip
26.             microphoneSource.clip = Microphone.Start(null, true, 60, 48000);
27.             while (Microphone.GetPosition(null) <= 0) ;
28.             // Play through audio source
29.             microphoneSource.timeSamples = Microphone.GetPosition(null);
30.             microphoneSource.loop = true;
31.             microphoneSource.Play();
32. #endif
33.         }
34.  
35.         public void StopRecording()
36.         {
37.             // Stop the recording inputs
38.             if (recordMicrophone)
39.             {
40.                 StopMicrophone();
41.                 audioInput.Dispose();
42.             }
43.             cameraInput.Dispose();
44.             // Stop recording
45.             videoRecorder.Dispose();
46.         }
47.  
48.         private void StopMicrophone()
49.         {
50. #if !UNITY_WEBGL || UNITY_EDITOR
51.             Microphone.End(null);
52.             microphoneSource.Stop();
53. #endif
54.         }
55.  

Weirdly enough, the videos with audio input plays in firefox and edge but fail to play in Chrome. We need the videos with microphone audio input to play in browsers as we allow the game play videos recorded from the app to be uploaded to our website.

Is anyone else facing this issue in the latest version 1.6.3?

@Lanre, please guide me on how to solve this?

 

Here's my log

 

Different platforms use different rendering API's. So whatever Android is using does not have anything to do with what iOS is using. See Player Settings in the Editor.

 

I'm submitting 1.6.4 with the fix later today. Should be live tomorrow or Wednesday.

 

Recording may fail when one of the dimensions is an odd number, but it depends on the encoder. Some encoders pad resolutions whereas others don't. So the best I can say is that to be safe, always make sure your recording resolution is even--at the very least. The safest thing to do is to record at a standard resolution: 640x480, 1280x720, or 1920x1080.

 

I got your email. I've been able to reproduce it, but I am not familiar with any way to fix it (especially because it is more of a Chrome issue than a NatCorder issue). I'll keep looking into it to see if I can find anything.

 

Only a video encoder is being created, which means that either your sample rate, channel count, or both are zero in the MP4Recorder constructor. Check your code.

 

Hi Lanre,
I'm testing ReplayCam example on my Iphone 6 - iOS 12, record work ok, but when preview by call Handheld.PlayFullScreenMovie("file://" + path), it not play sound, then I check it on camera roll, it has sound
I don't know it's problem of API or something else

 

Check that your phone is not on silent mode. This is not a NatCorder issue.

 

I can alter the webcam view shown on iOS for a front facing camera, but the recorded MP4 is rotated 180 degrees, so looks flipped upside down.

Code (CSharp):

1. void Update () {
2.             // Record frames
3.             if (videoRecorder != null && webcamTexture.didUpdateThisFrame) {
4.                 webcamTexture.GetPixels32(pixelBuffer);
5.                 videoRecorder.CommitFrame(pixelBuffer, recordingClock.Timestamp);
6.             }
7.         }

I think it's to do with the GetPixels32, but I'm stuck.

Can anyone help?

 

You can either rotate the pixel buffer in memory (somewhat complicated, but possible) or use the GPU (blit to a rendertexture with a shader that performs rotation) and readback the rotated pixel buffer to commit.

 

Till 1.6.0 or 1.6.1 version, the videos recorded with audio input played well in all the browsers. I am facing this issue after upgrading to 1.6.2 or 1.6.3. In the meantime, I have raised it as an issue to chrome and also posted on their support forum. But till I get any reply from there, I was really hoping if you can help figure out why only videos with audio input have such problems, the ones without audio input seem to play fine in all the browsers.

 

As far as I've seen, this is a Chrome issue (especially because it works on other browsers). Let's see what they find.

 

Thanks. Can you possibly help with the coding - I don't understand