NatCorder - Video Recording API

The crash in the logs you shared don't show a NatCorder exception at all. Instead, the crash looks to be coming from Unity (notice that the call stack is entirely from libunity.so):

Code (CSharp):

1. 10-03 10:58:33.353 12187 12187 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2. 10-03 10:58:33.353 12187 12187 F DEBUG   : Build fingerprint: 'google/walleye/walleye:8.1.0/OPM4.171019.021.E1/4769646:user/release-keys'
3. 10-03 10:58:33.353 12187 12187 F DEBUG   : Revision: 'MP1'
4. 10-03 10:58:33.353 12187 12187 F DEBUG   : ABI: 'arm'
5. 10-03 10:58:33.353 12187 12187 F DEBUG   : pid: 11272, tid: 11312, name: UnityMain  >>> mycompany.myapp.android <<<
6. 10-03 10:58:33.353 12187 12187 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x34
7. 10-03 10:58:33.353 12187 12187 F DEBUG   : Cause: null pointer dereference
8. 10-03 10:58:33.353 12187 12187 F DEBUG   :     r0 00000000  r1 00000024  r2 00000000  r3 00000001
9. 10-03 10:58:33.353 12187 12187 F DEBUG   :     r4 00000b32  r5 d2b40b84  r6 cc3c7190  r7 000001e7
10. 10-03 10:58:33.353 12187 12187 F DEBUG   :     r8 c9262410  r9 d539b000  sl d277eaf8  fp d277ea84
11. 10-03 10:58:33.353 12187 12187 F DEBUG   :     ip 0000016c  sp d277e990  lr d2c1edbc  pc d2c1a7ac  cpsr 600b0010
12. 10-03 10:58:33.356 12187 12187 F DEBUG   :
13. 10-03 10:58:33.356 12187 12187 F DEBUG   : backtrace:
14. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #00 pc 0031a7ac  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
15. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #01 pc 0031edb8  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
16. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #02 pc 00240a14  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
17. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #03 pc 00244728  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
18. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #04 pc 00630d30  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
19. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #05 pc 004d48c0  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
20. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #06 pc 001a3fbc  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/lib/arm/libunity.so
21. 10-03 10:58:33.356 12187 12187 F DEBUG   :     #07 pc 00011423  /data/app/mycompany.myapp.android-EaKRlc5BbZ-MJtj3GHPV2Q==/oat/arm/base.odex (offset 0x11000)
22. 10-03 10:58:35.028  1120 12188 W ActivityManager: Process mycompany.myapp.android has crashed too many times: killing!

 

Good to see that you were able to stumble upon the cause. Declaring that you will record a media type (video is always expected, audio is optional) and not committing any samples of said media type will result in undefined behaviour (previous versions of NatCorder would simply crash the application immediately StopRecording was called). We don't log it explicitly because the MPEG writer will.

 

While recording the video, you can use a shader to rotate the video. NatCorder works by recording textures, so you can use a shader to transform each frame texture.

If you are using a `CameraRecorder` to record, you can set the `CameraRecorder.recordingMaterial` to a material with a shader that does what you want (in your case, rotating the frame texture) and the camera recorder will use this material to transform what the camera sees before sending the resulting texture to the hardware encoder.

Check out the GreyWorld example; it illustrates using a custom shader for recording with shader effects.

 

That's weird. In Audio Settings, what is your DSP buffer size set to?

I don't think the sample rate affects this at all, as Unity will resample the audio to match that of the audio engine (whatever AudioSettings.outputSampleRate is).

This might happen on the very first recording, but should not persist.

Can you email me a repro? `AudioSampleProvider` is apparently an experimental API introduced recently. This means there's more to this issue than meets the eye. I'll need to inspect the setup myself.

 

This is a problem a lot of developers, including us, are facing with Unity's Microphone API. We created our own microphone API called NatMic. It will be released sometime next week.

 

It’s totally a waste of money, I had purchased this API after reading all comments, but when I implemented this in my UNITY App for Android, it’s working very slow and most of the time it stopped working and crashed, I’d tried many ways to make it fast but I didn’t get any solution, even I’d try to contact NatCorder developers but they didn’t reply me. This is really shaming for NatCorder team. Please don’t make fool to developers who trust.
Everyplay far and far better than NetCoder. Please don’t trust on comments who want to buy NatCoder API.

 

Hello Lanre,

I bought your asset NatCorder yesterday, in order to use it in my Android/iOS game. I use Unity 2018.2.

Everything works well in the editor, I press the record button and start recording, I press it again and stop recording and the debuggerText shows where the movie was saved, and the movie plays in the video player that I created.

Now the problem is that when I run my game in Android, the "void OnRecordEnd(string path)" doesn't fire so nothing happens after I stop recording and I can't find any recorded video saved anywhere in my Android device.

Could you please help me to figure out if I have something wrong in my script?

I would appreciate any help with this issue.

Here is the script I am using for recording, stopping, and playing the movie in my game:

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Video;
3. using UnityEngine.UI;
4. using NatCorderU.Core;
5. using NatCorderU.Core.Recorders;
6. using System.Collections;
7. using System.Collections.Generic;
8.  
9. public class RecorderScript : MonoBehaviour
10. {
11.  
12.     [Header("Recording")]
13.     public Container container = Container.MP4;
14.     [Space]
15.     [Header("To switch the record button function between REC/STOP")]
16.     public bool isRecording = false;
17.     [Space]
18.     [Header("To Rotate the button while recording")]
19.     public RectTransform recButton;
20.     public Text debuggerText;
21.  
22.     Camera cam;
23.     CameraRecorder videoRecorder;
24.     List<GameObject> buttonList = new List<GameObject>();
25.  
26.     private void Start()
27.     {
28.         cam = GameObject.FindWithTag("MainCamera").GetComponent<Camera>();
29.         //debuggerText.gameObject.SetActive(false);
30.     }
31.  
32.  
33.  
34.     /// <summary>
35.     /// This is called from a button event trigger to start recording gameplay, that's why is public
36.     /// </summary>
37.     public void StartRecording()
38.     {
39.         if (!isRecording)
40.         {
41.             isRecording = true;
42.             StartCoroutine(RotateButton());
43.             AudioFormat audioFormat = new AudioFormat(AudioSettings.outputSampleRate, (int)AudioSettings.speakerMode);
44.             int width = 720;
45.             int height = width * Screen.height / Screen.width;
46.             int framerate = container == Container.GIF ? 10 : 30;
47.             VideoFormat videoFormat = new VideoFormat(width, height, framerate);
48.             NatCorder.StartRecording(container, videoFormat, audioFormat, OnRecordEnd);//Start recording
49.             videoRecorder = CameraRecorder.Create(cam);//Create a camera recorder to record the camera
50.             if (container == Container.GIF)
51.             {
52.                 videoRecorder.recordEveryNthFrame = 5;
53.             }
54.             cam.gameObject.AddComponent<AudioGetterScript>();
55.         }
56.         else
57.         {
58.             StopRecording();
59.         }
60.     }
61.  
62.  
63.  
64.     public void StopRecording()
65.     {
66.         isRecording = false;
67.         videoRecorder.Dispose();// Destroy the camera recorder
68.         NatCorder.StopRecording();// Stop recording
69.         Destroy(cam.gameObject.GetComponent<AudioGetterScript>());
70.     }
71.  
72.  
73.  
74.     void OnRecordEnd(string path)
75.     {
76.         //debuggerText.gameObject.SetActive(true);
77.         debuggerText.text = "Saved to: " + path;
78.         // Playback the video
79. //#if UNITY_EDITOR
80.         PlayMovie(path);
81. //#elif UNITY_IOS
82. //            Handheld.PlayFullScreenMovie("file://" + path);
83. //#elif UNITY_ANDROID
84. //        Handheld.PlayFullScreenMovie(path);
85. //#endif
86.     }
87.  
88.  
89.  
90.     IEnumerator RotateButton()
91.     {
92.         while (isRecording)
93.         {
94.             recButton.Rotate(0.0f, 0.0f, -400.0f * Time.deltaTime);
95.             yield return null;
96.         }
97.         recButton.localRotation = Quaternion.identity;
98.         StopCoroutine(RotateButton());
99.     }
100.  
101.  
102.  
103.     void PlayMovie(string path)
104.     {
105.         GameManagerScript.GmScript.controlIsActiveGM = false;
106.         HideUi();
107.         VideoPlayer videoPlayer = cam.gameObject.AddComponent<VideoPlayer>();
108.         videoPlayer.playOnAwake = false;
109.         videoPlayer.renderMode = VideoRenderMode.CameraNearPlane;
110.         videoPlayer.url = path;
111.         videoPlayer.Play();
112.         videoPlayer.loopPointReached += EndReached;
113.     }
114.  
115.     void EndReached(VideoPlayer vp)
116.     {
117.         GameManagerScript.GmScript.controlIsActiveGM = true;
118.         ShowUi();
119.         Destroy(cam.gameObject.GetComponent<VideoPlayer>());
120.         debuggerText.text = System.String.Empty;
121.         //debuggerText.gameObject.SetActive(false);
122.     }
123.  
124.  
125.     void HideUi()
126.     {
127.         GameObject[] tempArray = GameObject.FindGameObjectsWithTag("TagButton");
128.         for (int i = 0; i < tempArray.Length; i++)
129.         {
130.             buttonList.Add(tempArray[i]);
131.             buttonList[i].SetActive(false);
132.         }
133.     }
134.  
135.  
136.     void ShowUi()
137.     {
138.         for (int i = 0; i < buttonList.Count; i++)
139.         {
140.             buttonList[i].SetActive(true);
141.         }
142.     }
143.  
144. }
145.  

Regards,
Carlos

 

I never got your email. Can you try emailing me again? Make sure that you send me the crash logs and let me know what device you are testing on.

 

Hm this is weird. Upload the full logs from logcat; it should have clues as to what exactly is going on.

 

This is my logCat filtered with only error messages toward the end you can find many references to NatCorder. Please let me know asap.

 

Please edit this post and remove the logs. Instead, paste the logs in a .txt file and upload that. Please do so immediately. Also, I see a ClassNotFoundException. Are you using Proguard when building on Android?

 

Hello Lanre,

I edited the post and uploaded the logcat in a text file. I don't know what Proguard is, but I am not using any other plugings if that is what Proguard is. Also my game runs perfectly in Android beside the issue with your asset. Ok, I found what Proguard is and no I am not using Proguard when publishing, it is unchecked. I am using Gradle

 

The logs indicate that a NatCorder Android class is missing. This can only happen in two cases: either the NatCorder.aar Android library is missing (should be located in Assets > NatCorder > Plugins > Android) or it is being stripped during final compilation to an APK. Since you are building with Gradle as opposed to Unity's internal build system, there's a chance that Proguard is actually stripping the class. Try either building from Unity with its internal build system, or making sure that the Gradle project builds in debug mode as opposed to release.

 

I tried a build with "Internal" as well and the same happened. As I said I am 100% sure Proguard is not used.

NatCorder.aar is in the plugin folder so that is not it.

What would the difference be between building with Gradle in debug mode instead of release?

What about that fact that I am using Stripping Level: Use micro mscorlib, could that be the problem? Or the fact I am using .NET 4.x Equivalent in Scripting Runtime Version?

I just tested the game using Stripping Level: Disable, and it didn't fix anything, also I tested it with .NET 3.5 and nothing neither, So clearly there is a problem that has nothing to do with these settings.

I need help fixing this as soon as possible. My deadline is getting very close.

 

Can you upload these logs? I need to confirm that it is the exact same crash happening.

Release builds usually enable minification and stripping whereas debug builds don't.

Not at all. .NET settings do not affect this at all.

Email me as I'm much more responsive over email.

 

I will. Thank you.

 

Got it and responded!

 

When I pause the game, can I pause the Recording ?

 

How do you mean "pause the game"? If you mean suspending the app, then this is not supported. You must stop recording before the app is suspended; not doing so results in undefined behaviour (on iOS for example, recording will fail).

If you mean simply suspending recording while the app is running (like when a menu is opened), simply stop sending frames to NatCorder. If you are using a `CameraRecorder`, you can destroy it and then recreate it when you want to resume recording.

 

Hello guys,

Lanre, is a wonderful and very responsive developer! I had an issue and he worked with me over email to find the cause and resolve it.

In case that someone else find the same issue, the solution was to not select Vulkan in Android player's settings. the problem I had was that everything worked in the editor but as soon as I ran the game in Android the plugin was not being imported to the device. After several tests Lanre found that the problem was that I had selected Vulkan and OpenGLS3 in the player settings, as soon as I removed Vulkan, everything started working perfectly.

Regards,
Carlos

 

The Game is Paused using Time.timescale = 0
At that point of time I want to pause the recording

 

I am using the NatCorder.StartRecording() function for starting the recording and NatCorder.StopRecording() for stopping the recording

 

If you mean simply suspending recording while the app is running (like when a menu is opened), simply stop sending frames to NatCorder. If you are using a `CameraRecorder`, you can destroy it and then recreate it when you want to resume recording.[/QUOTE]
I tired destroying the CameraRecorder and then re-creating it and disabled the line for committing the frames. But in the final video, the video becomes still where I have paused the game

 

Hi Lanre, I've had two reports of crashes from the Play Store that look like they were caused by the app. Unfortunately I only have the stack traces:

Xiaomi Redmi 4X (santoni), Android 6.0
java.lang.Error: FATAL EXCEPTION [NatCorder Audio Encoding Thread] Unity version : 2018.2.9f1 Device model : Xiaomi Redmi 4X Device fingerprint: Xiaomi/santoni/santoni:6.0.1/MMB29M/V8.5.2.0.MAMMIED:user/release-keys Caused by
at android.media.MediaMuxer.nativeWriteSampleData (Native Method)
at android.media.MediaMuxer.writeSampleData (MediaMuxer.java:348)
at com.yusufolokoba.natcorder.mp4.MP4Recorder.writeSampleData (MP4Recorder.java:130)
at com.yusufolokoba.natcorder.mp4.Encoder.drainEncoder (Encoder.java:105)
at com.yusufolokoba.natcorder.mp4.Encoder.run (Encoder.java:125)
at java.lang.Thread.run (Thread.java:818)

Acer Chromebook R11 (C738T) (cyan_cheets), Android 7.1
java.lang.Error: FATAL EXCEPTION [NatCorder Video Encoding Thread] Unity version : 2018.2.9f1 Device model : google Acer Chromebook R11 (CB5-132T / C738T) Device fingerprint: google/cyan/cyan_cheets:7.1.1/R69-10895.56.0/5006116:user/release-keys
Caused by
at com.yusufolokoba.natcorder.mp4.Encoder.drainEncoder (Encoder.java:105)
at com.yusufolokoba.natcorder.mp4.Encoder.run (Encoder.java:131)
at java.lang.Thread.run (Thread.java:761)

 

@Lanre

This crash happens after I've finish recording with Natcorder. I have crashalytics on as well which is saying the source of the crash is coming from Natcorder.

Code (CSharp):

1. Caused by java.lang.IllegalStateException: Failed to stop the muxer
2.        at android.media.MediaMuxer.nativeStop(MediaMuxer.java)
3.        at android.media.MediaMuxer.stop(MediaMuxer.java:447)
4.        at com.yusufolokoba.natcorder.mp4.MP4Recorder.encoderFinished(MP4Recorder.java:103)
5.        at com.yusufolokoba.natcorder.mp4.Encoder.run(Encoder.java:127)
6.        at java.lang.Thread.run(Thread.java:764)

 

This approach will work except for one minor issue: timestamps. When you recreate a `CameraRecorder` to resume recording, it will use the current timestamps, and this timestamp will account for the time skip as a result of pausing the game. There isn't a clean way to handle this currently, but it makes an excellent new feature. Shoot me an email and I'll send you a build with a new API for handling this.

 

The first crash looks like one we are working to fix. I am not sure what's causing the second (I would need to see the full logs to know, but you likely don't have it). I'll see what I can find.

 

Does crashlytics show you the name of the thread that crashed? If so, is it the audio encoding thread?

 

@Lanre
It does show the thread, but it doesn't occur in the audio encoding thread, it occurs in the NatCorder Video Encoding Thread.

Code (CSharp):

1. android.os.BinderProxy.transactNative (Binder.java)
2. com.android.internal.os.RuntimeInit$KillApplicationHandler.uncaughtException (RuntimeInit.java:116)
3. com.crashlytics.android.core.CrashlyticsUncaughtExceptionHandler.uncaughtException (CrashlyticsUncaughtExceptionHandler.java:51)
4. com.unity3d.player.m.uncaughtException (Unknown Source:105)
5. java.lang.ThreadGroup.uncaughtException (ThreadGroup.java:1068)
6. java.lang.Thread.dispatchUncaughtException (Thread.java:1955)

 

Do you record with audio? Or just video?

 

I record with audio. This is the snippet of code I'm using to record.

Code (CSharp):

1.                 NatCorder.StartRecording(m_container, videoFormat, audioFormat, OnReplay);
2.                 m_videoRecorder = CameraRecorder.Create(Camera.main);
3.                 // If recording GIF, skip a few frames to give a real GIF look
4.                 if (m_container == Container.GIF)
5.                     m_videoRecorder.recordEveryNthFrame = 5;
6.                 // Start microphone and create audio recorder
7.                 AudioListener audioListener = Camera.main.GetComponent<AudioListener>();
8.                 if (m_recordMicrophoneAudio && audioListener != null && Microphone.devices.Length > 0) {
9.                     StartMicrophone();
10.                     m_audioRecorder = AudioRecorder.Create(audioListener, m_audioSource);
11.                 }
12.             }
13.         }
14.         private void StartMicrophone ()
15.         {
16.             #if !UNITY_WEBGL || UNITY_EDITOR // No `Microphone` API on WebGL :(
17.             // Create a microphone clip
18.             // int minFreq = 0;
19.             // int maxFreq = 0;
20.             // Microphone.GetDeviceCaps(null, out minFreq, out maxFreq);
21.             m_audioSource.clip = Microphone.Start(null, true, 60, 48000);
22.             while (Microphone.GetPosition(null) <= 0) ;
23.             // Play through audio source
24.             m_audioSource.timeSamples = Microphone.GetPosition(null);
25.             m_audioSource.loop = true;
26.             m_audioSource.Play();
27.             #endif
28.         }
29.         public void StopRecording()
30.         {
31.             if (NatCorder.IsRecording)
32.             {
33.                 // Stop playing mic audio
34.                 if (m_recordMicrophoneAudio) {
35.                     UnityEngine.Microphone.End(null);
36.                     m_audioSource.Stop();
37.                     m_audioRecorder.Dispose();
38.                 }
39.                 // Stop recording
40.                 m_videoRecorder.Dispose();
41.                 NatCorder.StopRecording();
42.             }          
43.         }

 

Are you able to get full logs from any of these crashes? Your code looks good. I haven't had any reports of this issue (stemming from the video encoding thread).

 

I have dropped a mail regarding the problem yesterday. Please Reply

 

How do I record multiple game cameras at the same time?
By registering CameraRecorder.Create() on each camera, you can see the flickering of the recorded image.

3D objects, UI, etc. are displayed separately for each camera.
I want to record everything on the screen together.

 

This is a bit more involved than creating multiple camera recorders. You have to composite the views of both cameras (which is what Unity does to render to screen). Because the process of compositing is very application-specific (split screen, depth, and so on), we don't include a recorder for multiple cameras. You have to implement it for your use case, but here is a starter.

 

Thanks.
I used this link to solve the problem.
However, in Graphics API Direct3D11, the screen is upside down and is recording, so you need to fix this problem.

 

The Windows backend flips the frame before sending it for encoding. If you are already correcting for this in your multi-camera recording shader, then you would need to modify NatCorder not to perform this inversion. Comment out these lines:

Code (CSharp):

1. var correctedFrame = AcquireFrame();
2. Graphics.Blit(frame, correctedFrame, transformMat);
3. RenderTexture.ReleaseTemporary(frame);

in Assets > NatCorder > Plugins > Managed > Platforms > NatCorderWindows.cs

 

Thank you for your reply.
I solved it by modifying the uv of the "Transform" Shader.
----------------------------------------------------------------------------
I use NatShare together.
With NatShare.SaveToCameraRoll (path), the video is saved via NatCorder and saved again via SaveToCameraRoll.
I do not want to double my storage capacity when video is saved.
Use SaveToCameraRoll to move the path, or I want to be able to access it.
Also, if you share files through Application.persistentDataPath(/Documents) in iOS, app store reviews may be rejected.


ps. Understand that my English is not good.

 

Hello,

Does this Asset work with ARKit and ARCore?

I am asking because I want to Record The Unity Scene that Includes the Augmented 3D Models with animations

another Question :
can i record the unity scene without the GUI Buttons and canvas elements?

please i want a clear answer for this because i didn't find anything in the comments aout augmanted reality

thanks in advance

 

You can delete the video in the app's documents directory after saving it to the camera roll. Simply use File.Delete with the video path.

 

Yup, it does.

Yup, simply set the canvas mode of your UI canvas to Overlay. Overlay canvases are 'invisible' to the rendering system, so NatCorder can't record it.

 

Oh! If you delete, you will not be able to use the sharing feature.
I don't know the path saved as "NatShare.SaveToCameraRoll"...

Code (CSharp):

1. NatShare.SaveToCameraRoll(path);
2. NatShare.ShareMedia(path);
3. File.Delete(path);

 

what about the objects inside the scene?

what about the objects inside the scene?
does the recording session include the augmented objects (such as cars, birds)?

 

This discussion should be on the NatShare thread. But you can share first, and with the new share callbacks added in NatShare 1.1f2, you can add a callback that will then save to the camera roll and delete the video.

 

Typically, you'll be recording a game camera with NatCorder. In this case, NatCorder records what the camera sees, which includes objects in the scene.

 

My videos are very dark on Android devices, I've figured out the problem is with our app's use of Linear Colorspace. Fortunately the videos work great on iOS, but now I have a complete app in Linear that I need to get working on Android. I realize Natcorder does not officially support Linear, but is there any way to make the video brighter without redoing all our assets for Gamma?

 

You can write a custom shader that does gamma correction, and apply the shader to your `CameraRecorder` using `recordingMaterial` property.

 

Hi,
Thanks for the awesome plugin @Lanre ! Im really enjoying using it.

Just a question about having multiple instances of recorders.

I have gameplay that I'm recording, then the player can pause, and record something from within this mode, then return to regular game play recording. Natcorder is setup as a singleton, is it capable of handling multiple recordings?

The flow would be like this:

- Start gameplay recording
- pause gameplay recording, go into pause mode

- record another thing happening and save out the clip while in a special "pause mode"

- return to gameplay, resume prior recording
- save out main gameplay clip

Do you have any advice?

Thanks!

Eric

 

Hey Eric. This is not possible. NatCorder can only record one video at any given point in time. When recording is 'paused', NatCorder is technically recording; you just aren't sending frames to be encoded.

 

NatCorder and NatMic: A Crash Course.

Please make sure to read this if you are using our NatCorder with our NatMic API for recording microphone audio.