NatCorder - Video Recording API

Can you be more descriptive about how it doesn't respond? Does StartRecording not work? Please email me the logs from logcat.

 

Can you email me the full log file (from app start). I need more information. Also, make sure that NatCorder's verbose mode is enabled when you take the logs.

 

Hi Alex. I responded to your email. Please send me some more info so that I can look into it. NatCorder has been tested and works out of the box with Vuforia. I have not tested it with EasyAR.

 

This is a good observation. Usually, IllegalStateExceptions in NatCorder mean that the encoder was not able to start, either because it does not support the video size or some similar setting. Also, most devices cannot handle more than 1920x1080 so that is something to watch out for.

 

There's crash when I call 'StopRecording' after 'BeginRecording' very shortly.

Caused by java.lang.NullPointerException: Attempt to invoke virtual method 'void com.yusufolokoba.natcorder.Recorder.writeSampleData(int, java.nio.ByteBuffer, android.media.MediaCodec$BufferInfo)' on a null object reference
at com.yusufolokoba.natcorder.Encoder.drainEncoder(Encoder.java:77)
at com.yusufolokoba.natcorder.VideoEncoder.onRender(VideoEncoder.java:128)
at com.yusufolokoba.natcam.rendering.GLRenderContext.run(GLRenderContext.java:153)
at java.lang.Thread.run(Thread.java:762)

 

How short do you call StopRecording? This is not a safe operation (given that the encoder takes some time to start, and cannot be interrupted during this process). I will look into this.

 

Maybe less than 1 sec.

I know this is not a normal situation,
but, StopRecording is performed by user. So I think this should be safe always.

Thanks.

 

This is now working for me with Easy AR but... It is still crashing after I do a few recordings. I can get four out of it before the app quits. This is on a Samsung S7 Edge.

I am also using Playmaker to call stop after 10 seconds of recording and then making it wait for another 10 seconds before allowing it to call up the share window.

What am I doing wrong? (because I am 100% sure it is user error)

Code (CSharp):

1. /*
2. *   NatCorder
3. *   Copyright (c) 2017 Yusuf Olokoba
4. */
5.  
6. namespace NatCorderU.Examples {
7.  
8.     using UnityEngine;
9.     using UnityEngine.UI;
10.     using System.Collections;
11.     using Core;
12.     using Extensions;
13.  
14.     public class ReplayCam2 : MonoBehaviour {
15.  
16.         /**
17.         * ReplayCam Example
18.         * -----------------
19.         * This example records the screen using the high-level `Replay` API
20.         * We simply call `Replay.StartRecording` to start recording, and `Replay.StopRecording` to stop recording
21.         * When we want mic audio, we play the mic to an AudioSource and pass the audio source to `Replay.StartRecording`
22.         * -----------------
23.         * Note that UI canvases in Overlay mode cannot be recorded, so we use a different mode (this is a Unity issue)
24.         */
25.  
26.        
27.         public string pathToReplay;
28.        
29.         [Header("UI")]
30.         public RawImage preview;
31.         public AspectRatioFitter aspectFitter;
32.  
33.         [Header("Recording")]
34.         public bool recordMicrophoneAudio;
35.         public AudioSource audioSource;
36.  
37.    
38.  
39.         public void StartRecording () {
40.             if (recordMicrophoneAudio) {
41.                 // Start the microphone
42.                 audioSource.clip = Microphone.Start(null, true, 15, 44100);
43.                 audioSource.Play();
44.                 // Start recording with microphone audio
45.                 Replay.StartRecording(Camera.main, Configuration.Default, OnReplay, audioSource, true);
46.             }
47.             // Start recording without microphone audio
48.             else Replay.StartRecording(Camera.main, Configuration.Default, OnReplay);
49.         }
50.  
51.         public void StopRecording () {
52.             // Stop the microphone
53.             if (recordMicrophoneAudio) Microphone.End(null);
54.             // Stop recording
55.             Replay.StopRecording();
56.         }
57.  
58.         void OnReplay (string path) {
59.             Sharing.GetThumbnail(path, OnThumbnail, 2f);
60.             this.pathToReplay = path;
61.             Debug.Log("Saved recording to: "+path);
62.             #if UNITY_IOS || UNITY_ANDROID
63.             // Playback the video
64.             Handheld.PlayFullScreenMovie(path);
65.             #endif
66.         }
67.         void OnThumbnail (Texture2D thumbnail) {
68.            
69.         }
70.        
71.         public void OnShare () {
72.             Sharing.Share(this.pathToReplay);
73. }
74.  
75.     }
76. }

 

I am considering just saving to the camera roll.

 

Hi,
Just curious, for ios, is metal a requirement? Im getting a few errors when recording. Im not using metal for this project by the way because of some other issues.
Thanks!

 

Your code looks good. Can you email me the full logs from logcat? Make sure that NatCorder.Verbose is set to true when you build.

 

Metal isn't a requirement; but we had faced this issue and fixes it. I'm just not sure if we published the fixed onto the Asset Store yet. Can you post a screenshot of the call stack? It is just to the left of what you took a screenshot of. Thanks.

 

Hi @Lanre, is this what you mean?

 

Will do, Thank you for your ongoing help and support.

 

Hey, Lanre.

I have to implement microphone recording with Vuforia till next month. (My boss said "ASAP, huh?")
But my video never recorded correctly with microphone, and video framerate is too low in iOS device.
Therefore, Can I ask about plan to release new version or commit detailed documentation?

 

The first one is what I was referring to. Shoot me an email with your invoice number and I will send you my build of the upcoming version. We made a fix for this issue a while ago.

 

Regarding the microphone issue, we have decided to implement our own native realtime Microphone API, so that we don't have to deal with the issues of Unity's API. I expect to complete implementing it later today and will start testing tomorrow. This issue is the only one left before we release version 1.1. Regarding your video framerate, I need more information to figure out why it is low. Email me so that I send instructions on how to profile.

 

Hey Guys, I have a simple question. I noticed the NatCorder.StartRecording function takes a callback that returns a default path where the video was saved to as well as a default filename.
How do I set the path myself? and can I choose the naming convention for the files?

 

NatCorder does not expose a way to do this. Instead, you can use the System.IO API's to copy the video to your target directory or rename the file.

 

Hey @wesense please send me an email. I would like to send you the fix we have implemented. Thank you!

 

I have added a fix for this in the upcoming version. Thanks for bringing it to my attention.

 

Could I also get the fix too if I send you an email?

 

Code (CSharp):

1. namespace NatCorderU.Examples {
2.  
3. using UnityEngine;
4. using UnityEngine.UI;
5. using System.Collections;
6. using Core;
7. using Extensions;
8.  
9.  
10. public class Marshmellows : MonoBehaviour {
11.  
12.     public string pathToReplay;
13.  
14.     // Called when recording button is pressed
15.     public void StartRec () {
16.         // Start recording a replay from the game camera
17.         Replay.StartRecording(Camera.main, Configuration.Default, ReRec);
18.     }
19.      
20.     // Called when stop button is pressed
21.     public void StopRec () {
22.         // Stop recording // NatCorder will finalize recording and invoke the replay callback
23.         Replay.StopRecording();
24.     }
25.  
26.     // Called by NatCorder when recording has been completed
27.     void ReRec (string path) {
28.         this.pathToReplay = path;
29.             Debug.Log("Saved recording to: "+path);
30.         // Show the replay to the user
31.         Handheld.PlayFullScreenMovie(path);
32.     }
33.    // Called when share button is pressed
34.     public void OnShare () {
35.     Sharing.Share(this.pathToReplay);
36.  
37. }
38. }
39. }

I tried making a really, really stripped back script based on the medium article. It works but I am getting a crash after a few uses on a simple scene with nothing but three buttons (Record, Stop, Share) and a particle effect.

 

Is there a way this plugin can tell me the length in time of the video after its been recorded?

 

Sure. I responded to your email. Your code above looks good.

 

This is not a feature. It is largely controlled by the client (since the client provides timestamps for video and audio frames), so you should simply save the time between your calls to StartRecording and StopRecording.

 

Hey Lanre,

Your plugin is amazing. Great work!

I'm having an issue that I'm hoping you can help with,

Importing the plugin into an empty project and running the ReplayCam scene crashes on my Pixel 2 device. Log is attached. Seems to work fine when I comment out the microphone code in ReplayCam.Start(). GrayWorld scene seems to work fine.

Are you able to offer any support? We'd be happy to test out anything you have in the works.

Best,
Sunny

 

Hi Sunny. Thanks for the compliment! Regarding this crash, we are currently working on it. I'll inform you once we have a fix. In the mean time, can you email me so that I can reach you directly. Thanks.

 

Is there a way to trim or slice a video after its been created?

 

NatCorder doesn't support this functionality.

 

Hi, Im attempting to reduce having previous recordings collect and take up memory. So I wrote this small file clean up script. It seems to work fine in the editor, but doesn't work on iOS. When I call Share on the new path, Im getting some file not found errors. What are other people doing to clean up recording files on iOS?

Thanks!

public string getRecordingFilepath(string path)

{

// new file

string basePath = Path.GetDirectoryName(path);

string extension = Path.GetExtension(path);

string newPath = Path.Combine(basePath, ("recording" + extension));

return newPath;

}

public IEnumerator cleanRecordings(string path)

{

// wait for file to be written

float timeout = Time.realtimeSinceStartup + 5f;

while (!File.Exists(path) && Time.realtimeSinceStartup < timeout)

{

yield return null;

}

// get new path

string newPath = getRecordingFilepath(path);

File.Copy(path, newPath, true);

File.Delete(path);

yield return null;

}

 

There are a few things to take note of:
- You don't need to wait for File.Exists. When the VideoCallback is invoked by NatCorder, the video is guaranteed to be present at the path returned
- On iOS, the Sharing API expects that the path be prepended by the 'file://' protocol, which the System.IO API's do not respect. So you might have to add it back before using any of the Sharing API's. Note how the path returned by NatCorder always has the prefix.

 

In the mean time, here is how we hack through the microphone stutter problem:

            

if (recordMicrophoneAudio)

{

   // Start the microphone

   audioSource.clip = Microphone.Start(null, true, 60, 44100);

   while (!(Microphone.GetPosition(null) > 0))

   {

   }

   audioSource.Play();

   // Start recording with microphone audio

   Replay.StartRecording(recordingCamera, Configuration.Default, (arg) => callback(arg), audioSource, true);

}

 

Hi!

I'm trying to get the thumbnail of a movie I just recorded:

Code (CSharp):

1.     private void OnReplay(string path) {
2.       Debug.Log("recording done, locally saved at " + path);
3.       _lastPath = path;
4.       Sharing.GetThumbnail(_lastPath, SetThumbnailBackground);
5.     }

This works fine on Android, but on iOS I get the following error:

"Error: Error Domain=NSCocoaErrorDomain Code=257 "The file couldn't be opened because you don't have permission to view it." "

I can save it to the camera roll just fine, so the file is there, and have enabled all the permissions for that. What am I missing?

-Wijnand

 

Thanks for bringing this to my attention. We are testing it now.

 

It turns out that we have actually fixed this issue in the upcoming release. In the meantime, you should be able to use this workaround:

Code (CSharp):

1. Sharing.GetThumbnail(_lastPath.Replace("file://", string.Empty), SetThumbnailBackground);

 

Hi @Lanre I'd love beta access! I sent you a PM.

 

Hi there. I responded to your PM.

 

Hello!

I'm trying to get the video I just recorded to appear in the gallery app on Android. I can see the video in the InternalStorage/Android/data/my.app.name/files folder, so I know the video is being saved correctly. I just need that same video to show up in the gallery app. I'm using the following code to try and do so:

Code (CSharp):

1. NatCorderU.Core.Platforms.NatCorderAndroid lSharingAndroid = new NatCorderU.Core.Platforms.NatCorderAndroid();
2.         lSharingAndroid.SaveToCameraRoll( recordingPath );

The NatCorderiOS version of this same code works just fine on iOS. I'm not getting any errors or logs that indicate what could be happening. I've tested this on an LG G6 running Android version 7.0. I'm using Unity version 2017.3.1f1 and I just bought NatCorder yesterday so I assume I'm using the latest version of the plugin. Any help would be greatly appreciated!

 

Hi there. You should not instantiate implementations of the ISharing interface. Instead, use the Sharing static class (it will maintain the implementation state). I have reproduced this issue and will add a fix to the upcoming update.

 

Thanks for the help!

 

Hello Lanre,

Apparently, many bugs in your library were fixed on your side during the last few weeks but I can't see any updated package on the asset store. I think it's important for developers to rely on a stable build of your awesome plugin instead of a bunch of small fixes sent in private.

Could you please update the package asap on the Asset Store so I can safely update your plugin?

Thank you very much for your hard work.

 

Hi Thomas. We wanted to wait so that we could batch as many updates into one. But I am submitting the new version this afternoon.

 

Awesome, thank you very much.

 

Hey Lanre,

I bought the plugin, thank you. I also have a couple of problems with the Android version when publishing to a Pixel 1.
- When calling Replay.StartRecording with an AudioSource or AudioListener the application freezes and crashes, I can't see anything in the logs which show the crash.
- The video is created within the application directory but calling Sharing.SaveToCameraRoll(path) does nothing.

Looking through the comments here it seems these may be bugs you've fixed in the new version, if it will take a while for Unity to approve to the store can I mail you the invoice number and get a new version when you've got it ready to go.

Thank you

 

Hi Lanre,

There is a plugin conflict when using both NatCam and NatCorder in the same Unity project, as both these libraries use the NatCamRenderDispatch and NatCamRenderPipeline.

For Android, there is a conflict with:

• NatCamRenderPipeline.aar

For iOS, there is a conflict with:

• NatCamRenderDispatch.h
• NatCamRenderDispatch.mm
• libNatCamRenderPipeline.a

I suggest restructuring the asset folders for these common libraries between NatCam and NatCorder.

 

I found that decreasing video resolution can help. Ex.
Replay.StartRecording(Camera.main,new Configuration(200,200,30), OnReplay);

 

We have fixed this in the upcoming update. It is caused by the audio thread calling into the JVM (invoking Java methods) without being attached prior.

This is also a bug in the current version that we have fixed.

I expect Unity to approve it quickly. I'll keep you posted.

 

This is a good catch. Both API's use the exact same version of the NatCam Render Pipeline. So you simply have to remove one set of duplicates. I will add this note to the readme.

We need to keep the plugins self-contained, which is why we never have more than one top-level directory for these plugins.

 

With the next update, this shouldn't be necessary. We have made significant optimizations to the encoding loop, slicing frame time down by over 20ms.