NatCorder - Video Recording API

Hi,

We recording on iPhone with custom Replay for Pause/Resuming. It seem it record all frame continuously and play smooth for all frame we want to record. But the total time length of the video including the pausing time, and the rest of video for extra time is the last frame we record.

Please fix the bug

Best regards

 

Hi.
Recently, i bought your plugin and it get along with my project quiet good. But I encounter 2 critical problems.
+ All recorded video with audio always have the same audio with the first video which i recorded.
+ When I build for iOS, XCode show this strange message:
Undefined symbols for architecture arm64:
"__SaveImage", referenced from:
_CallNative__SaveImage_m3148203869 in Bulk_CaptureAndSaveLib_0.o
_CallNative_SaveImageToCameraRoll_m734463760 in Bulk_CaptureAndSaveLib_0.o
(maybe you meant: _CallNative__SaveImage_m3148203869)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I think this error cause by your libSharing.a
By the way, my game will be alpha test in a few days. Could you provide me quick solutions ?
Best regard.

 

We don't impose any restriction at all on recording size. You can start recording with a 4K resolution. But you have to be careful because 4K is not easy for a lot of devices; the memory requirements are pretty big.

 

Can you share your code?

 

This is an easy fix. Simply open Microphone.cs in NatCorder > Plugins > Managed > Microphone and at the top of the file, comment out the PERPETUAL_CLIP definition.

NatCorder does not export these symbols; it looks like another plugin that you must be using. The error shows Bulk_CaptureAndSaveLib_0.o.

 

Hi,

So far I'm really happy with the release, great asset!
But I was wondering if there is a way to handle any errors that might come up while recording?
I couldnt find anything in the documentation about it, but what happens if, for example, the device on which you record is out of storage space? I would like to give the user some feedback if this is the case and the recording isn't succesful.

-Wijnand

 

I'm happy to hear this!

This is practically impossible, which is why we didn't add an error handler. Recorded videos are typically less than 30MB in size due to the H.264 codec's compression properties. If a device has less than 30MB storage left, chances are that your app would not even be able to be installed in the first place. Such little amounts of storage space left are impossible to have these days.

 

Hi,

The custom Replay just similar with your replay code, the different is in video recorder component, disable OnRenderImage, and in my OnRenderImage send RenderTexture (larger than screen resolution) to commit with timestamp

Now we update the timestamp to match with video framerate (also match to Time.CaptureFrameRate = 30.0f for replay recording)
//videoCustomRecorder.timestamp += Time.unscaledDeltaTime;
videoCustomRecorder.timestamp += (1.0f / 30.0f);

The bug is in audio recorder component, your code continue send audio samples to record even in pause state, so the total video length include pausing time.

Now we disable recording audio to record video only

please fix that bug

Best regards

 

This is not a bug in our implementation. You are modifying our sources, so you have to properly handle all the nuances of our implementation. Check the source of the AudioRecorder component. It checks for the `Replay.IsPaused` flag before incrementing the timestamp and committing audio samples. You have to change this to the flag you are using in your custom Replay script.

Consider showing me your code so I can tell you what is good and what could lead to undefined behaviour. I very strongly recommend against modifying our code.

 

Thank you for replying,

After checking the code again, my custom recorder recording from AudioListener from Camera, and Your replaycam record from audio source, so if call pausing on audio source, it will not feed the data while pausing and that code run correctly.

Would you like to provide example to record video and audio from audiolistener attached in Camera with supporting pause & resume ? while pausing recording, the sound continue playing.

Best regards,

 

I'm using ARKit to create an experience using augmented reality and we bought your asset.
I just saw that you have a fix to make it work with other renderer plugins.

When we implemented your natcorder in the projects the crashes started to appear.

We did, the copy paste thing with extern "c" void from arkit, but we don't use pretty well xcode or objective c so, nothing worked.

Can you help us with this?

Anyone has the proper setup to make this work with arkit ?

 

Hello,
Do the package support Metal backend on iOS/Mac ? . Running with metal, it seem memory leak.

Best regards

 

Hi everyone,

I am having an issue with permissions in my Unity ARCore Android app using NatCorder. My app does not prompt for Storage and Microphone permissions, even though they are including in my android manifest:

Code (CSharp):

1.   <uses-permission android:name="android.permission.RECORD_AUDIO" />
2.   <uses-feature android:name="android.hardware.microphone" android:required="false" />
3.   <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

The app correctly asks for Camera permission. As I understand the ARCore unity package takes care of that.

I think I am missing some sort of "checking / asking for the permission" elsewhere in my code, but I have not found a way to do that in my Unity C# scripts.

Any help is GREATLY appreciated!

 

Our audio recorder script already does this. When paused, it halts its timestamp and does not forward audio samples to NatCorder for encoding.

 

No setup is required for using ARKit (we made this improvement in the current version). Can you email me details about the crash you are facing? Specifically, send me the logs and a screenshot of Xcode when the crash occurs (I would like to see the call stack).

 

Yes, NatCorder supports Metal. We have made improvements to the iOS backend that we will release next week. We have made recording much more memory efficient and performant on iOS, just like we did to Android in the current release.

 

This page might be useful. NatCorder only requests the WRITE_EXTERNAL_STORAGE permission, not the other two.

 

I saw that page. I made sure that I am NOT skipping the permissions dialogue, yet my app does not ask for WRITE_EXTERNAL_STORAGE permission. Hmm...

 

Not sure what could be going on, given that. Report it to Unity.

 

Will do!

 

On iOS recording video from the microphone causes the game audio to play through the phone's earpiece.

Is there anyway to select the microphone source (only one seems listed in iOS) or disable the speaker change in the source?

 

Hi Lanre
I'm using replay api to simply record a movie from my Vuforia App on Samsung S6.
When I press the "rec" button I have a small lag (a couple of seconds) in which Unity freeze (only on mobile, not in Editor) before to start to record.
Is there some optimization I can use to try to avoid or minimize this lag?
Thank-you!

 

This is not controlled by NatCorder. If you are using the ReplayCam example, you can see that we use the UnityEngine.Microphone API. Consider contacting Unity about this issue.

 

If you are recording using microphone audio, then this is probably where the lag comes from (we have to wait until the microphone has started). Apart from this, some setup is required to start recording which is why there might be a slight hiccup.

 

Hi, iam try to use NatCorder with Playmaker actions, to record my app, and its works, but its save the video file into Android/data/com.-apsname-./files directory. Always. No matter with path var i am using. And so its can not share this file. When im tying to send it VIA email = its brings me read error. Can you help me pls/ I am not so good in scripting.

 

NatCorder does not expose a way to set the save directory. It will always save to the app's documents directory. Can you show your sharing code? NatCorder's Sharing API works when sharing emails too. Do you have an error log?

 

Hi,
We tried combining NatCam and NatCorder together, which worked fine until the latest version of NatCam.
In an empty project, we first imported NatCorder and then NatCam, which gave a conflict between two duplicate libraries in the two packages. When we removed the libraries from the NatCorder package, we got the NatCam sample to build, but it crashed on iOS.
First adding NatCam and then NatCorder didn't work either, we couldn't get it to build because there were missing namespaces in several classes.

Error:

Spoiler

NatCam: Initialized NatCam 1.6 iOS backend

NatCamU.Core.NatCam:.cctor()

NatCamU.Core.NatCamBehaviour:Start()

NatCamU.Core.Examples.MiniCam:Start()



(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

2018-04-09 16:37:08.572445+0200 NatcamTest[509:173957] +[Utilities formatWithPreviewResolution:width:height:]: unrecognized selector sent to class 0x1038747e0

2018-04-09 16:37:08.610439+0200 NatcamTest[509:173957] Uncaught exception: NSInvalidArgumentException: +[Utilities formatWithPreviewResolution:width:height:]: unrecognized selector sent to class 0x1038747e0

(

    0   CoreFoundation                      0x000000018566eda4 <redacted> + 252

    1   libobjc.A.dylib                     0x00000001848285ec objc_exception_throw + 56

    2   CoreFoundation                      0x000000018567bfd8 <redacted> + 0

    3   CoreFoundation                      0x00000001856745c8 <redacted> + 1380

    4   CoreFoundation                      0x000000018555a41c _CF_forwarding_prep_0 + 92

    5   NatcamTest                          0x00000001035c8a40 +[NatCam setPreviewResolution:width:height:] + 84

    6   NatcamTest                          0x0000000102c418c8 NatCamBehaviour_Start_m2456301555 + 368

    7   NatcamTest                          0x0000000102c41748 MiniCam_Start_m1694645034 + 20

    8   NatcamTest                          0x0000000102feb188 _Z31RuntimeInvoker_Void_t1185182177PFvvEPK10MethodInfoPvPS4_ + 20

    9   NatcamTest                          0x00000001035bad6c _ZN6il2cpp2vm7Runtime6InvokeEPK10MethodInfoPvPS5_PP15Il2CppException + 68

    10  NatcamTest                          0x00000001032c030c _Z23scripting_method_invoke18ScriptingMethodPtr18ScriptingObjectPtrR18ScriptingArgumentsP21ScriptingExceptionPtrb + 100

    11  NatcamTest                          0x00000001032c7004 _ZN19ScriptingInvocation6InvokeEP21ScriptingExceptionPtrb + 60

    12  NatcamTest                          0x00000001032cc688 _ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtrP21ScriptingExceptionPtr + 1180

    13  NatcamTest                          0x00000001032cc884 _ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtr + 84

    14  NatcamTest                          0x00000001032cbad4 _ZN13MonoBehaviour16DelayedStartCallEP6ObjectPv + 80

    15  NatcamTest                          0x0000000103180f64 _ZN18DelayedCallManager6UpdateEi + 636

    16  NatcamTest                          0x00000001032238a8 _Z10PlayerLoopv + 452

    17  NatcamTest                          0x0000000103086c9c _ZL19UnityPlayerLoopImplb + 36

    18  NatcamTest                          0x0000000102c2ab08 UnityRepaint + 140

    19  NatcamTest                          0x0000000102c2a9f4 -[UnityAppController(Rendering) repaintDisplayLink] + 88

    20  GPUToolsCore                        0x000000010522c5f0 -[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 176

    21  QuartzCore                          0x00000001896ce3c0 <redacted> + 832

    22  IOKit                               0x00000001858d78a0 IODispatchCalloutFromCFMessage + 492

    23  CoreFoundation                      0x00000001855fcb20 <redacted> + 188

    24  CoreFoundation                      0x0000000185617ae8 <redacted> + 56

    25  CoreFoundation                      0x0000000185617230 <redacted> + 440

    26  CoreFoundation                      0x0000000185614c80 <redacted> + 2456

    27  CoreFoundation                      0x0000000185534da8 CFRunLoopRunSpecific + 552

    28  GraphicsServices                    0x0000000187517020 GSEventRunModal + 100

    29  UIKit                               0x000000018f51578c UIApplicationMain + 236

    30  NatcamTest                          0x0000000102c23b00 main + 160

    31  libdyld.dylib                       0x0000000184fc5fc0 <redacted> + 4

)

2018-04-09 16:37:08.611568+0200 NatcamTest[509:173957] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[Utilities formatWithPreviewResolution:width:height:]: unrecognized selector sent to class 0x1038747e0'

*** First throw call stack:

(0x18566ed8c 0x1848285ec 0x18567bfd8 0x1856745c8 0x18555a41c 0x1035c8a40 0x102c418c8 0x102c41748 0x102feb188 0x1035bad6c 0x1032c030c 0x1032c7004 0x1032cc688 0x1032cc884 0x1032cbad4 0x103180f64 0x1032238a8 0x103086c9c 0x102c2ab08 0x102c2a9f4 0x10522c5f0 0x1896ce3c0 0x1858d78a0 0x1855fcb20 0x185617ae8 0x185617230 0x185614c80 0x185534da8 0x187517020 0x18f51578c 0x102c23b00 0x184fc5fc0)

libc++abi.dylib: terminating with uncaught exception of type NSException

 

Hi, Iam Using this scripts
https://github.com/dumbgamedev/natCorder
And set the recording path as / sdcard. But when iam not set this var, video is not rocording

 

This is coming from NatCam. Delete it and reimport it into your project.

 

You are not supposed to set `recordingPath`. It is set by the Playmaker action once recording is completed (and then you can use it).

 

Even if i leave recPath var. empty - still recording to the Android/data/com.-apsname-./files directory .. i think its me.... but i dont know what to do with this.

 

As I mentioned, you are not supposed to set the path. NatCorder does not use the path at all. NatCorder will record the video and the Playmaker bindings will set `recordingPath` to the path that NatCorder used.

 

But if i didnt set the rec path var - its brings me that error, i feels so stupd))))

 

So iam using one recpath var.to use it for sharing and videp playback

 

I'm not too sure about this issue. I will reach out to the developer who created the Playmaker bindings. For now you can simply set the recording path value to something random so you don't get an error. Ideally, the Playmaker bindings should expose a callback for when the recording is completed.

I strongly recommend writing C# code because it offers much greater flexibility with how you can use NatCorder.

 

Hello!

I have purchased your NatCorder asset, and I got it to work very well with my project!

I have one observation and question:
When I record more than one video (with audio), then all recordings after the first recording save the audio from the first recording, and do not save new audio.

For example, the first recording saves video1 and audio1, but the second recording saves video2 and audio1.
I can't tell if it doesn't record any audio the second time, or if it records it but continues to save the first audio clip or what.

What would you recommend as the best way to solve this?


Thank you!

 

It's actually an easy fix. I presume you are using the ReplayCam example (or a custom implememtation thereof). In the `StartRecording` function, immediately below the call to `Extensions.Microphone.StartRecording`, add this:

Code (CSharp):

1. audioSource.clip = Extensions.Microphone.StartRecording();
2. audioSource.timeSamples = UnityEngine.Microphone.GetPosition(null); // Add this

 

Hi!
Recording a video, both on Editor and on Android, I've noticed a decrease of performances of the app. Looking at the profiler (in Editor) it seem to me that during recording Natcorder take a lot of cpu..
This is the profiler


And this is the script I'm using

Code (CSharp):

1.  
2. IEnumerator Start() {
3.         initColor = recordButton.color;
4.         // Request microphone
5.         yield return Application.RequestUserAuthorization(UserAuthorization.Microphone);
6.         if (!Application.HasUserAuthorization(UserAuthorization.Microphone)) yield break;
7.  
8.         setButton.setState(shareButton, false);
9.     }
10.  
11.  
12.     public void StartRecording()
13.     {
14.         if (!isRecording)
15.         {
16.             isRecording = true;
17.             recordButton.color = Color.red;
18.             const float DownscaleFactor = 1;
19.             var configuration = new Configuration((int)(Screen.width * DownscaleFactor), (int)(Screen.height * DownscaleFactor));
20.             // Start the microphone
21.             audioSource.clip = NatCorderU.Extensions.Microphone.StartRecording();
22.             audioSource.loop = true;
23.             audioSource.Play();
24.             // Start recording with microphone audio
25.             Replay.StartRecording(cam, configuration, OnReplay, audioSource, true);
26.         }
27.         else
28.         {
29.             isRecording = false;
30.             StopRecording();
31.         }
32.     }
33.  
34.  
35.     public void StopRecording()
36.     {
37.         recordButton.color = initColor;
38.         audioSource.Stop();
39.         NatCorderU.Extensions.Microphone.StopRecording();
40.         Replay.StopRecording();
41.         setButton.setState(shareButton, true);
42.     }
43.  
44.  
45.     void OnReplay(string path)
46.     {
47.         Debug.Log("Saved recording to: " + path);
48.         // Get the video thumbnail
49.         Sharing.GetThumbnail(path, OnThumbnail, 5f); // Get the thumbnail after 5 seconds of video
50.                                                      // Save the path to the replay so we can use it in `OnShare`
51.         this.pathToReplay = path;
52.     }
53.  
54.     // Invoked by NatCorder once thumbnail has been loaded
55.     void OnThumbnail(Texture2D thumbnail)
56.     {
57.         // Do stuff with the thumbnail
58.         // ...
59.     }
60.  
61.     // Invoked by UI when user clicks share button
62.     public void OnShare()
63.     {
64.         // Share the replay using the native sharing UI
65.         Sharing.Share(this.pathToReplay);
66.     }

Maybe I'm doing something wrong?

 

This is different on Android and in the Editor. Here's an explanation of what's going on:

I'll get Android out of the way first. If you are seeing a performance drop while recording, then make sure that you are using multithreaded rendering in your app. This greatly improves recording performance.

Now, for the actual behaviour you are observing in the Editor, this is behaviour that affects the macOS, Windows, and WebGL backends. On these backends, NatCorder must provide raw pixel data to the encoders (as opposed to iOS and Android, where there are 'fast paths' that allow us to provide GPU textures). The only way to do this by issuing a Texture2D.ReadPixels call, which introduces a lot of processing time to the pipeline. This is what is reflected in the Profiler.

 

Thank's!
One more question, as far as I can understand there's no way to specify a recording path, isn't it (for example to save into Gallery, with some plugin), isn't it?

 

No, you cannot set the recording path. If you want to save to the gallery, use the Sharing.SaveToCameraRoll function (in the NatCorderU.Extensions namespace).

 

Hi all! We just published a new tutorial:

NatCorder Tutorials: Audio Workflows.

Check it out!

 

I tried that solution. I still get a conflict with the libraries "libNatCamRenderPipeline.a", "NatCamRenderDispatch.h" and "NatCamRenderDispatch.m", since they're included in both the NatCam and the Natcorder package. I tried deleting the libraries from either package, but it still results in the error I posted above (on iOS only). I'm using the latest NatCam Core and NatCorder. This problem didnt exist with a previous version of NatCorder (which we reverted to now, for the time being, we're using NatCam 1.6f1 and NatCorder 1.0f1).

 

Hi. Now i trying to setup record without playmaker plugin and that what ive done.
If i remove sharing code - everything works, What iam doing wrong?

 

Hi.
I faced a similar issue, I observe one sec lag when start recording.
And also I noticed this happens only the first time for each launch.
After second recording, it starts recording smoothly.
I guess some components need to get prepared for first time and after that it can recycle already prepared component. This happens only when I use microphone audio so I think this is the bottleneck.
Can I pre-warm microphone audio at the launch moment instead of first recording?
I'm using ReplayCam example.
iOS 11.3

 

Can you send me an email? The crash you posted here has nothing to do with NatCorder. The files that you need to delete when using both NatCam and NatCorder are listed in the README.md.

 

Oh this is simple. The `Sharing` API is in the `NatCorderU.Extensions` namespace, so at the top of your script, add `using NatCorderU.Extensions`.

 

This is certainly related to the microphone then. The first time that Extensions.Microphone.StartRecording is called, the function will sit and wait till the microphone has started playing. Subsequently, you will not notice any lag because the microphone is never stopped.

 

Apparently Unity has been aware of this issue for years but has elected to do nothing.

I was able to find a quick fix on GitHub: https://github.com/cbaltzer/UnitySpeakerFix

And changed the start recording function to this:

Code (CSharp):

1. public static AudioClip StartRecording (string deviceName = null) {
2.             #if !UNITY_WEBGL || UNITY_EDITOR // No `Microphone` API on WebGL :(
3.             clip = clip ?? UnityEngine.Microphone.Start(deviceName, true, 60, 48000);
4.             #endif
5.  
6.             #if UNITY_IOS
7.             iPhoneSpeaker.ForceToSpeaker();
8.             #endif
9.  
10.             #if !UNITY_WEBGL || UNITY_EDITOR // No `Microphone` API on WebGL :(
11.             while (UnityEngine.Microphone.GetPosition(deviceName) <= 0) ;
12.             #endif
13.             return clip;
14.         }

If it is possible to include this in your plugin I think it would be a great addition. Other video plugins on the asset store are capable of recording audio on iOS without dipping the audio. But none of them have the performance, documentation, and stability your plugin has. Honestly I think this is the best written and documented plugin I've used. 5 star review coming soon.

 

This is a wonderful find. They don't have a license included so we can't include it with NatCorder, but I have added a note to the README.md. Because we have put significant emphasis on the simplicity of NatCorder's API, we route both video and audio through Unity. The significant advantage is a unified, uncomplicated API for developers to use; and the disadvantage (depending on how you view it) is that we can't make custom routes in native code for piping things like mic audio. Overall, I think we made the absolute right choice