NatCorder - Video Recording API

[DETAIL]
It's not possible to record video with Microphone in current version. (I'm not sure, you have already added something in latest beta)

Assigning AudioClip to AudioSource.clip is not a solution. It makes echo effect
(Microphone -> Speaker -> MIcrophone -> Speaker
......... forever)

 

You can record on Windows and macOS (using FFmpeg), not Windows Universal.

You can share videos using the native sharing UI (provided by iOS/Android).

No, this is not possible. The native sharing UI simply offers all sharing options to the user.

Yes. Whether you use NatCam or WebCamTexture, you can record anything that can be rendered to a texture.

 

NatCorder doesn't support sharing to specific places, like YouTube. The Share function provided by the Sharing API uses the path to the video to open the native sharing modal for the OS.

 

You can't record with an `AudioClip` (we already considered adding this, it poses too many issues). Instead, you should use the `AudioSource` overload and hook up your `AudioClip` to play from that audio source. We have added a boolean in the `StartRecording(..., AudioSource)` overload that will prevent the echo effect. We are designing an example scene that will properly illustrate the workflow for recording a replay with microphone audio.

 

Loving the plugin so far. Want to set up sharing so I can test, and I've been trying to use the code from the medium article.

Some things are coming up red as the names don't exist. I'm heading the script with "using NatCorderU.Core;" - was there anything else I should be adding so that it recognizes the sharing namespaces?

Thanks

 

I believe you need to import NatCorderU.Extensions. We might change this before the release version is completed.

 

Thanks. This added a couple of the names, however, the "DisplayThumbnail" is red, and so is "pathToReplay". Any idea what I add to make those work?

 

The gist you are referring to simply uses made-up names to illustrate using the Sharing API. In practice, these will be functions that do things with the video thumbnail and video path respectively.

 

Oh right - so it doesn't already have sharing set up? Is it to be used with another sharing plugin or something?

 

I'm not sure what you mean. The Sharing API exposes the Share and GetThumbnail API's. The gist you are referencing was made up to illustrate how these API's would be used. It isn't actually part of an example or project.

 

Ah no prob. I was hoping there'd be an example or documentation on how to share the video. I'll mess around with it until I come up with something. Thanks!

 

I'd like to get access to the current beta of the NatCorder API. If possible let me know.

 

Totally--the beta is still available for testing. Shoot me a PM with your email address and I'll share it with you.

 

My version doesn't have `StartRecording` with boolean param.

Did you added it to latest beta? Can you give me again?

 

We haven't made another beta build yet; we are still making additions and bug fixes.

 

If you give me some examples how to recording with mic, it really helps me.

I already changed all of my code with NatCorder instaed Everyplay.

Thanks

 

Hello Lanre,

I tried your Beta build with the sample project and I am experiencing some inconsistent behaviours for iOS:
- In iPhones it is working fine (iPhone 6s with iOS 11, iPhone 5s with iOS 10, iPhone 5c with iOS 9) - it records a video and properly shows the replay afterwards.
- In iPads:

• iPad 5 (Air - A1474) with iOS 11 - sometimes crashes when debugging via XCode with no relevant logs; when it doesn't crash it does not show the video for the replay.
• iPad 4 (A1458) with iOS 10 - sometimes crashes when debugging via XCode and gives the log mentioned in the "spoiler" bellow; gives same behaviour as the iPad above.
• iPad Air 2 (A1566) with iOS 11 - works fine, generates a video and shows the replay.

Spoiler: iPad 4 with iOS 10 error log

NatCorder: Initialized NatCorder 1.0 iOS backend
NatCam Dispatch: Initialized main dispatch
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
NatCorderU.Core.Platforms.NatCorderiOS:StartRecording(Configuration, SaveCallback, IAudioSource)
NatCorderU.Core.NatCorder:StartRecording(Configuration, SaveCallback)
NatCorderU.Core.Replay:StartRecording(Camera, Configuration, SaveCallback)
NatCorderU.Examples.RecordScreen:ToggleRecording()
NatCorderU.Examples.RecordScreen:Update()

(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
2018-01-10 14:15:23.870298+0100 natcorderbeta[587:242761] NatCorder Error: Failed to record video frame for time 17779154944 due to invalid recorder state: 3
(repeats over every frame...)
2018-01-10 14:15:27.912812+0100 natcorderbeta[587:242761] NatCorder Error: Failed to record video frame for time 22284019712 due to invalid recorder state: 3
2018-01-10 14:15:27.913229+0100 natcorderbeta[587:242761] NatCorder Logging: Encoder detected EOS marker
2018-01-10 14:15:27.913672+0100 natcorderbeta[587:242761] NatCorder Logging: Encoder finishing
Saved recording to: file:///var/mobile/Containers/Data/Application/CA9C806F-FFFC-474A-8236-3E0CC917EB6E/Documents/recording_2018_01_10_14_15_23_339.mov
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
NatCorderU.Examples.RecordScreen:OnReplay(String)
NatCorderU.Core.Platforms.<OnVideo>c__AnonStorey1:<>m__0()
System.Collections.Generic.List`1:ForEach(Action`1)
NatCamU.Dispatch.IDispatch:Update()
NatCamU.Dispatch.<OnFrame>c__Iterator0:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
NatCam Dispatch: Released dispatcher
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
NatCamU.Dispatch.IDispatch:Release()
NatCorderU.Core.Platforms.<OnVideo>c__AnonStorey1:<>m__0()
System.Collections.Generic.List`1:ForEach(Action`1)
NatCamU.Dispatch.IDispatch:Update()
NatCamU.Dispatch.<OnFrame>c__Iterator0:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
[prepareAsset]Error: The operation could not be completed
An unknown error occurred (-12893)

Have you ever experienced this?

 

As a follow up to my post, I tried adding Vuforia 7 to the project (with Unity 2017.3.0f3). Due to the default nature of the new Vuforia library, it activates itself on the main camera and shows the camera view in the background of the NatCorder sample scene.

When I started recording, the app crashes immediately on all devices except for the iPhone 5c with iOS 9. While debugging in XCode, it breaks in an EXC_BAD_ACCESS (code=1, address=0x0) error.

Could you consider looking into this interaction of NatCorder and Vuforia?

 

I would like to participate in the beta, this looks really interesting.

Best Regards

 

On Android it seems to work ok, but in iOS:
09.3.1 OK
10.3.2 CRASH when it tries to start recording EXC_BAD_ACCESS (code=1, address=0x0)
11.2.1 CRASH when it tries to start recording EXC_BAD_ACCESS (code=1, address=0x0)

 

@leonardo_demartino Did you also have Vuforia active while you tried to record?

 

Nope, I don't use Vuforia at all.

 

We are still building the examples. Sorry for the delay; I've been really busy over the past few days.

 

I have come across this sometime in the past, but I wasn't able to find the cause. What recording configuration were you using? Log the configuration width, height, framerate, and bitrate and let me know what it is. The issue might be happening because the encoder doesn't support the configuration.

 

This occurs because of native plugin rendering. iOS doesn't support dynamic linking, so you have to manually register a render delegate in native code. The problem is that Unity's API for doing so doesn't account for more than one plugin needing a native render delegate. The fix goes something like this:

- Build your Xcode project and open it, then open Vuforia's VuforiaNativeRendererController.mm file (not sure if it is still called this, but it should).
- Open NatCamRenderDispatch.mm and copy the prototype definitions for VuforiaSetGraphicsDevice and VuforiaRenderEvent from the Vuforia controller file into this one.
- Import VuforiaRenderDelegate.h into NatCamRenderDispatch.mm and in shouldAttachRenderDelegate, set the render delegate to a new instance of VuforiaRenderDelegate (basically copy what is in the same function in VuforiaNativeRendererController.mm).
- In OnRenderDispatch, invoke VuforiaRenderEvent with passing in the eventID argument.
- Comment out the entire VuforiaNativeRendererController source file.

 

Please email me a screenshot of the stack trace in Xcode and send me the full logs (from app start).

 

I was using the Default Configuration from the NatCorder API. It may have something to do with the Screen.width and Screen.height as the video resolution being too much for these devices. I will try applying specific settings and test it again as soon as I have some time.

Thank you for these instructions, I will also give them a try when I have some time. Do you foresee a more seamless way for NatCorder to integrate with other plugins that require a native render delegate? I think a lot more people will try to use NatCorder for AR apps.

 

Thanks for the beta package Lanre.

Integration was a breeze, the performance is great (only tested default quality) and I can confirm it is capable of recording ARCore sessions.

Only thing I noticed is that the UI in overlay is not recorded which could be ok in case this is not required. Anyway, this is stated in your readme with a possible workaround to achieve this.

Great job!

 

Yes, do let me know the results.

Not until Unity updates their native render plugin registration API. I will update the readme with instructions, but the process will not be uniform across different native render plugins because they all register for native rendering in different ways (NatCorder and Vuforia are a good example).

 

Perfect!

This has been a known issue with Unity (I don't think they've addressed it). When the UI canvas mode renders as an overlay, it isn't visible in calls to Graphics::Blit. You have to change the UI canvas render mode to screen space camera or world space.

 

Thanks for sharing the beta.
I did a really quick and simple first test on iOS with just an animated Mixamo character and the basic recording functionality:

Code (CSharp):

1. Replay.StartRecording(Camera.main, Configuration.Default, HandleSessionComplete, audioListener);

Unfortunately, the output video appears to be quite glitchy with some stuttering and flickering. I used an iPhoneX with iOS 11.1.1 to do the recording. The app was built using Unity 2017.3.0p2 and Xcode 9.2.
Any ideas what might be causing these visual errors?
https://drive.google.com/file/d/1y1kxT71wihNJbchobrr_8R0YrzMt02C0/view?usp=sharing

Edit: tested on an iPhone 6 with the same results.

 

Ah, I think I found the problem. Even your simple spinning cube in the RecordScreen example scene starts glitching if the framerate is set to 60. Do I need to configure something to make 60fps work glitch free?

Code (CSharp):

1.         void Awake()
2.         {
3.             Application.targetFrameRate = 60;
4.         }

 

Hi @peterahou , I had the same effects in Android devices when I was messing about with the bitrate in the video recording configuration. I don't know much yet about the details of bitrate vs framerate vs video resolution, but I would suggest you take a look into that.

 

Thanks for the suggestion
Unfortunately, I don't see any difference – the glitches still appear even when I double the bitrate used by the default configuration and reduce the resolution to 1280x720.
https://drive.google.com/file/d/1IRo4w6-_791gCU0pFQc6NkTo497tKWZZ/view?usp=sharing

 

I was planning on buying EasyAR pro for the video recording feature but have not been able to get it to work at all, even though I have a pro trial key.

NatCorder was up and running in minutes! Great job!
I have experienced a few crashes on Android and it usually happens on the second recording. I will try tweaking a few things and am sure I will figure it out.

The good news is that it works with EasyAR basic which means it will be very useful for people wanting to record image tracking AR.

What price are you planning to put on it when it is released? (apologies if you have already answered this)

 

So, just a quick follow up. It appears that I can only get a glitch free recording if I force the framerate to 30. It's not enough to use a Configuration with the framerate parameter set to 30 making the recorder run at 30fps, I have to set Application.targetFrameRate to make everything run at 30fps.
Is anyone else using iOS and able to get different results?

 

Further testing: As soon as I hit record in my EasyAR powered app it does start feeling sluggish. There is definite stuttering when I move my android phone that isn't there when I am not recording. If I leave it recording for too long (30 seconds) it crashes the app.

 

That's a good find. I have opened an issue for this and I will try to reproduce the issue.

I've noted this. This looks like a bug in both the Replay API and NatCorder.

 

Bitrate shouldn't affect this. The issue comes around because Unity isn't completing the call to Blit on the frame before NatCorder encodes the frame.

 

Can you email me the full logs from logcat with the crash? It will be very useful in debugging.

We haven't settled on a price yet. I'll make sure to post once we do.

 

This makes sense with our current suspicions. I'll work on a fix and follow up with you for testing.

 

What device are you testing on? Are you always running AR, even when you aren't recording? Regarding the crash after a while, send me the logs for that.

 

I am using a Galaxy S7 Edge to test on and I am running the AR constantly.

 

I decided to move on to the sharing API and see if I could get it to work on iOS. Unfortunately, it doesn't seem to work at all. Has anyone had any luck with this?

Edit: Ah, my bad. Looking at the source I see that this has not been implemented yet

 

Quick update: If I turn MSAA off on the recording camera, it no longer crashes and its as smooth as butter!

I spoke to soon. It seemed smoother but I may have imagined it as the stuttering is still there if I move the phone too quickly and it still crashes after a while. I will send a crash log.

 

I've had similar reports of MSAA and stuttering. Do send me the logs.

 

Well, I'm seeing issues with stuttering and glitches both with and without MSAA. Also, it appears that my animated character moves faster in the video than it does in the realtime scene when I play it back using Unity's built in video player. In fact, the last frame of the video is displayed for a couple of seconds at the end of the video, suggesting some framerate mismatch or something like that. This happens at both 30 and 60fps.

 

Hi! Thanks for putting together this asset!
Just tried exporting the example project (the rotating cube), but the app freezes when I double tap. Is there something I need to do when exporting to make it work?
I'm on a Google Pixel, latest Android.
Thanks

NullReferenceException: Object reference not set to an instance of an object: at NatCorderU.Core.Replay+CameraRecorder.OnRenderImage (UnityEngine.RenderTexture src, UnityEngine.RenderTexture dst) [0x00000] in <filename unknown>:0
(Filename: Line: -1)
NatCorder Error: Cannot acquire frame when NatCorder is not recording

 

Hi all, I would love to participate in the BETA program. Recording video and audio within our APPs is a challenge that I have had to face many times and never with a 100% success... Thank you very much for working on this!

 

I want to Become a Beta Tester, my email is 382309009@qq.com or zhoujinhao2599@icloud.com, please email to me , thank you .