NatCorder - Video Recording API

New user here and not very familiar w the tool, so please pardon my noobishness. I'm on a deadline so I will do some more research but wanted to at least put this out here in the meantime -

Capturing a video from user device, and I want to make sure the video always writes the file as 1080x1920. Current code bit looks like this:

var width = Screen.width % 2 == 0 ? Screen.width : Screen.width - 1;
var height = Screen.height % 2 == 0 ? Screen.height : Screen.height - 1;
(.......)
recorder = new MP4Recorder (width, height, frameRate, sampleRate, channelCount);

In some device cases this is yielding, say, 444x960, or 828x1792. So my thought was "ok I'll just fix those vars, width = 1080 and height 1920" but then this still yields some distortion in the image. Again this is new territory for me, so I might be thinking of something either very simple to adjust or really complex, I am not sure - anybody have any thoughts? Thanks for reading!

 

Awesome thanks so much! Also, if there's any work around fix you can think of that'd be greatly appreciated. We're shipping in the next week or so. And optimally we can avoid the mem leak (although it is pretty small, so not end of the world I suppose).

 

You'll typically get stretching if you are rendering within an AR framework, and the screen's aspect ratio doesn't match that of the recording. AR frameworks render slightly differently, so you won't be able to use an arbitrary resolution. What you can do is to aspect-scale your recording resolution, so keep either the width or height fixed then compute the other using the screen's aspect.

 

Hello,

Im using NAT-CORDER + NAT-MIC, i need to share the recording on facebook or maybe other social platforms, can it do that too???

 

Use NatShare.

 

Where can it share videos to exactly? FB, Whats App, where?

Will this code work to share on FB or Whats App for Android and IOS??

Code (CSharp):

1.     public void ShareThis(string text = null, string videoPath = null, Texture2D texture = null)
2.     {
3.         var payload = new SharePayload();
4.  
5.         if (text != null) payload.AddText(text);
6.         if (texture != null) payload.AddImage(texture);
7.         if (videoPath != null) payload.AddMedia(videoPath);
8.         if ((text != null) || (videoPath != null) || (texture != null)) {
9.             payload.Commit();
10.         }
11.     }

 

NatShare uses the native share sheet, so the target apps depend on what apps are installed and support receiving share payloads.

 

I profiled this in Instruments and apparently there isn't actually a leak:

Interestingly though, I can reproduce the memory increase when recording from an AudioSource. I'm guessing that the native framework (AVFoundation) is pooling some of the intermediate buffers that are used to provide raw PCM data to the AAC encoder, and likely isn't releasing that memory back to the OS. But Instruments doesn't consider this a leak.

 

Oh wow, thanks Lanre! That's really good to know. We'll keep an eye on it to see if it results in any memory problems. But good to know it's not being considered a leak in iOS land. Thanks for looking into this!

 

it support WebGL?

 

I implement for webGL but when i start recording show error

1. exception thrown: TypeError: Object doesn't support property or method 'captureStream',TypeError: Object doesn't support property or method 'captureStream' at _NCCreateWEBMRecorder (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:57731) at Module[dynCall_iiiii] (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:417376) at invoke_iiiii (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:311094) at Module[dynCall_v] (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:424075) at browserIterationFunc (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:155780) at Browser.mainLoop.runIter (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:158860) at Browser_mainLoop_runner (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:157305)
2. SCRIPT438: SCRIPT438: Object doesn't support property or method 'captureStream'

 

hello ,
How do get color32 form videoRecorder?.CommitFrame

 

There is an independent, open-source WebGL recorder called WebCorder.

I've updated the documentation to mention this, but WebCorder does not support Microsoft Edge. It only supports Chrome, Firefox, and Safari.

 

The commit frame function is a sink. You don't get anything from it; you instead send data to it.

 

In fact, how can I preview before Create a video?

 

The problem I have is storing a array from IntPtr or Texture2D that causes the mobile to crash, but how do you comit these in the buffer that this does not happen?

 

You should already know what you are recording beforehand. For example, if you are recording a game camera using a CameraInput, you would already be able to see what the camera sees on screen. This is your responsibility, not NatCorder's.

See the docs for info on it. There are code examples littered all over the docs.

 

Thanks for the sample code,
But this is exactly my problem:
// We've collected a bunch of frames we want to turn into a video
Texture2D [] frames = new [] {...};
In Android, a array of Texture2D can not be defined, this will cause the program to crash.
my question is :
How do you Commit Color32 in the buffer?
That the mobile does not crash?

 

What do you mean by "a presentation of Texture2D can not be defined"? You can commit a Color32[] by sending it to CommitFrame. You must guarantee that all the pixel buffers have the same width and height as was specified when the recorder was created.

 

a array of Texture2D

 

When I define an array of Texture2D.
As the size of this array increases, the mobile crashes.

 

I implement for webGL but when i start recording show error

1. exception thrown: TypeError: Object doesn't support property or method 'captureStream',TypeError: Object doesn't support property or method 'captureStream' at _NCCreateWEBMRecorder (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:57731) at Module[dynCall_iiiii] (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:417376) at invoke_iiiii (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:311094) at Module[dynCall_v] (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:424075) at browserIterationFunc (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:155780) at Browser.mainLoop.runIter (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:158860) at Browser_mainLoop_runner (blob:null/3f7e7cc4-3c14-4de6-a135-5306d7795c84:8:157305)
2. SCRIPT438: SCRIPT438: Object doesn't support property or method 'captureStream'

i use webcorder and test it Microsoft edge

 

This means you are running out of memory. So you will have to manage your memory. This has nothing to do with NatCorder.

 

I mentioned this above. WebCorder doesn't support Microsoft Edge.

 

Yes that's right ,
But you are also committing the same data.
Why doesn't this happen to you?
Thank you for your help.

 

NatCorder doesn't copy buffers. The buffers you commit are pinned and consumed by the encoder by the time that CommitFrame returns. As such it is the client's responsibility to ensure that they are not leaking memory from any buffers they send. Instead of keeping your textures in memory, you can destroy them when you are done with them.

 

The problem is that we have a section inside the program where the user edits the video frame,
So I had to save the frames to the hard drive and that slowed down the program.
Do you know a way to help me?

 

I don't know of anything you can do besides re-designing your app's flow around this limitation. It sounds more extensive than making one change here or there.

 

Ok thanks for the reply

 

Any one know about save file of webm. WebGL file save in blob url how i get video from blob url and how to play it

 

Unity's VideoPlayer should support playing back videos from blob URL's.

 

hello ,
I followed this link for offline recording and the video was recorded well.
https://docs.natsuite.io/natcorder/workflows/offline-recording
But I have trouble adding audio to this video.
Do I have to record audio online?
Or can the sound be added offline? How can this be done?

 

thanks for reply

 

Hii Lanre ,

is this possible to backgraound transparent for GIF recoding(using NatCorder) ?

 

Hi Lanre! Any possibility to save .mp4 file in WebGL? Save .webm file that cannot play by unity video player.

 

NatCorder only allows for video to be recorded when doing offline recording. You can't do offline recording for audio.

 

Unfortunately, no. NatCorder does not support transparent GIF's.

 

WebCorder doesn't support recording MP4's (specifically, the H.264 codec). The reason is that it is not widely supported. See this question on StackOverflow for more info.

 

I want to record audio online
But the Texture is offline.
How do I do this?

 

Ok thanks for reply

 

This isn't possible. You have to either record both video and audio online; or record only video offline. These are the only two supported use cases.

 

After CommitFrame, it is not possible to edit the frame?

I bought this package and NatDevice with financial problems, but both have problems, please help me with this.

 

CommitFrame is a sink. Once you send a frame to it, it forwards the frame to the hardware encoder. There is no way to edit the frame from there. You should do all your edits before committing the frame.

 

Can you play the movie at high speed once, to get the status online, but not save at high speed at the end of the movie?

I really apologize for taking your time.

 

I would like to know if the videos appear directly in the Android and iOS gallery or if I need to do this manually / programmatically so that it appears in the user gallery. Another question is whether I can capture still images as well, such as a screenshot and save to JPG, PNG or just videos and GIFs?

 

I'm not sure what you mean here. Basically, you have two options when recording with NatCorder. You can either record a bunch of video frames in quick succession like in a for-loop (this is called offline recording); or you can record video frames with or without audio frames in realtime (this is called online recording).

 

Videos are always recorded to the app's private documents directory. To copy to the camera roll, you can use a native gallery plugin like NatShare.

NatCorder currently only supports recording MP4 videos and animated GIF images. Unity already provides a way to capture screenshots.

 

Can't Natcorder paste audio (mp3 or wav) to movie (mp4)?

 

No, NatCorder does not support any form of pasting. This type of functionality is effectively video editing. NatCorder is not a video editor; it is strictly a video recorder.

 

Hi,

Im using the old NAT-CORDER + NAT-MIC to record a video + audio from the screen. After around 15 to 30 seconds of recording the IOS app runs out of memory and crashes,

seems it gets stuck for a long time at audioDevice.StopRecording(); even if i only record for 5 seconds.

WARNING -> applicationDidReceiveMemoryWarning()

WARNING -> applicationDidReceiveMemoryWarning()

WARNING -> applicationDidReceiveMemoryWarning()

WARNING -> applicationDidReceiveMemoryWarning()

This happens on IOS, the same exact code works fine on Android, any suggestions?