NatCorder - Video Recording API

Can you try restarting your machine, or running on a different machine? What OS version?

 

After some many more test, discarded a pc problem, from new project it works, then reimported natCorder and example scene work, then a test scene, i made it work with the same code, im narrow down posibilities by trial and error.

Edit: this is my first crash with the Editor offering me to report a bug, this is the error from UPM:

Code (CSharp):

1. [2020-03-20T03:10:32.700Z][WARN] Failed to determine if package 'com.unity.formats.fbx@2.0.1-preview.3' is compatible with Unity version '2019.3.0f3': Error: '2018.3.2018.3.4f1' is not a valid Unity version
2.  

this is something related to natCorder?

 

e

Just compare both videos you will see the difference. With the old plugin whenever I press share button parental gate popped up, after clearing the parental gate its started playing model building up with the rotation after my model build-up done I'm stopped Natcorder recording it gave the path to the recorded video, Using Natshare I'm sharing that video

With new plugin whenever I press share button parental gate popped up, after clearing the parental gate ("Swipe left with two fingers") it has to start model build-up animation, but those model build-up happening in background visually it's not showing after completion of the model build-up its showing Share screen

 

This has nothing to do with NatCorder. Reach out to Unity.

 

What version of Unity are you running on? There was an issue with the screen hanging when recording on Unity 2018, but it was fixed in Unity 2019.2 (it was a Unity issue, not a NatCorder issue).

 

hello there,
I am trying make a video from a raw image (image sources changes every 5 sec) and I didnt get anything from your examples all are about webcam.
is there any simple example or tutorial for gameobject capturing and saving?
I am gonna try first on pc then mobile

 

You can record a game camera which looks at your RawImage. Or you can record the texture that is being displayed on your raw image. Look at the README to understand exactly how recording works.

 

I read the README
but dont get it what to write for MP4Recorder options u write ...
raw image is a component in an empty object called screen

var texture = screen.GetComponent<RawImage>().texture;

for (int i = 0; i < 150; i++)

        {

            recorder.CommitFrame(cameraTexture.GetPixels32(), clock.timestamp);

            await Task.Delay(10);

        }

I get error: texture does not contain a definition for 'GetPixels32'
if there is a full code for this pls send.

 

A RawImage's `texture` property returns a `Texture` object. But `GetPixels32` is a method only on a `Texture2D` object. If you know your texture being displayed is a Texture2D, then cast it to one and your code should work. I recommend looking at Unity's documentation on these little semantics.

 

since your answer I am trying to convert Texture to Texture2D but got no success
I am using a Render Texture in Assets and select it as Raw Images Texture and as I mentioned I get the texture. the render texture I used is 2D

 

If you are working with a RenderTexture, you will first have to readback the pixel data from the GPU (somewhat of the equivalent of Texture2D's GetPixels32) then commit the data to NatCorder. You have two options for doing this: AsyncGPUReadback and Texture2D.ReadPixels. See the CameraInput source for how we use both of these for recording game cameras.

 

I know very less about unity and your solutions are too complicated
let me tell you what I wanna do
I want to merge videos and images as much as I want and save as single file
for that I added an empty object and it has a videoplayer and raw image components. I have a render texture for video players target texture and to show the video I set that texture to raw images texture. and in code I set raw images texture to video players target texture or loaded pictures texture.
in code I work on raw images texture. as I understand its Texture not RenderTexture and I cannot commit frame to it.
now if you know any other way for project to work with your product please tell me I am willing to change or with given information if you have a better understandable solution that work for me too or if you have a fully working demo about recording a game UI please send and I try to understand that

 

Hello,

I need to record part of the screen (with sound) and send it in real-time to a server as a streaming video file.

This means while its recording it has to encode the image and audio as a video stream, and give it to me constantly in pieces so i can send it to a server as a streaming video file like HLS or something like that.

If it cant do all that, which parts can it do? What format can it give me so i can convert it to HLS/RTMP or some other live stream video format.

Thanks

 

NatCorder doesn't support any kind of streaming whatsoever. There isn't anything you can use NatCorder for with this objective.

 

I thought NatCorder records video and sound, so im sure it can give me the stream of images as its being recorded (even if i have to hack the code myself), so i can convert it to a video stream myself?

 

This part of recording has nothing to do with NatCorder; it's all Unity code. If you want to capture what your game camera sees, either use ScreenCapture.CaptureScreenshot or re-render the game camera to a RenderTexture and perform a readback of the pixel data from the GPU. NatCorder was architected deliberately to not have to deal with any of the gory details of sourcing the image data that is committed. Unity provides API's for all this.

 

So what exactly is NatCorder doing then?

 

Encoding to MP4, HEVC, and GIF. That's why it's a video recorder.

 

So its taking the screen shots and audio, passing it to the native encoder (Android, IOS, Windows MAC) to be encoded to MP4, HEVC, GIF etc correct?

 

At the highest level, yes. In practice it's more complicated due to how different hardware encoders are exposed by their respective platforms.

 

So its encoding it in real time ( file RenderTextureInputs.cs CommitAndroid(), CommitIOS(), CommitSync() etc), so why cant i get that encoded data?

 

All RenderTextureInput is fetching the pixel data from the GPU, the same one you can do with Unity's AsyncGPUReadback (which is why we've removed this class in more recent versions of the API). You can't get the encoded data because we don't even have access to it. Most encoding API's don't expose the H.264 stream; instead they mux it directly to the file. And even if they did expose it, the H.264 NALU's are not what you want. You would have to mux the NALU's into an MPEG-TS stream, then package it into a HLS stream. None of this is trivial. NatCorder is not your solution for this problem.

 

So how do the live video streaming apps work?

 

This is not my area of expertise. Agora might be a good place to start.

 

hello again,
I to readback the pixel data from the render texture.
I know it works I can put the texture2D to another raw image
now back to your commitFrame method
instead of 150 frames I added the comitFrame method to Uptade method and for testing I added a button when clicked it will call finishWriting

    async Task Update()

    {

        if (!stopRec)

        {

            if (!isDone)

            {

                Debug.Log("recording");

                Texture text = screen.GetComponent<RawImage>().texture;

                Texture2D text2D = convertTexture(text);

                recorder.CommitFrame(text2D.GetPixels32(), clock.timestamp);

            }

            else

            {

                Debug.Log("stopped");

                var recordingPath = await recorder.FinishWriting();

                Debug.Log(recordingPath);

                stopRec = true;

            }

        }

    }

it never return from recorder.FinishWriting();
dont get recording path on console
and I get an .mp4 file with 0 kB and vlc player doesnt play.
pls help

 

Does your code ever log "stopped"? Can you share the full code?

 

yes a lot of times
I opened a new project just to test It plays a videoclip

public class pt : MonoBehaviour

{



    public GameObject screen;

    MP4Recorder recorder = new MP4Recorder();

    RealtimeClock clock = new RealtimeClock();

    bool isDone = false;

    bool stopRec = false;

    async void Start()

    {



    }



    async Task Update()

    {

        if (!stopRec)

        {

            if (!isDone)

            {

                Debug.Log("recording");

                Texture text = screen.GetComponent<RawImage>().texture;

                Texture2D text2D = convertTexture(text);

                recorder.CommitFrame(text2D.GetPixels32(), clock.timestamp);

            }

            else

            {

                Debug.Log("stopped");

                var recordingPath = await recorder.FinishWriting();

                Debug.Log(recordingPath);

                stopRec = true;

            }

        }

    }

    Texture2D convertTexture(Texture tex)

    {

        Texture2D videoFrame = new Texture2D(2, 2);

        RenderTexture renderTexture = tex as RenderTexture;





        if (videoFrame.width != renderTexture.width || videoFrame.height != renderTexture.height)

        {

            videoFrame.Resize(renderTexture.width, renderTexture.height);

        }

        RenderTexture.active = renderTexture;

        videoFrame.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);

        videoFrame.Apply();

        RenderTexture.active = null;

        return videoFrame;

    }

    public void clicked()

    {

        isDone = true;

    }





}

I am very new to unity and to be fair didnt get your work in details I might done it wrong

 

@Lanre still waiting for your answer...

 

A few issues with your code:

• Update should not be async. Record frames in update, but stop recording in your `clicked` method.
• Your `convertTexture` method leaks memory because it allocates a new `Texture2D` every time it is called.
• Your initialization code doesn't look correct. `MP4Recorder` doesn't have a parameter-less constructor, so that initializer should prevent your code from even compiling.

A common pattern is to have explicit `StartRecording` and `StopRecording` functions. The `StartRecording` function can then create the recorder and anything else that will be needed during recording (this will include your `videoFrame` texture). I recommend looking at the example code, specifically the ReplayCam code for how recording is organized with those two methods.

 

Hello, i have a problem (with the 1.6 version) on Unity 2019.3.0f6 :

On windows plateform, in the unity editor, the recorder create a video with with very few display problems (in a complex scene you can see a little bit) BUT after a build of my project, the recorder create a video with a lot of display problems (with the same settings...).

I haven't tested the new version (1.7.1) yet but I don't know if it will fix the problem.

have you ever had this kind of problem and were you able to fix it?

(the compression effect is the problem, it is not a gif problem)


https://i.ibb.co/dkVZhFh/Unity-Windows-Build.gif

 

Can you share your recording code? What version of Windows (including minor version)? I recommend upgrading to 1.7.1 and trying that.

 

EDIT: it appears that this was due to files still existing in the project from an older version of NatCorder that was not cleaned up in the upgrade. Leaving this here in case anyone else encounters the same issue.

I'm getting an issue when I try to build on iOS.

Undefined symbols for architecture arm64:

  "_NCEncodeSamples", referenced from:

      _MediaRecorderBridge_EncodeSamples_m7E9330CCF63AEE98059BBB4011CE869F33018C9B in Assembly-CSharp.o

  "_NCEncodeFrame", referenced from:

      _MediaRecorderBridge_EncodeFrame_mD122667E89A002540F902029824AB8A5BF26C800 in Assembly-CSharp.o

  "_NCStopRecording", referenced from:

      _MediaRecorderBridge_StopRecording_m2F1E61E1673BB0A94C7D37617DDC55FE8300A6D1 in Assembly-CSharp.o

  "_NCStartRecording", referenced from:

      _MediaRecorderBridge_StartRecording_m16086725960E6789DC668280C83105900432F4F1 in Assembly-CSharp.o

Is there something I need to do to make sure it includes the library?

I'm using version 1.7.1.

 

Delete and reimport NatCorder into your project. Then delete and rebuild your Xcode project from Unity.

 

Can Asset record audio feed from microphone?

 

NatCorder can record audio; it is up to the developer to commit audio samples either directly or using the helper script for recording from an AudioSource or AudioListener. NatCorder will not record the hardware microphone for you (you have to provide it with the audio data). We have our NatDevice API which records the microphone, so you can use that for sourcing the microphone data.

 

Hello, I've just downloaded the plugin, I wanted understand how it works exactly but Unity Editor keeps crashing every time I press play for any of the sample scenes but the scene plays if it doesn't contain NatCorder code.

This was tested in an empty project.

I've uploaded the editor log and Apple's crash log.

Thanks

 

The editor crashes when the WebCamTexture is starting (see thread 0 call stack), and the crashed thread indicates the cause:

Code (CSharp):

1. Thread 87 Crashed:: Dispatch queue: com.apple.root.default-qos\
2. 0   libsystem_kernel.dylib            0x00007fff7167713e __terminate_with_payload + 10\
3. 1   libsystem_kernel.dylib            0x00007fff71691516 abort_with_payload_wrapper_internal + 119\
4. 2   libsystem_kernel.dylib            0x00007fff71691521 abort_with_payload + 9\
5. 3   com.apple.TCC                     0x00007fff698410d1 __CRASHING_DUE_TO_PRIVACY_VIOLATION__ + 163\
6. 4   com.apple.TCC                     0x00007fff6984102e __TCCAccessRequest_block_invoke.114 + 500\
7. 5   com.apple.TCC                     0x00007fff6983ebc5 __tccd_send_message_block_invoke + 231\
8. 6   libxpc.dylib                      0x00007fff71775004 _xpc_connection_reply_callout + 36\
9. 7   libxpc.dylib                      0x00007fff71774f8c _xpc_connection_call_reply_async + 69\
10. 8   libdispatch.dylib                 0x00007fff714d7548 _dispatch_client_callout3 + 8\
11. 9   libdispatch.dylib                 0x00007fff714ee080 _dispatch_mach_msg_async_reply_invoke + 369\
12. 10  libdispatch.dylib                 0x00007fff714e648f _dispatch_kevent_worker_thread + 1324\
13. 11  libsystem_pthread.dylib           0x00007fff717307ab _pthread_wqthread + 362\
14. 12  libsystem_pthread.dylib           0x00007fff717305c3 start_wqthread + 15\

Make sure you've given Unity permission to access your webcam.

 

Thanks for your reply, I looked into the code and the line

webCamTexture.Play();

Was crashing the editor and you're right, it was crashing because Unity has no permissions, however for some reason Unity Hub, Unity 2019.3.7f1, and Mac Catalina have some conflicts that do not allow Unity Hub to ask for permissions automatically so it won't show in the Security & Privacy.

I found the solution in this issue
https://issuetracker.unity3d.com/is...a-permission-and-crashes-on-privacy-violation

Comment by LAUNZONE, MAR 30, 2020

If anybody is havnig the same problem, this might be your solution. Thanks everyone!

 

Hello,
I'm currently using your plugin and it works perfectly on Windows in the Unity Player, but on Android it just crashes. Here is my code:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. using NatCorder;
4. using NatCorder.Clocks;
5. using NatCorder.Inputs;
6.  
7. public class RecordingScript : MonoBehaviour
8. {
9.     [Header("Recording")]
10.     public bool recordMicrophone = true;
11.     public bool recording = false;
12.  
13.     private IMediaRecorder videoRecorder;
14.     private CameraInput cameraInput;
15.  
16.     public void StartRecording()
17.     {
18.         Debug.Log($"Data: {Screen.width} {Screen.height}");
19.         if (recording) return;
20.  
21.         var frameRate = 60;
22.         var recordingClock = new RealtimeClock();
23.         videoRecorder = new MP4Recorder(
24.             1080,
25.             1920,
26.             frameRate,
27.             0,
28.             0,
29.             recordingPath => {
30.                 Debug.Log($"Saved recording to: {recordingPath}");
31.             }
32.         );
33.         // Create recording inputs
34.         cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
35.         recording = true;
36.     }
37.  
38.     public void ToggleMicrophone()
39.     {
40.         this.recordMicrophone = !recordMicrophone;
41.     }
42.  
43.     public void StopRecording()
44.     {
45.         if (!recording) return;
46.  
47.         cameraInput.Dispose();
48.         videoRecorder.Dispose();
49.         recording = false;
50.     }
51. }

When I click the button to call the StartRecording button I see no error, but when I click the StopRecording button the app crashes with this error:

Code (CSharp):

1. 2020-04-07 15:12:23.091 800-4299/? D/cie: Enter CIE::onProcess
2. 2020-04-07 15:12:23.091 800-4302/? I/GRALLOC: LockFlexLayout: baseFormat: 11, yStride: 1280, ySize: 921600, uOffset: 921600,  uStride: 1280
3. 2020-04-07 15:12:23.092 800-4288/? E/DeviceBaseImpl: [HWA_CAM3]getAfObjectDistance get ANDROID_HW_AF_OBJECT_DISTANCD fail
4. 2020-04-07 15:12:23.092 800-4288/? D/Misc____: [1a162fd_64D] onProcessFrame(670), afDistance(0.000000)
5. 2020-04-07 15:12:23.092 800-4294/? D/HwFaceAttr: [1a162fd_64D] [isNeedCalc] mRunningFrameCount=163, skip this frame
6. 2020-04-07 15:12:23.091 800-4302/? I/GRALLOC: LockFlexLayout: baseFormat: 11, yStride: 1280, ySize: 921600, uOffset: 921600,  uStride: 1280
7. 2020-04-07 15:12:23.106 800-4302/? I/IppAlgoPortOut: [HWA_CAM3]this=0x77fcd602f0, ipptype=256, enter deliver(), pBuf=0x78beb11510
8. 2020-04-07 15:12:23.106 778-1589/? I/Camera3-Stream: returnBuffer: begin(417)
9. 2020-04-07 15:12:23.107 778-1589/? I/Camera3-Stream: returnBuffer: end(417)
10. 2020-04-07 15:12:23.107 2904-4181/it.auties.scream I/GRALLOC: LockFlexLayout: baseFormat: 11, yStride: 1280, ySize: 921600, uOffset: 921600,  uStride: 1280
11. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0000000000000000
12. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
13. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
14. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: Build fingerprint: 'HUAWEI/MAR-LX1AEEA/HWMAR:10/HUAWEIMAR-L21A/10.0.0.193C431:user/release-keys'
15. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: Revision: '0'
16. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: pid: 2904, tid: 3342, name: UnityMain  >>> it.auties.scream <<<
17. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x0   0000000000000000  x1   000000710e200010  x2   00000000007e9000  x3   0000000000000000
18. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x4   000000710e9e9010  x5   00000000007e9000  x6   ff0d0204ff0d0204  x7   ff0c0305ff0c0305
19. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x8   0000000000000000  x9   0000000000000000  x10  00000000007e9000  x11  0000000000000000
20. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x12  ff0d0206ff0d0206  x13  ff0d0204ff0d0206  x14  00003aa335cd5275  x15  000005709e6aa71a
21. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x16  000000714b1a2f40  x17  00000072625f0b40  x18  000000713760e354  x19  00000000007e9000
22. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x20  000000710e200010  x21  0000000000000000  x22  0000000000000000  x23  0000000000000000
23. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x24  0000007135e244a4  x25  0000000000000001  x26  0000000000000001  x27  000000710a695b30
24. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     x28  000000710a695b30  x29  000000714a85de20  x30  0000007136335a14
25. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH:     sp   000000714a85ddc0  pc   00000072625f0ac8  pstate 0000000020000000
26. 2020-04-07 15:12:23.112 2904-3342/it.auties.scream E/CRASH: backtrace:

Android Device: Huawei p30 lite, EMUI 10
Unity: 2018
Plugin Version: 1.6.6
Graphics APIs: OpenGLES3, Vulcan
Multithreaded Rendering: true

 

Hi there. Can you share the full logs in a .txt attachment? The backtrace will contain the cause of the crash (but it is truncated in the logs you shared). Also, you should upgrade to NatCorder 1.7.1 on the Asset Store.

 

Can I? I'm using Unity 2018, it says that Unity 2019+ is required. By the way here is the log:

 

All NatSuite API's work on Unity 2018.3+. The logs don't contain a backtrace still. Can you try updating to 1.7.1 then running in debug mode instead of release?

 

Hello, I lost my Android phone and bought a new one today. Now NATCorder is crashing on the new phone Android 9 (Samsung Galaxy A20). It worked fine on my old phone.

The function call it crashes on is this

Code (CSharp):

1.         videoRecorder = new MP4Recorder(w, h, 30, sampleRate, channelCount, OnReplay);

Here is the log, any idea how to fix

2020-04-07 20:29:26.494 21280-21316/com.app.appE/Unity: AndroidJavaException: android.media.MediaCodec$CodecException: Error 0xfffffc0e
android.media.MediaCodec$CodecException: Error 0xfffffc0e
at android.media.MediaCodec.native_configure(Native Method)
at android.media.MediaCodec.configure(MediaCodec.java:1960)
at android.media.MediaCodec.configure(MediaCodec.java:1889)
at com.yusufolokoba.natcorder.MP4Recorder.<init>(MP4Recorder.java:42)
at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
at com.unity3d.player.UnityPlayer.c(Unknown Source:0)
at com.unity3d.player.UnityPlayer$e$2.queueIdle(Unknown Source:72)
at android.os.MessageQueue.next(MessageQueue.java:395)
at android.os.Looper.loop(Looper.java:181)
at com.unity3d.player.UnityPlayer$e.run(Unknown Source:32)
at UnityEngine.AndroidJNISafe.CheckException () [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.AndroidJNISafe.NewObject (System.IntPtr clazz, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.A

 

Upgrade to NatCorder 1.7.1, and check your recording width and height. Chances are they are values that cause the encoder to puke.

 

I'm using 1.6.1, but it seems 1.7.1 i cant download it because it requires Unity 2019.2.17 or higher but I'm using Unity 2018.3 which was the version i been using since i bought NATCorder.

Why did you force people to use Unity 2019.2??? I cant use that version.

 

Unity 2019.2 is not required. The Asset Store page only says this because that was the version used to submit the update. NatCorder will work with Unity 2018.3+.

 

Unity wont let me download it. So how can i get it?

 

Same thing, I just tried

 

I'll submit the next update with 2018.3. This should fix the issue.

 

I installed Unity 2019.2 and used that to download NatCorder update, then search for the package on the computer, then import that into 2018.3.

The whole NatCorder package is different including all the name spaces, so all my code is broken