NatCorder - Video Recording API

Can you share your crash logs? Also, make sure to dump the full call stack when the crash occurs.

 

Hi Lanre,

After doing logging the debug lines, I was able to pinpoint the crash to MP4Recorder.cs Specifically Bridge.CreateMP4Recorder()

Code (CSharp):

1. Bridge.CreateMP4Recorder(
2.                 Utility.GetPath(@".mp4"),
3.                 width,
4.                 height,
5.                 frameRate,
6.                 sampleRate,
7.                 channelCount,
8.                 videoBitRate,
9.                 keyframeInterval,
10.                 audioBitRate,
11.                 out var recorder
12.             );
13.             if (recorder != IntPtr.Zero)
14.             {
15.                 Debug.Log("Natcorder Debug: Return Recorder");
16.                 return recorder;
17.             }
18.             Debug.Log("Natcorder Debug: Failed to create MP4Recorder");
19.             throw new InvalidOperationException(@"Failed to create MP4Recorder");

 

Looks like something about the recording path. Are you modifying the recording path in any way? Can you log the value of the path that gets sent to `Bridge.CreateMP4Recorder`?

 

Question: Is Natcorder compatible with recording Post Process effects of cameras? Currently, the recording records everything except the Post Process of the scene.

Thanks!

 

Hey there, yes but it requires two steps:

1. Modify the `CameraInput` constructor and change the format of the `frameDescriptor` to `RenderTextureFormat.ARGBHalf`.
2. Add a tonemapping filter in your post-processing stack.

 

Hello Yusuf,
Just wanted to ask if NatCorder currently officially supports Apple Silicon architecture?

Thank you!

 

Issue) Videos recorded on Android12 devices will play abnormally on IOS.

Record device: SM-G996 (Galaxy S21+ 5G, OneUI4.0 Beta)
Play device : iPhone11 Pro

When playing a video recorded on an Android 12 device using NatCorder on an iPhone 11, the audio plays faster than the video.
Under the same conditions, when recorded on devices with android 11 or lower, it plays normally.

Sample video
android12 recording: https://drive.google.com/file/d/1BQLCQAA6eIJd0Qxpb6AY-7IXWUV1fkDj/view?usp=sharing
android10 recording: https://drive.google.com/file/d/1fV8DEqUBBaMUUTcVe2MIZ4tVVSjNn33Z/view?usp=sharing

I checked the codec, bitrate, framerate, but couldn't find the exact cause. Any help would be appreciated.

 

Yup, it supports Apple Silicon and Intel architectures on macOS.

 

Both videos play normally on my MacBook Pro. I'm not sure what the issue is. Have you tried playing on other iOS devices?

 

link : https://drive.google.com/file/d/1xZ5Kj8DE9kx1fxVx0qqDNiM2FyBeRaXK/view?usp=sharing

This is a video I tested with M1 Macbook Air.
When played a Google Drive link, it plays good.
But if play a video using QuickTime Player, the audio is played so fast.

 

I don't believe there is any fix that NatCorder can implement for this issue. NatCorder doesn't perform the encoding process by itself; it relies on platform API's like MediaCodec on Android. This issue would be from your Android 12 device's encoder.

 

export file can use unityplayer url?

 

I don't understand your question.

 

Hi Lanre,
i just bought your natCorder and i have a strage issue with karaoke example all audio files saved are speed up.
i'm using unity 2020.3.23 on windows 10. Can you help me? thanks

 

Hey there, can you share a screenshot of your project's Audio Settings? I presume you didn't make any modifications to the Karaoke example code?

 

ok thanks for the replay.
I found that the problem was in audio settings. I had quad speaker settings enabled.
When i switch to stereo all works.
Is it a bug? is it possibile use this addon while using quad speaker settings?
thanks a lot.

 

The WAV recorder should be able to record an arbitrary number of audio channels. The reason why the example creates sped-up audio is because of how we initialize the channel count on the recorder:

Code (CSharp):

1. (int)AudioSettings.speakerMode

If you look at Unity's `AudioSpeakerMode` enumeration, you'll see that Mono is set to 1, Stereo is set to 2, but Quad is set to 3 (instead of 4).

 

Hey, I would like to know if Natcorder works properly on Chromebooks.

When I tried the build on a Chromebook, I got a DllNotFoundException for NatCorder at MP4Recorder.cs:33.
Do I have to change anything for it to work on Chromebooks?

Currently using NatCorder 1.7.2 & Unity 2019.2.17.
I've tried it using NatCorder 1.8.1 & unity 2019.4.33, and it still shows the same error.

 

From what I can see, Chromebooks run on ChromeOS which is a derivative of Linux. NatCorder does not support Linux.

 

Have you resolved？

 

Hey @Lanre
I am using NatCorder and NatRenderer aar files to record video on my android studio project. I successfully created MP4Recorder and I commit bytebuffer which is copied from bitmap with the code below

                val bitmap = getDrawingCache()

                val buffer = ByteBuffer.allocate(bitmap.rowBytes * bitmap.height)

                bitmap.copyPixelsToBuffer(buffer)

                recorder.commitFrame(buffer, time)

However my app is crash as soon as record start with this error

Code (CSharp):

1. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** **
2. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: Build fingerprint: 'xiaomi/mido/mido:7.0/NRD90M/V11.0.2.0.NCFMIXM:user/release-keys'
3. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: Revision: '0'
4. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: ABI: 'arm64'
5. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: pid: 16045, tid: 16890, name: NatRender GLRen  >>> com.concept.story <<<
6. 2021-12-28 16:46:49.653 16892-16892/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
7. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG: Abort message: 'Invalid address 0x7b81950c passed to free: value not allocated'
8. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG:     x0   0000000000000000  x1   00000000000041fa  x2   0000000000000006  x3   0000000000000008
9. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG:     x4   0000000000000000  x5   0000000000008080  x6   0000000000000000  x7   0000000000000038
10. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG:     x8   0000000000000083  x9   ffffffffffffffdf  x10  0000000000000000  x11  0000000000000001
11. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG:     x12  ffffffffffffffff  x13  0000000000000000  x14  0000000000000000  x15  00223a1190a66c0b
12. 2021-12-28 16:46:49.657 16892-16892/? A/DEBUG:     x16  0000007f8b86eed0  x17  0000007f8b818510  x18  00000000ffffffff  x19  0000007f61dc24f8
13. 2021-12-28 16:46:49.658 16892-16892/? A/DEBUG:     x20  0000000000000006  x21  0000007f61dc2450  x22  0000000000000000  x23  0000007f8b879800
14. 2021-12-28 16:46:49.658 16892-16892/? A/DEBUG:     x24  0000007f8b87986c  x25  80cbd74fee0b053f  x26  0000007f8914cf83  x27  0000007f8914cfb4
15. 2021-12-28 16:46:49.658 16892-16892/? A/DEBUG:     x28  0000000000000002  x29  0000007f61dc0cd0  x30  0000007f8b8159a0
16. 2021-12-28 16:46:49.658 16892-16892/? A/DEBUG:     sp   0000007f61dc0cb0  pc   0000007f8b818518  pstate 0000000060000000
17. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG: backtrace:
18. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #00 pc 000000000006b518  /system/lib64/libc.so (tgkill+8)
19. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #01 pc 000000000006899c  /system/lib64/libc.so (pthread_kill+64)
20. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #02 pc 0000000000023ee8  /system/lib64/libc.so (raise+24)
21. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #03 pc 000000000001c96c  /system/lib64/libc.so (abort+52)
22. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #04 pc 0000000000020ef4  /system/lib64/libc.so (__libc_fatal+104)
23. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #05 pc 000000000008fd50  /system/lib64/libc.so (ifree+1304)
24. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #06 pc 000000000008fdd4  /system/lib64/libc.so (je_free+128)
25. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #07 pc 000000000034d988  /system/lib64/libart.so (_ZN3art3JNI29ReleasePrimitiveArrayCriticalEP7_JNIEnvP7_jarrayPvi+664)
26. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #08 pc 00000000000fe848  /system/lib64/libart.so (_ZN3art8CheckJNI29ReleasePrimitiveArrayCriticalEP7_JNIEnvP7_jarrayPvi+740)
27. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #09 pc 00000000000c3908  /system/lib64/libandroid_runtime.so
28. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #10 pc 00000000021ab550  /system/framework/arm64/boot-framework.oat (offset 0x1990000) (android.opengl.GLES20.glTexSubImage2D+220)
29. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #11 pc 00000000000d2b68  /system/lib64/libart.so (art_quick_invoke_static_stub+600)
30. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #12 pc 00000000000df5b0  /system/lib64/libart.so (_ZN3art9ArtMethod6InvokeEPNS_6ThreadEPjjPNS_6JValueEPKc+252)
31. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #13 pc 000000000028e688  /system/lib64/libart.so (_ZN3art11interpreter34ArtInterpreterToCompiledCodeBridgeEPNS_6ThreadEPNS_9ArtMethodEPKNS_7DexFile8CodeItemEPNS_11ShadowFrameEPNS_6JValueE+312)
32. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #14 pc 0000000000288744  /system/lib64/libart.so (_ZN3art11interpreter6DoCallILb1ELb0EEEbPNS_9ArtMethodEPNS_6ThreadERNS_11ShadowFrameEPKNS_11InstructionEtPNS_6JValueE+444)
33. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #15 pc 000000000055ab88  /system/lib64/libart.so (MterpInvokeStaticRange+356)
34. 2021-12-28 16:46:50.042 16892-16892/? A/DEBUG:     #16 pc 00000000000c5714  /system/lib64/libart.so (ExecuteMterpImpl+15380)

This error is comes from below method

public synchronized void commitFrame(ByteBuffer pixelBuffer, long timestamp) {

        Semaphore commitToken = new Semaphore(0);

        this.renderContextHandler.post(() -> {

            GLES30.glBindTexture(3553, this.frameTexture);

            GLES30.glTexSubImage2D(3553, 0, 0, 0, this.width, this.height, 6408, 5121, pixelBuffer);

            commitToken.release();

            this.blitEncoder.blit(this.frameTexture);

            this.renderContext.setPresentationTime(timestamp);

            this.renderContext.swapBuffers();

        });

        try {

            commitToken.acquire();

        } catch (InterruptedException var6) {

        }

    }

when I debug code specifically it crash in this line

 GLES30.glTexSubImage2D(3553, 0, 0, 0, this.width, this.height, 6408, 5121, pixelBuffer); 

Perhaps, you could help me to understand what I am doing wrong here?

 

Wasn't able to repro. What version of NatCorder are you using?

 

You've got an interesting use case. Nothing looks out of the ordinary, except maybe your pixel buffer has the wrong size. Make sure that it is RGBA8888, and that its byte length is exactly `width * height * 4`.

 

Hello, loving the asset so far, and I love how active you have been in this forum!

I need a bit of help. NatCorder is crashing my Android app when I use it with Proguard enabled.
It crashes the moment I press the Record button inside the example ReplayCam scene.
It works perfectly if Proguard is not used (i.e. Minify is set to None).

What should I be adding to my proguard-user.txt file?

I have tried these different attempts separately with no luck:

Attempt 1:
-keep class com.yusufolokoba.natcorder.* { *; }
-keep class com.yusufolokoba.natcam.* { *; }
-keep class com.yusufolokoba.natrender.* { *; }

Attempt 2:
-keep class com.yusufolokoba.**

Attempt 3:
-keep class com.yusufolokoba.** { *; }​

 

Thank you!

Code (CSharp):

1. - keep class api.natsuite.**

This covers all the NatML API's (NatML, NatCorder, NatDevice, NatShare, and so on).

 

Thanks, adding this to proguard-user.txt worked!

-keep class api.natsuite.** { *; }

 

Perfect!

 

Hi Lanre,

NatCorder 1.8.1 does not work with Google Cardboard XR Plugin v1.11.0. Starting recording freezes the screen. The latest compatible version of NatCorder is 1.7.3.

Can you fix this issue in the next version?

 

I'll need more information on this. Can you share the full app logs in a .txt attachment? If it freezes the screen, then it sounds like a Unity bug--not a NatCorder bug. An issue like this will specifically do with rendering the game camera(s) to texture, something handled entirely within Unity.

 

Hello! I've had a great experience using the asset, however, I've faced one issue that I haven't been able to resolve so far and would deeply appreciate some help with.

I'm using a custom AudioMixerAttachment, as I need to combine Unity audio with audio from an external source. Everything works out well, except for 1 use case:
If the recorded AudioSource is not playing when recording starts, performance starts critically dropping after 1-2 seconds of recording, ultimately resulting in a crash/freeze on mobile devices.

I managed to figure out a work-around: playing a looped, empty (silent) AudioClip while AudioSource isn't being used solves the problem. But I would love to replace this temporary solution and find the actual source of the bug before developing my app's audio system further.

Here are the key code snippets. I can provide full scripts via email, if needed.

AudoMixerAttachment.cs

Code (CSharp):

1. private bool _mute;
2. private Func<float[]> _getMicrophoneSample;
3. private Action<float[]> _commitSamples;
4.  
5. public void Init(bool mute, Func<float[]> getMicrophoneSample, Action<float[]> commitSamples)
6. {
7.     _getMicrophoneSample = getMicrophoneSample;
8.     _mute = mute;
9.     _commitSamples = commitSamples;
10. }
11.  
12. private void OnAudioFilterRead (float[] data, int channels) {
13.     // Get first sample from microphone samples queue
14.     var microphoneSample = _getMicrophoneSample();
15.     // If it's not null - mix it with Unity audio sample, otherwise - leave only Unity audio sample
16.     var mixedData = microphoneSample != null ? AudioBufferMixer.MixAndClampFloatBuffers(data, microphoneSample) : data;
17.  
18.     // Commit audio to recorder
19.     _commitSamples(mixedData);
20.  
21.     // Mute audio playback if needed
22.     if (_mute)
23.         Array.Clear(data, 0, data.Length);
24. }

ScreenRecorder.cs

Code (CSharp):

1. public bool TryToStartRecording(int recordWidth, int recordHeight)
2. {
3.     ...
4.  
5.     _recordingClock = new RealtimeClock();
6.  
7.     // Initialize the recorder
8.     _videoRecorder = new MP4Recorder(
9.         recordWidth,
10.         recordHeight,
11.         framerate,
12.         sampleRate,
13.         (int)AudioSettings.speakerMode,
14.         recordingPath => {
15.             ...
16.         }
17.     );
18.  
19.     // Create recording inputs
20.     _cameraInput = new CameraInput(_videoRecorder, _recordingClock, ...);
21.  
22.     ...
23.  
24.     // Add mixer attachment to add microphone audio from flutter to recording
25.     _audioMixerAttachment = audioSource.gameObject.AddComponent<AudioMixerAttachment>();
26.     _audioMixerAttachment.Init(false, GetMicrophoneSample, CommitSamples);
27.  
28.  
29.     // FIXME: the workaround described above
30.     if (!audioSource.isPlaying)
31.     {  
32.         audioSource.clip = silenceClip;
33.         audioSource.Play();
34.     }
35.  
36.     ...
37. }
38.  
39. // Get first sample in queue, if queue is empty - return null
40. private float[] GetMicrophoneSample()
41. {
42.     return _microphoneSamples.Count > 0 ? _microphoneSamples.Dequeue() : null;
43. }
44.  
45. // Commit audio samples to video recorder
46. private void CommitSamples(float[] data)
47. {
48.     // AndroidJNI call copied from NatCoder's sources
49.     // Recording crashes on Android without this
50.     AndroidJNI.AttachCurrentThread();
51.     _videoRecorder.CommitSamples(data, _recordingClock.Timestamp);
52. }

Thank you in advance and have a nice evening!

 

Hey there, I'm pretty sure this doesn't happen in the latest version of NatCorder, so upgrade to 1.8.1 on the Asset Store and you should be all set.

 

Hello, I have a problem using this asset. I hope you can help me. Thank you
When recording video, I want to record microphone sound and unity built-in sound at the same time
I can use audioinput to receive the audiolister parameter, but the sound of the microphone will be played out from the device
My English is not very good

 

NatCorder does not support this out of the box. You'll have to come up with a way to mix audio, which is not easy.

 

Hello, I need to record an Unity built-in audio source (from AVProVideo for instance) *and* the microphone. I think i have a sampling issue between both sources.
I understood that I need to mix the audio sources, but I don't know how ..
If someone has the same issue (or a solution ..)

Thanks in advance !

 

I haven't found any easy solutions for this.

 

I've upgraded to the last version of NatCorder, but it didn't help
I've also changed the scripts a bit accordingly to NatCorder's API changes:

ScreenRecorder.cs

Code (CSharp):

1. ...
2.  
3. // Initialize the recorder
4. _videoRecorder = new MP4Recorder(
5.     recordWidth,
6.     recordHeight,
7.     framerate,
8.     sampleRate,
9.     (int)AudioSettings.speakerMode
10. );
11.      
12. ...

I tried commenting out

_videoRecorder.CommitSamples(...)

line to try and isolate the issue, but that didn't stop the freeze from happening. It's fine if I change video recorder's sample rate and channel count to 0 though, which leaves me thinking that video recorder not getting a non-empty sample somewhere near recording start causes the lag.

 

And did you found a hard (or not easy) solution ?

 

Hello,

Maybe your `AudioMixerAttachment.cs` would be interesting for my needs. Is that possible to send it to me ?
My main issue is precisely to mix audio sources ..

Thanks in advance !

 

Sure thing, here it is: https://gist.github.com/UnimaginaryUnit/89cbcb2c45c24460f200ec227ef4bb0f
I'm not doing any resampling though, so you have to make sure that your in-engine audio has the same sample rate as your external audio. There's also some unexpected behavior I'm facing when recording inactive audio sources, which I described above. Apart from that - those scripts worked out fine for me, hope they will for you!

 

Hm commenting out `CommitSamples` will definitely reproduce this issue, but it should only ever happen on Windows (not other platforms). Also, just so we can isolate this to just NatCorder, can you use a plain `AudioInput` instead of your custom mixer?

 

Hey @Lanre Your plugin is awesome, working like a charm. I tried to record a video which is playing through unity's videoPlayer everything works fine except the audio. audio recorded using audioSource has a lots of noise in it. audio output mode in videoplayer setting is AudioSource and give a reference to audiosource we recorded. Please help me with issue. audio is fine in editor but in recorded clip it has lots of noise.

 

Hey there, does the recorded video itself have noise? The goal is to isolate this behaviour to either NatCorder or Unity's VideoPlayer. Can you share the raw recorded video?

 

https://drive.google.com/drive/folders/1aoJDBKhmKCY3pMCeJfitl0kai79HG5UI?usp=sharing here I uploaded 2 recorded video and original video for you to test.

 

The issue is indeed only reproducing on Windows! Using a plain `AudioInput` didn't stop the issue from re-occuring on Windows though, but it's not as critical for me, as my app targets mobile platforms. Thank you for the help!

 

@Lanre Hey there. I have problem with IOS 15
When I'm making GIF, and trying to save, I'm getting this error

finalizeDestination:3189: *** ERROR: image destination does not have enough images (6/2147483647)
Saved animated GIF image to: /var/mobile/Containers/Data/Application/E53709C0-B8DF-4705-91B8-DE2B5F5206FD/Documents/recording_2022_01_20_17_46_16_708.gif.


Function works great on IOS 14

What can be the problem? is it some sort of permission issue?

 

It sounds like the volume is very high. I recommend checking your setup to make sure that you don't have any audio settings that could be causing this.

 

Yeah on Windows it's a known issue.

 

It's a bug in iOS 15. I've added a workaround for the next NatCorder update.

 

Thanks you. Will wait for it.