NatCorder - Video Recording API

This sounds like a bug that used to happen on Metal (iOS or macOS) where the screen would freeze when recording was enabled. Unity fixed it in 2019.2. What version of Unity are you using?

 

I update Unity to 2020,this problem has solved. Thanks for your help.

 

Hi Lanre, I really need to screen record a Screen Space - Overlay Canvas. I know in your documentation it says that we can't https://docs.natsuite.io/natcorder/api/camerainput#creating-the-recorder-input but is there any way to work around this..? or any solution?

 

This isn't possible. It is a Unity restriction, not a NatCorder restriction. See this post.

 

Hi. Is 1.8.0 available ? Asset Store is still on 1.7.3

 

Not just yet, though it has been in a beta for quite some time. You can always PM or email me to get the build.

 

@Lanre

In the documentation it states that timestamps should be monotonic, does this mean that both video and audio commits need to be in chronological order?

It is also stated that "the very first timestamp for either an audio or video frame must be zero", is this for both first video and first audio frame? And in the case of both initially commits being at 0, does this mean we should be padding our silence if there is no audio initially?

Thanks

 

Yes, frames within a given track must be strictly monotonic (i.e, current frame timestamp > previous frame timestamp). This doesn't hold across tracks (video and audio), so one track can be 'behind' another, temporally speaking.

It's either-or, so any one.

Though I haven't tested this, if the time difference is negligible (only a few frames), then no need to commit silence.

 

Thanks @Lanre

 

@Lanre

I've done some debugging on the video issues we've seen and thought you migth be interested.

We've had issues on Android of our audio buffer going over the max of 2048 samples per channel, and my predecessor resolved this by comitting the remaining samples on the next frame. This seemed to work fine as the audio just continues on as if there is no space between audio commits.

However the time jump caused the video and audio to come out of sync and a shorter video. This was easy to resolve by just doing multiple commits per frame with the timstamp being offset by amount of nanoseconds the samples being commited prior to the remainder.

However this also highlighted another issue - not committing audio frames when there is silence. If I commit audio like follows

Code (CSharp):

1.  
2. VIDEO 0s --------------------------------------------------------------------- 40s
3. AUDIO 0s -- no commit -- 10s -- commit -- 20s -- no commit -- 30s -- commit -- 40s
4.  

Then on Windows / iOS / macOS, I'll end up with something like this, with silence filling up the end of the video.

Code (CSharp):

1.  
2. VIDEO 0s ----------------------------------------------------------------- 40s
3. AUDIO 0s -- no audio -- 10s -- audio -- 20s -- audio -- 30s -- no audio -- 40s
4.  

However on Android, it just shortens the video regardless of the video commits and skips the initial period where no audio samples were commited.

Code (CSharp):

1.  
2. VIDEO 10s -------------------------- 30s
3. AUDIO 0s -- audio -- 10s -- audio -- 20s
4.  

I'm going to try padding the audio samples with the appropriate amount of 0s when there is silence between commits

 

Are you limiting your sample buffers to not be longer than 2048 samples per channel? And how many channels is your audio data? On Android, the MP4 and HEVC recorders can accept sample buffers that have up to 16384 samples (so divide by the number of channels you have). This was increased from 8192 samples over 17 months ago.

Yup sounds like this should work.

This is pretty odd, and is likely an implementation detail from Android's `MediaMuxer`. NatCorder will wait until all encoded chunks from both encoders have been sent to the muxer before finalizing the session. So it must be that the muxer is truncating the streams to the shorter of the two.

 

No dropped samples and we have two channels. The plugin is told our sample rate and without looking at the our source I'm guessing it will output as that.

It will however occassionally feed us more samples so it could just be giving us the audio frames further into the future.

I think the change to limit by 2048 per channel when commiting was done quite a while back back in early development when an issue appeared, so if it has been increased since I'll give 8192 a go.

 

We're currently seeing an issue where a video recorded on a high end / latest iOS device records at the transcoding profile High@L5. This video then, when loaded on an Amazon Fire device, fails to play because that transcoding profile is not supported. The highest profile that device supports is High@L4. Is there a way to set the transcoding profile in Natcorder to something that's not the device's default?

 

Hi,

The first frame is black on Android.
I use Unity 2019.4 / NatCorder 1.7.3 with NatRender(GLESReadback)
when readback.request, NativeArray<byte> filled zero.

 

Hey there. This is an interesting one; I've never had it come up before. I'll look into exposing this as a setting, though I can't make any guarantees that it will be exposed. What I could do is make a custom build of libNatCorder.a (the iOS library) that hard-codes the encoding profile.

 

This sounds like a bug. Do you mind sharing your readback code? Also, when NatCorder 1.8.0 is released, you wouldn't have to use NatRender directly, as the NatCorder Performance Extensions (NCPX) library provides these constructs.

 

Thanks Lanre for the reply! A custom build of libNatCorder.a would be greatly appreciated. Would you be able to set the iOS profile to High@L4? That should work fine on the devices we're having issues with. And yes, in the future, it'd be great to have this configurable if possible.

Also, some context might be helpful! Our app allows you to record videos that are then uploaded to our server. A section of our app allows you to view these saved videos. These videos are associated with the user's account, so they can login to the same account on a different device. Thus, if they make a video on a newer iOS device, switch over to a lower end Kindle device using the same account, and try to view saved video - playback fails.

Thanks again!

 

We're investigating possibly adjusting our resolution of the video which might force it down to a lower transcoding profile Lanre. Will keep you posted!

 

I've use your NatRender (https://github.com/natsuite/NatRender)
And use NatCorder ReplayCam Sample scene.
Project Setting:
Auto Graphic API -> Off
Graphics API-> [OpenGLES3]
Multithreaded Rendering-> On
ScriptBackEnd-> IL2CPP
Target Architectures->[ARM64]

CameraInput.cs Constructor

Code (CSharp):

1.              
2. #if UNITY_EDITOR
3.             readback = new AsyncReadback();
4. #elif UNITY_ANDROID
5.             readback = new GLESReadback(width, height, true);
6. #elif UNITY_IOS
7.             readback = new MTLReadback(width, height, true);
8. #endif
9.  

CameraInput.cs OnFrame

Code (CSharp):

1.                 readback.Request(frameBuffer, buffer => {
2.                     if (pixelBuffer != null) {
3.                         buffer.CopyTo(pixelBuffer);    
4.                         recorder.CommitFrame(pixelBuffer, timestamp);
5.                     }
6.                 });

 

Hi.
There is an error.

Unity version : 2020.2.4f1

Platform : PC(Window standalone)

My source :
mediaRecorder = new MP4Recorder(1280, 720, 30f, 48000, 2);

error log :
: Unable to load DLL 'NatCorder': The specified module could not be found.
at NatSuite.Recorders.MP4Recorder+<>c__DisplayClass2_0.<.ctor>b__0 (NatSuite.Recorders.Internal.Bridge+RecordingHandler callback, System.IntPtr context) [0x00000] in <00000000000000000000000000000000>:0
.
.

Help me...

 

This would be the preferable solution so you don't keep needing custom NatCorder builds going into the future. Another option is to transcode videos on your server, an approach which is very common.

 

Can you PM me with your invoice number and email address? I'll send you the NatCorder 1.8.0 beta build which you can use with NCPX. If the issue is reproducible with NCPX, I'll open a bug report.

 

Hey Lanre! Just a follow up. Resolution is the solve here it seems. Basically we were recording, on higher end devices, a higher resolution. This pushes the transcoding profile to a more modern High@L5 which supports the higher resolution but is not compatible on lower end devices. If we record at a lower resolution, it automatically gets assigned the lower High@L4 transcoding profile which works on the lower end device. And yes, for existing videos that have this problem, we're going to do some backend transcoding for cross platform support. Thank a lot for you help!

 

I will send you the package. Please remove your invoice number because this information should not be public.

 

This sounds good. I'm glad you found an easy workaround!

 

I tested it by apply the 1.8.0 beta package with NCPX
using the GLESTextureInput in the ReplayCam example.
However, the problem with a black frame was not solved.

 

OK. Thankyou. I will wait.

 

In this case open an issue on the NCPX repo. Make sure to add information about what device and OS version you are running on when you create the issue.

 

For some reason, WeTransfer is failing to send you the file. Do you have another email address I can send it to?

 

jck9906@naver.com
Thank you

 

Sorry. It's my fault.
I changed the Architecture (in Build Settings) from x86 to x86_64 and it works fine.
Thank you!

 

Crash on Android - signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr (Flutter + Unity)
E/AndroidRuntime(25654): FATAL EXCEPTION: UnityMain
E/AndroidRuntime(25654): Process: com.example.example_flutter_natcorder, PID: 25654
E/AndroidRuntime(25654): java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
E/AndroidRuntime(25654): Version '2019.4.22f1 (9fdda2fe27ad)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
E/AndroidRuntime(25654): Build fingerprint: 'OnePlus/OnePlus3/OnePlus3T:9/PKQ1.181203.001/1911042108:user/release-keys'
E/AndroidRuntime(25654): Revision: '0'
E/AndroidRuntime(25654): ABI: 'arm64'
E/AndroidRuntime(25654): Timestamp: 2021-04-09 11:25:06+0530
E/AndroidRuntime(25654): pid: 25654, tid: 25796, name: UnityMain >>> com.example.example_flutter_natcorder <<<
E/AndroidRuntime(25654): uid: 10411
E/AndroidRuntime(25654): signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
E/AndroidRuntime(25654): x0 0000000000000000 x1 00000000000064c4 x2 0000000000000006 x3 0000000000000008
E/AndroidRuntime(25654): x4 000000000a013128 x5 000000000a013128 x6 000000000a013128 x7 00000000001fffff
E/AndroidRuntime(25654): x8 0000000000000083 x9 ac39d5a6e35d00c0 x10 fffffff87ffffbdf x11 ac39d5a6e35d00c0
E/AndroidRuntime(25654): x12 ac39d5a6e35d00c0 x13 fffffff87ffffbdf x14 00000000000000cb x15 0000007d6e3788c8
E/AndroidRuntime(25654): x16 0000007d6e36c2b0 x17 0000007d6e27e088 x18 0000007ce8800080 x19 0000000000006436
E/AndroidRuntime(25654): x20 00000000000064c4 x21 0000000000000083 x22 0000007cc8d25800 x23 0000007b8b8f8000
E/AndroidRuntime(25654): x24 0000000000000014 x25 00000000ffffffff x26 0000000000000012 x27 0000000000000005
E/AndroidRuntime(25654): x28 0000007ce80a5107 x29 0000007cbd7a67f0
E/AndroidRuntime(25654): sp 0000007cbd7a67b0 lr 0000007d6e26f4d0 pc 0000007d6e26f4f0
E/AndroidRuntime(25654):
E/AndroidRuntime(25654): backtrace:
E/AndroidRuntime(25654): #00 pc 00000000000224f0 /system/lib64/libc.so (abort+112) (BuildId: 055dd78cff796ef05668266325ca65a8)
E/AndroidRuntime(25654): #01 pc 000000000046e080 /system/lib64/libart.so (art::Runtime::Abort(char const*)+1208) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #02 pc 00000000000095a8 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+720) (BuildId: 2865ab4ce5ce5dc6a4f92b1fb8e61a31)
E/AndroidRuntime(25654): #03 pc 00000000002e9e6c /system/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1660) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #04 pc 00000000002ea0e8 /system/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+196) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #05 pc 000000000049ef7c /system/lib64/libart.so (art::Thread:ecodeJObject(_jobject*) const+700) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #06 pc 00000000000fcd10 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art:anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+96) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #07 pc 00000000000fc004 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art:anonymous namespace)::JniValueType)+580) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #08 pc 00000000000fb5a8 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art:anonymous namespace)::JniValueType*)+628) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #09 pc 00000000000ebb4c /system/lib64/libart.so (art:anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+688) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #10 pc 000000000004a720 <anonymous:0000007cc88ff000>
E/AndroidRuntime(25654):
E/AndroidRuntime(25654): at libc.abort(abort:112)
E/AndroidRuntime(25654): at libart.art::Runtime::Abort(char const*)(Abort:1208)
E/AndroidRuntime(25654): at libbase.android::base::LogMessage::~LogMessage()(~LogMessage:720)
E/AndroidRuntime(25654): at libart.art::JavaVMExt::JniAbort(char const*, char const*)(JniAbort:1660)
E/AndroidRuntime(25654): at libart.art::JavaVMExt::JniAbortF(char const*, char const*, ...)(JniAbortF:196)
E/AndroidRuntime(25654): at libart.art::Thread:ecodeJObject(_jobject*) const(DecodeJObject:700)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art:anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)96)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art:anonymous namespace)::JniValueType)580)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art:anonymous namespace)::JniValueType*)628)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)688)
E/AndroidRuntime(25654): at Unknown.0x7cc8949720(Unknown Source:0)
E/AndroidRuntime(25654): FATAL EXCEPTION: UnityMain
E/AndroidRuntime(25654): Process: com.example.example_flutter_natcorder, PID: 25654
E/AndroidRuntime(25654): java.lang.Error: FATAL EXCEPTION [UnityMain]
E/AndroidRuntime(25654): Unity version : 2019.4.22f1
E/AndroidRuntime(25654): Device model : OnePlus ONEPLUS A3003
E/AndroidRuntime(25654): Device fingerprint: OnePlus/OnePlus3/OnePlus3T:9/PKQ1.181203.001/1911042108:user/release-keys
E/AndroidRuntime(25654):
E/AndroidRuntime(25654): Caused by: java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
E/AndroidRuntime(25654): Version '2019.4.22f1 (9fdda2fe27ad)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
E/AndroidRuntime(25654): Build fingerprint: 'OnePlus/OnePlus3/OnePlus3T:9/PKQ1.181203.001/1911042108:user/release-keys'
E/AndroidRuntime(25654): Revision: '0'
E/AndroidRuntime(25654): ABI: 'arm64'
E/AndroidRuntime(25654): Timestamp: 2021-04-09 11:25:06+0530
E/AndroidRuntime(25654): pid: 25654, tid: 25796, name: UnityMain >>> com.example.example_flutter_natcorder <<<
E/AndroidRuntime(25654): uid: 10411
E/AndroidRuntime(25654): signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
E/AndroidRuntime(25654): x0 0000000000000000 x1 00000000000064c4 x2 0000000000000006 x3 0000000000000008
E/AndroidRuntime(25654): x4 000000000a013128 x5 000000000a013128 x6 000000000a013128 x7 00000000001fffff
E/AndroidRuntime(25654): x8 0000000000000083 x9 ac39d5a6e35d00c0 x10 fffffff87ffffbdf x11 ac39d5a6e35d00c0
E/AndroidRuntime(25654): x12 ac39d5a6e35d00c0 x13 fffffff87ffffbdf x14 00000000000000cb x15 0000007d6e3788c8
E/AndroidRuntime(25654): x16 0000007d6e36c2b0 x17 0000007d6e27e088 x18 0000007ce8800080 x19 0000000000006436
E/AndroidRuntime(25654): x20 00000000000064c4 x21 0000000000000083 x22 0000007cc8d25800 x23 0000007b8b8f8000
E/AndroidRuntime(25654): x24 0000000000000014 x25 00000000ffffffff x26 0000000000000012 x27 0000000000000005
E/AndroidRuntime(25654): x28 0000007ce80a5107 x29 0000007cbd7a67f0
E/AndroidRuntime(25654): sp 0000007cbd7a67b0 lr 0000007d6e26f4d0 pc 0000007d6e26f4f0
E/AndroidRuntime(25654):
E/AndroidRuntime(25654): backtrace:
E/AndroidRuntime(25654): #00 pc 00000000000224f0 /system/lib64/libc.so (abort+112) (BuildId: 055dd78cff796ef05668266325ca65a8)
E/AndroidRuntime(25654): #01 pc 000000000046e080 /system/lib64/libart.so (art::Runtime::Abort(char const*)+1208) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #02 pc 00000000000095a8 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+720) (BuildId: 2865ab4ce5ce5dc6a4f92b1fb8e61a31)
E/AndroidRuntime(25654): #03 pc 00000000002e9e6c /system/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1660) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #04 pc 00000000002ea0e8 /system/lib64/libart.so (art::JavaVMExt::JniAbortF(char const*, char const*, ...)+196) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #05 pc 000000000049ef7c /system/lib64/libart.so (art::Thread:ecodeJObject(_jobject*) const+700) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #06 pc 00000000000fcd10 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art:anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)+96) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #07 pc 00000000000fc004 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art:anonymous namespace)::JniValueType)+580) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #08 pc 00000000000fb5a8 /system/lib64/libart.so (art:anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art:anonymous namespace)::JniValueType*)+628) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #09 pc 00000000000ebb4c /system/lib64/libart.so (art:anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)+688) (BuildId: f1eeccb4c9a016b8729a9ad774977d5d)
E/AndroidRuntime(25654): #10 pc 000000000004a720 <anonymous:0000007cc88ff000>
E/AndroidRuntime(25654):
E/AndroidRuntime(25654): at libc.abort(abort:112)
E/AndroidRuntime(25654): at libart.art::Runtime::Abort(char const*)(Abort:1208)
E/AndroidRuntime(25654): at libbase.android::base::LogMessage::~LogMessage()(~LogMessage:720)
E/AndroidRuntime(25654): at libart.art::JavaVMExt::JniAbort(char const*, char const*)(JniAbort:1660)
E/AndroidRuntime(25654): at libart.art::JavaVMExt::JniAbortF(char const*, char const*, ...)(JniAbortF:196)
E/AndroidRuntime(25654): at libart.art::Thread:ecodeJObject(_jobject*) const(DecodeJObject:700)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::CheckInstance(art::ScopedObjectAccess&, art:anonymous namespace)::ScopedCheck::InstanceKind, _jobject*, bool)96)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::CheckPossibleHeapValue(art::ScopedObjectAccess&, char, art:anonymous namespace)::JniValueType)580)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art:anonymous namespace)::JniValueType*)628)
E/AndroidRuntime(25654): at libart.art:anonymous namespace)::CheckJNI::GetObjectClass(_JNIEnv*, _jobject*)688)
E/AndroidRuntime(25654): at Unknown.0x7cc8949720(Unknown Source:0)

 

Can you provide some context? When does the crash happen? What device and OS version? Can you share your recording code?

 

Hi.
Frame drop is too severe during recording.
1-2 seconds is fine. But after that, the app freezes (same in the editor)

Unity version : 2020.2.4f1
Platform : PC(Window standalone)
Renderpipeline : URP

My source :
private void StartRecord()
{
mediaRecorder = new MP4Recorder(1280, 720, 30f, 48000, 2);
realtimeClock = new RealtimeClock();
cameraInput = new CameraInput(mediaRecorder, realtimeClock, Camera.main);
audioInput = new AudioInput(mediaRecorder, realtimeClock, audioListener);
}

 

Hi @Lanre, We're having some issues pausing on Android. If we stop and start committing several times, regardless of the next commit timestamp it'll put a long delay in the video like we jumped a few seconds forward on our next committed timestamp.
However there doesn't seem to be a consistent way to produce it, the simplest way is just to start and stop several times.

 

There is usually freezing when you create a recorder that expects audio (as in your case) but don't commit any audio samples to the recorder. This is a weird Windows-only behaviour, and it's not clear why MediaFoundation does this. So for you, I would check that the audio listener is actually receiving/processing audio samples. Create a script, add an OnAudioFilterRead method, log the sample buffer size, and attach the script to your audio listener (and play to see the logs).

 

This happens if you aren't 'pausing' timestamps. Can you share your pause/resume code?

 

Our timestamps aren't driven by a realtime clock, but by the number of audio samples output by the audio engine we use. So when we pause, our audio engine is just not outputting samples anymore.

Code (CSharp):

1.     private void CommitAudioBuffer(float[] buffer)
2.     {
3.         Recorder.CommitSamples(buffer, Timestamp);
4.         int samples = buffer.Length / kNumAudioChannels;
5.         Timestamp += SamplesToNanoseconds(samples, kAudioSampleRate);
6.     }

Other than this, the only other thing we do is make sure the timestamps are monotonic so samples / frames won't be commit if the timestamp hasn't incremented.

A single pause seems fine (though I can't guarantee it without more testing), but if we pause and unpause several times that is when delays in the video and audio are introduced. However the delays aren't for every subsequent pause after they begin, it is seemingly random when they are added.
I don't believe them to be silence in the audio samples as the video itself freezes too, and it continues again from where it was before it was paused (and is still in sync with the audio) and no video frames after unpausing have been lost.

 

How are you handling pausing video frames? If the pauses in the video aren't exactly at the boundaries of when you pause recording in code, then that could be a separate issue. Can you share a video with the pause?

 

CommitVideoFrame is called at the end of a renderpass where we do extra rendering for the video frame. I'll look at producing a video with the issue that I can share.

Code (CSharp):

1.     public void CommitVideoFrame(ref CommandBuffer cmd, RenderTexture framebuffer)
2.     {
3.         long timestamp = Timestamp;
4.  
5.         timestamp = (VideoTimestamp == -1) ? 0 : timestamp; // first frame should always start at 0
6.  
7.         // this is here to be sure our video timestamps are monotonic
8.         if (timestamp == VideoTimestamp)
9.         {
10.             return;
11.         }
12.  
13.         VideoTimestamp = timestamp;
14.  
15.         if (SystemInfo.supportsAsyncGPUReadback)
16.         {
17.             cmd.RequestAsyncReadback(
18.                 framebuffer,
19.                 0,
20.                 request =>
21.                 {
22.                     var nativeArray = request.GetData<byte>();
23.                     CommitVideoFrame(nativeArray, timestamp);
24.                 }
25.             );
26.         }
27.         else
28.         {
29.             UnsupportedAsyncReadback_CommitVideoFrameAsyncEntry(framebuffer, timestamp);
30.         }
31.     }
32.  
33.     private async void UnsupportedAsyncReadback_CommitVideoFrameAsyncEntry(RenderTexture framebuffer, long timestamp)
34.     {
35.         // Wait for the next frame to readback so we don't delay the current frame
36.         await new WaitForEndOfFrame();
37.         await new WaitForUpdate();
38.  
39.         // Read pixels from RenderTexture
40.         var oldRT = RenderTexture.active;
41.         {
42.             RenderTexture.active = framebuffer;
43.             UnsupportedAsyncReadbackTexture.ReadPixels(new Rect(0, 0, UnsupportedAsyncReadbackTexture.width, UnsupportedAsyncReadbackTexture.height), 0, 0);
44.  
45.             var nativeArray = UnsupportedAsyncReadbackTexture.GetRawTextureData<byte>();
46.             CommitVideoFrame(nativeArray, timestamp);
47.         }
48.         RenderTexture.active = oldRT;
49.     }
50.  
51.     private void CommitVideoFrame(Unity.Collections.NativeArray<byte> nativeArray, long timestamp)
52.     {
53.         if (nativeArray.IsCreated)
54.         {
55.             unsafe
56.             {
57.                 IntPtr nativeArrayPtr = new IntPtr(
58.                     Unity.Collections.LowLevel.Unsafe.NativeArrayUnsafeUtility.GetUnsafePtr(nativeArray)
59.                 );
60.  
61.                 if (nativeArrayPtr != IntPtr.Zero)
62.                 {
63.                     Recorder.CommitFrame(nativeArrayPtr, timestamp);
64.                 }
65.             }
66.         }
67.     }

 

I should note that I like how your code is crafted to avoid making allocations; looks like you've hand-optimized recording. Anyway, I'm not sure that this is the relevant recording code. What I'm specifically looking to see is how you handle pausing and resuming, though it seems like since you're not using recorder inputs or clocks, you simply suspend sending frames to the recorder. How are the input timestamps calculated? I don't see any incrementing; only (re)assignment.

 

Ah sorry, this was mentioned in my second reply

Though I did miss the samples to nanoseconds method

Code (CSharp):

1.     public static long SamplesToNanoseconds(int samples, int sampleRate)
2.     {
3.         return (long)(((double)samples * 1000000000.0) / (double)sampleRate);
4.     }

So our audio engine is just paused and it no longer outputs samples while paused. We had to stop using the realtime clock as any hiccups in Unity would cause gaps in our audio samples which caused the video and audio to get out of sync enough to be noticeable, even if it only happened once at the beginning of recording.

 

This is for audio timestamps and it checks out. How about video?

 

Video is driven by audio, so it uses the audio timestamp, it is not as obvious as my quick edit just named it Timestamp instead of AudioTimestamp, but you can see long timestamp is assigned from it

 

This doesn't seem correct. Your audio timestamps are based on the sample rate (44.1KHz I think). Using this as a video timestamp would means your video stream is running at 44,100 frames per second. Your video timestamps need to be computed independently, based on the frame rate of the app (or however often you are committing video frames).

 

Sorry I don't quite understand what you mean. We're only submitting videos once a frame at most, the timestamp used for the video frame that gets committed is calculated using the number of audio samples we have committed overall and the sample rate (48k ) to determine how many nanoseconds worth of time that is.

So if before a video frame we have committed 48k samples (its actually a little more complicated as we use pending samples too from our buffer in the calculation), we know it should have a timestamp of 1 second which is 1e9 nanoseconds. This means our video frames are in sync with the audio engine's output.

 

Yeah this is what I don't understand. You have to confirm that your synthesized video timestamps aren't causing uneven timing in the video. This could easily be your culprit.

 

Yup, NatCorder supports ARFoundation (ARCore, ARKit), Wikitude, and Vuforia. I have an example project for ARFoundation with NatCorder in the works.

How you choose to start and stop recording is up to you. NatCorder is used programmatically, giving you full flexibility.

NatCorder always saves recordings to the app's private documents directory. You can use the free NatShare plugin to copy the recording to the camera roll on iOS and Android.

 

Hi Lanre! Just saw the Natcorder update. First, amazing. This actually seems to address some memory pressure we've been seeing on low-end Android devices. Question - We're targeting devices in the Amazon ecosystem that use Android 22. Is there a reason why this newer version targets 24 API and above? Is there anything we can do to lower the Android requirement to 22 on our end?

 

Hi, I found 2 problems using the latest NatCorder

My setup
Unity 2020.3.4 LTS
Natcorder 1.8
NCPX
Android IL2CPP, gles2, gles3

Natrender

1.On Android device, the first frame was always black.
the code used for recording

Code (CSharp):

1. recorder = new MP4Recorder(width, height, 30, microphone?.sampleRate ?? 0, microphone?.channelCount ?? 0);
2.         cameraInput = new CameraInput(recorder, clock, camList.ToArray());
3.         if (Application.platform == RuntimePlatform.Android)
4.             cameraInput.textureInput = new GLESTextureInput(recorder, multithreading: true);
5.         else if (Application.platform ==  RuntimePlatform.IPhonePlayer)
6.             cameraInput.textureInput = new MTLTextureInput(recorder, multithreading: true);
7.         microphone?.StartRunning((sampleBuffer, timestamp) => recorder.CommitSamples(sampleBuffer, clock.timestamp));

2.I have tried recording model with stencil buffer shader, but the record video result in black frame.
this is the log shown in android log cat

Spoiler: Logs?

2021/04/20 16:44:54.245 21097 21179 Info Unity Record video
2021/04/20 16:44:54.245 21097 21179 Info Unity CameraCaptureButton:RecordVideo()
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.Events.UnityAction:Invoke()
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.Events.UnityEvent:Invoke()
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.EventSystems.EventFunction`1:Invoke(T1, BaseEventData)
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocessTouchPress(PointerEventData, Boolean, Boolean)
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocessTouchEvents()
2021/04/20 16:44:54.245 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocess()
2021/04/20 16:44:54.245 21097 21179 Info Unity
2021/04/20 16:44:54.263 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 12 Seconds
2021/04/20 16:44:54.348 21097 21179 Debug MPEG4Writer PreAllocation disabled. fallocate : Operation not supported on transport endpoint, 95
2021/04/20 16:44:54.352 21097 23112 Info OMXClient IOmx service obtained
2021/04/20 16:44:54.357 21097 23112 Info ExtendedACodec setupVideoEncoder()
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c06 = 2141391878
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c04 = 2141391876
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c00 = 2141391872
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c09 = 2141391881
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c0a = 2141391882
2021/04/20 16:44:54.357 21097 23112 Warn OMXUtils do not know color format 0x7fa30c08 = 2141391880
2021/04/20 16:44:54.358 21097 23112 Warn OMXUtils do not know color format 0x7fa30c07 = 2141391879
2021/04/20 16:44:54.358 21097 23112 Warn OMXUtils do not know color format 0x7f000789 = 2130708361
2021/04/20 16:44:54.358 21097 23112 Info ACodec [OMX.qcom.video.encoder.avc] using color format 0x15 in place of 0x7f420888
2021/04/20 16:44:54.359 21097 23112 Warn AString ctor got NULL, using empty string instead
2021/04/20 16:44:54.359 21097 23112 Info ACodec setupAVCEncoderParameters with [profile: Baseline] [level: Level1]
2021/04/20 16:44:54.359 21097 23112 Info ACodec [OMX.qcom.video.encoder.avc] cannot encode HDR static metadata. Ignoring.
2021/04/20 16:44:54.359 21097 23112 Info ACodec setupVideoEncoder succeeded
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec [OMX.qcom.video.encoder.avc] configure, AMessage : AMessage(what = 'conf', target = 11) = {
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t color-format = 2135033992
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t i-frame-interval = 2
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec string mime = "video/avc"
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t width = 896
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t bitrate = 10000000
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec float frame-rate = 30.000000
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t height = 1920
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t flags = 1
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec int32_t encoder = 1
2021/04/20 16:44:54.359 21097 23112 Info ExtendedACodec }
2021/04/20 16:44:54.360 21097 23112 Info ACodec [OMX.qcom.video.encoder.avc] input format is AMessage(what = 0x00000000) = {
2021/04/20 16:44:54.360 21097 23112 Info ACodec string mime = "video/raw"
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t stride = 896
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t slice-height = 1920
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t color-format = 21
2021/04/20 16:44:54.360 21097 23112 Info ACodec Buffer *image-data = 0xb400006f8ce5edc0
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t color-range = 0
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t color-standard = 0
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t color-transfer = 0
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t width = 896
2021/04/20 16:44:54.360 21097 23112 Info ACodec int32_t height = 1920
2021/04/20 16:44:54.360 21097 23112 Info ACodec }
2021/04/20 16:44:54.362 21097 23112 Info ACodec [OMX.qcom.video.encoder.avc] output format is AMessage(what = 0x00000000) = {
2021/04/20 16:44:54.362 21097 23112 Info ACodec int32_t bitrate = 10000000
2021/04/20 16:44:54.362 21097 23112 Info ACodec int32_t max-bitrate = 10000000
2021/04/20 16:44:54.362 21097 23112 Info ACodec int32_t frame-rate = 30
2021/04/20 16:44:54.362 21097 23112 Info ACodec string mime = "video/avc"
2021/04/20 16:44:54.362 21097 23112 Info ACodec int32_t width = 896
2021/04/20 16:44:54.362 21097 23112 Info ACodec int32_t height = 1920
2021/04/20 16:44:54.362 21097 23112 Info ACodec }
2021/04/20 16:44:54.365 21097 21179 Verbose NatSuite NatCorder: MP4Recorder prepared video encoder with format: {color-format=2135033992, i-frame-interval=2, mime=video/avc, width=896, bitrate=10000000, frame-rate=30.0, height=1920}
2021/04/20 16:44:54.367 21097 23118 Debug CCodec allocate(c2.android.aac.encoder)
2021/04/20 16:44:54.368 21097 23118 Info Codec2Client Available Codec2 services: "default" "software"
2021/04/20 16:44:54.369 21097 23118 Info CCodec setting up 'default' as default (vendor) store
2021/04/20 16:44:54.371 21097 23118 Info CCodec Created component [c2.android.aac.encoder]
2021/04/20 16:44:54.371 21097 23118 Debug CCodecConfig read media type: audio/mp4a-latm
2021/04/20 16:44:54.372 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: algo.buffers.max-count.values
2021/04/20 16:44:54.372 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: output.subscribed-indices.values
2021/04/20 16:44:54.372 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: input.buffers.allocator-ids.values
2021/04/20 16:44:54.372 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: output.buffers.allocator-ids.values
2021/04/20 16:44:54.373 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: algo.buffers.allocator-ids.values
2021/04/20 16:44:54.373 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: output.buffers.pool-ids.values
2021/04/20 16:44:54.373 21097 23118 Debug ReflectedParamUpdater extent() != 1 for single value type: algo.buffers.pool-ids.values
2021/04/20 16:44:54.373 21097 23118 Info CCodecConfig query failed after returning 9 values (BAD_INDEX)
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2 config diff is Dict {
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 coded.bitrate.value = 64000
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 coded.pl.level = 0
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 coded.pl.profile = 8192
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 coding.aac-sbr-mode.value = 3
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 input.buffers.max-size.value = 2048
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 input.delay.value = 0
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig string input.media-type.value = "audio/raw"
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig string output.media-type.value = "audio/mp4a-latm"
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 raw.channel-count.value = 1
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig c2::u32 raw.sample-rate.value = 44100
2021/04/20 16:44:54.373 21097 23118 Debug CCodecConfig }
2021/04/20 16:44:54.374 21097 23118 Debug CCodec [c2.android.aac.encoder] buffers are bound to CCodec for this session
2021/04/20 16:44:54.374 21097 23118 Debug CCodecConfig no c2 equivalents for channel-mask
2021/04/20 16:44:54.374 21097 23118 Debug CCodecConfig no c2 equivalents for aac-profile
2021/04/20 16:44:54.374 21097 23118 Debug CCodecConfig no c2 equivalents for flags
2021/04/20 16:44:54.374 21097 23118 Debug CCodecConfig no c2 equivalents for encoder
2021/04/20 16:44:54.374 21097 23118 Warn Codec2Client query -- param skipped: index = 1107298332.
2021/04/20 16:44:54.374 21097 23118 Debug CCodec setup formats input: AMessage(what = 0x00000000) = {
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t aac-sbr-mode = 3
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t channel-count = 1
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t max-input-size = 16384
2021/04/20 16:44:54.374 21097 23118 Debug CCodec string mime = "audio/raw"
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t sample-rate = 44100
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t channel-mask = 16
2021/04/20 16:44:54.374 21097 23118 Debug CCodec } and output: AMessage(what = 0x00000000) = {
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t aac-sbr-mode = 3
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t bitrate = 64000
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t channel-count = 1
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t level = 0
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t max-bitrate = 64000
2021/04/20 16:44:54.374 21097 23118 Debug CCodec string mime = "audio/mp4a-latm"
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t profile = 2
2021/04/20 16:44:54.374 21097 23118 Debug CCodec int32_t sample-rate = 44100
2021/04/20 16:44:54.374 21097 23118 Debug CCodec }
2021/04/20 16:44:54.374 21097 21179 Verbose NatSuite NatCorder: MP4Recorder prepared audio encoder with format: {channel-mask=16, sample-rate=44100, mime=audio/mp4a-latm, channel-count=1, bitrate=64000, aac-profile=2, max-input-size=16384}
2021/04/20 16:44:54.389 21097 23118 Warn Codec2Client query -- param skipped: index = 1342179345.
2021/04/20 16:44:54.389 21097 23118 Warn Codec2Client query -- param skipped: index = 2415921170.
2021/04/20 16:44:54.390 21097 23118 Error FMQ grantorIdx must be less than 3
2021/04/20 16:44:54.390 21097 23118 Error FMQ grantorIdx must be less than 3
2021/04/20 16:44:54.390 21097 23118 Debug CCodecBufferChannel [c2.android.aac.encoder#795] Created input block pool with allocatorID 16 => poolID 17 - OK (0)
2021/04/20 16:44:54.391 21097 23118 Info CCodecBufferChannel [c2.android.aac.encoder#795] Created output block pool with allocatorID 16 => poolID 517 - OK
2021/04/20 16:44:54.391 21097 23118 Debug CCodecBufferChannel [c2.android.aac.encoder#795] Configured output block pool ids 517 => OK
2021/04/20 16:44:54.391 21097 23118 Debug CCodecBufferChannel [c2.android.aac.encoder#795] start: updating output delay 0
2021/04/20 16:44:54.391 21097 23118 Error ion ioctl c0044901 failed with code -1: Inappropriate ioctl for device
2021/04/20 16:44:54.406 21097 21179 Debug AudioRecord set(): inputSource 1, sampleRate 44100, format 0x1, channelMask 0x10, frameCount 1792, notificationFrames 0, sessionId 0, transferType 0, flags 0, opPackageName com.spacears.space.android uid -1, pid -1
2021/04/20 16:44:54.413 21097 21179 Debug AudioRecord start(602): sync event 0 trigger session 0, mSessionID=4377
2021/04/20 16:44:54.424 21097 21179 Debug AudioService broadcastRecorderState:com.spacears.space.android
2021/04/20 16:44:54.424 21097 21179 Verbose NatSuite NatDevice: AudioDevice 1 started recording with format: 1@44100Hz
2021/04/20 16:44:54.426 21097 21179 Info Unity Capture Video Screenshot done, start recording....
2021/04/20 16:44:54.426 21097 21179 Info Unity <DoCaptureThumbnailForVideo>d__28:MoveNext()
2021/04/20 16:44:54.426 21097 21179 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2021/04/20 16:44:54.426 21097 21179 Info Unity
2021/04/20 16:44:54.478 21097 23112 Info ACodec [OMX.qcom.video.encoder.avc] output format is AMessage(what = 0x00000000) = {
2021/04/20 16:44:54.478 21097 23112 Info ACodec int32_t bitrate = 10000000
2021/04/20 16:44:54.478 21097 23112 Info ACodec int32_t max-bitrate = 10000000
2021/04/20 16:44:54.478 21097 23112 Info ACodec int32_t frame-rate = 30
2021/04/20 16:44:54.478 21097 23112 Info ACodec string mime = "video/avc"
2021/04/20 16:44:54.478 21097 23112 Info ACodec int32_t width = 896
2021/04/20 16:44:54.478 21097 23112 Info ACodec int32_t height = 1920
2021/04/20 16:44:54.478 21097 23112 Info ACodec }
2021/04/20 16:44:54.484 21097 23123 Error Utils csd0 too small
2021/04/20 16:44:54.484 21097 23123 Error ExtendedUtils csd0 too small
2021/04/20 16:44:54.484 21097 23123 Debug MPEG4Writer fpathconf _PC_FILESIZEBITS:64
2021/04/20 16:44:54.484 21097 23123 Debug MPEG4Writer File size limit set to 4503599627370495 bytes implicitly
2021/04/20 16:44:54.484 21097 23123 Debug MPEG4Writer MP4WtrCtrlHlpLooper Started
2021/04/20 16:44:54.484 21097 23123 Info MPEG4Writer limits: 4503599627370495/0 bytes/us, bit rate: -1 bps and the estimated moov size 3192 bytes
2021/04/20 16:44:54.484 21097 23132 Debug MPEG4Writer kWhatNoIOErrorSoFar
2021/04/20 16:44:54.487 21097 23133 Info MPEG4Writer setStartTimestampUs: 0
2021/04/20 16:44:54.487 21097 23133 Info MPEG4Writer Earliest track starting time: 0
2021/04/20 16:44:54.501 21097 22899 Error native motion_tracking_context.cc:1331 Unable to find measurements corresponding to VIO status at timestamp 25781854800335.
2021/04/20 16:44:54.536 21097 23118 Error FMQ grantorIdx must be less than 3
2021/04/20 16:44:54.539 21097 22854 Error FMQ grantorIdx must be less than 3
2021/04/20 16:44:54.540 21097 23134 Info MPEG4Writer setStartTimestampUs: 103892
2021/04/20 16:44:55.279 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 13 Seconds
2021/04/20 16:44:55.707 21097 22899 Error native motion_tracking_context.cc:1331 Unable to find measurements corresponding to VIO status at timestamp 25783054583928.
2021/04/20 16:44:56.291 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 14 Seconds
2021/04/20 16:44:56.653 21097 22984 Info native motion_analysis_calculator.cc:626 Analyzed frame 801
2021/04/20 16:44:57.307 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 15 Seconds
2021/04/20 16:44:57.667 21097 23119 Debug CCodecBufferChannel [c2.android.aac.encoder#795] DEBUG: elapsed: n=4 [in=0 pipeline=0 out=0 smoothness=4]
2021/04/20 16:44:57.667 21097 23119 Debug PipelineWatcher DEBUG: elapsed 0 / 4
2021/04/20 16:44:58.323 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 16 Seconds
2021/04/20 16:44:59.339 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 17 Seconds
2021/04/20 16:44:59.390 21097 23121 Debug BufferPoolAccessor2.0 bufferpool2 0xb400006fb6609828 : 5(81920 size) total buffers - 4(65536 size) used buffers - 0/5 (recycle/alloc) - 4/240 (fetch/transfer)
2021/04/20 16:44:59.390 21097 23121 Debug BufferPoolAccessor2.0 evictor expired: 1, evicted: 1
2021/04/20 16:44:59.986 21097 22984 Info native motion_analysis_calculator.cc:626 Analyzed frame 901
2021/04/20 16:45:00.281 21097 21179 Error ACameraMetadata getConstEntry: cannot find metadata tag 65578
2021/04/20 16:45:00.355 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 18 Seconds
2021/04/20 16:45:01.000 21097 23119 Debug CCodecBufferChannel [c2.android.aac.encoder#795] DEBUG: elapsed: n=4 [in=0 pipeline=0 out=0 smoothness=4]
2021/04/20 16:45:01.000 21097 23119 Debug PipelineWatcher DEBUG: elapsed 0 / 4
2021/04/20 16:45:01.371 21097 21266 Info AudioTrack isLongTimeZoreData zoer date time 19 Seconds
2021/04/20 16:45:01.700 21097 22899 Error native motion_tracking_context.cc:1331 Unable to find measurements corresponding to VIO status at timestamp 25789053501741.
2021/04/20 16:45:01.809 21097 23130 Debug AudioRecord stop mSessionID=4377
2021/04/20 16:45:01.809 21097 23130 Debug AudioRecord stop(602): mActive:1
2021/04/20 16:45:01.844 21097 23130 Debug AudioService broadcastRecorderState:com.spacears.space.android
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop mSessionID=4377
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop(602): mActive:0
2021/04/20 16:45:01.844 21097 23130 Debug AudioService broadcastRecorderState:com.spacears.space.android
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop mSessionID=4377
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop(602): mActive:0
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop mSessionID=4377
2021/04/20 16:45:01.844 21097 23130 Debug AudioRecord stop(602): mActive:0
2021/04/20 16:45:01.844 21097 23130 Verbose NatSuite NatDevice: AudioDevice 1 stopped recording
2021/04/20 16:45:01.855 21097 23123 Verbose NatSuite NatCorder: MP4Recorder video encoder encountered EOS
2021/04/20 16:45:01.855 21097 23124 Verbose NatSuite NatCorder: MP4Recorder audio encoder encountered EOS
2021/04/20 16:45:01.856 21097 23118 Debug CCodecBufferChannel [c2.android.aac.encoder#795] MediaCodec discarded an unknown buffer
2021/04/20 16:45:01.856 21097 23118 Info chatty uid=10374(com.spacears.space.android) identical 1 line
2021/04/20 16:45:01.856 21097 23118 Debug CCodecBufferChannel [c2.android.aac.encoder#795] MediaCodec discarded an unknown buffer
2021/04/20 16:45:01.856 21097 23118 Info hw-BpHwBinder onLastStrongRef automatically unlinking death recipients
2021/04/20 16:45:01.859 21097 21179 Info Unity Stop record video
2021/04/20 16:45:01.859 21097 21179 Info Unity CameraCaptureButton:StopRecord()
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.Events.UnityAction:Invoke()
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.Events.UnityEvent:Invoke()
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.EventSystems.EventFunction`1:Invoke(T1, BaseEventData)
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocessTouchPress(PointerEventData, Boolean, Boolean)
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocessTouchEvents()
2021/04/20 16:45:01.859 21097 21179 Info Unity UnityEngine.EventSystems.StandaloneInputModulerocess()
2021/04/20 16:45:01.859 21097 21179 Info Unity
2021/04/20 16:45:01.872 21097 21179 Info ExoPlayerImpl Init bdc1fce [ExoPlayerLib/2.8.4] [apollo, M2007J3SG, Xiaomi, 30]
2021/04/20 16:45:01.877 21097 23123 Debug MPEG4Writer reset()
2021/04/20 16:45:01.877 21097 23123 Debug MPEG4Writer Audio track stopping. Stop source
2021/04/20 16:45:01.877 21097 23133 Verbose MediaWriter Track event err/info msg:101, trackId:1, type:1000,val:-1011
2021/04/20 16:45:01.877 21097 23133 Info MPEG4Writer Received total/0-length (220/0) buffers and encoded 220 frames. - Video
2021/04/20 16:45:01.878 21097 23134 Verbose MediaWriter Track event err/info msg:101, trackId:2, type:1000,val:-1011
2021/04/20 16:45:01.878 21097 23134 Info MPEG4Writer Received total/0-length (313/0) buffers and encoded 313 frames. - Audio
2021/04/20 16:45:01.878 21097 23134 Info MPEG4Writer Audio track drift time: 0 us
2021/04/20 16:45:01.878 21097 23123 Debug MPEG4Writer Audio track source stopping
2021/04/20 16:45:01.878 21097 23123 Debug MPEG4Writer Audio track source stopped
2021/04/20 16:45:01.878 21097 23123 Debug MPEG4Writer Audio track stopped. Status:0. Stop source
2021/04/20 16:45:01.878 21097 23123 Debug MPEG4Writer Video track stopping. Stop source
2021/04/20 16:45:01.878 21097 23123 Debug MPEG4Writer audio will wait 0ns for video to stop
2021/04/20 16:45:01.879 21097 23123 Warn MPEG4Writer Timed-out waiting for video track to reach final audio timestamp !
2021/04/20 16:45:01.880 21097 23123 Debug MPEG4Writer Video track source stopping
2021/04/20 16:45:01.880 21097 23123 Debug MPEG4Writer Video track source stopped
2021/04/20 16:45:01.880 21097 23123 Debug MPEG4Writer Video track stopped. Status:0. Stop source
2021/04/20 16:45:01.880 21097 23123 Debug MPEG4Writer Duration from tracks range is [7333244, 7375738] us
2021/04/20 16:45:01.880 21097 23131 Debug MPEG4Writer 0 chunks are written in the last batch
2021/04/20 16:45:01.883 21097 23123 Debug MPEG4Writer WriterThread stopped. Status:0
2021/04/20 16:45:01.883 21097 23123 Info MPEG4Writer Adjust the moov start time from 0 us -> 0 us
2021/04/20 16:45:01.885 21097 23123 Info MPEG4Writer The mp4 file will not be streamable.
2021/04/20 16:45:01.885 21097 23123 Info MPEG4Writer MOOV atom was written to the file
2021/04/20 16:45:01.885 21097 23123 Debug MPEG4Writer release()
2021/04/20 16:45:01.896 21097 23123 Debug MPEG4Writer MP4WtrCtrlHlpLooper stopped
2021/04/20 16:45:01.896 21097 23123 Debug MPEG4Writer Top 5 write durations(microseconds): #1:53 #2:57 #3:67 #4:87 #5:1317
2021/04/20 16:45:01.896 21097 23123 Debug MPEG4Writer reset()
2021/04/20 16:45:01.899 21097 23123 Debug MPEG4Writer Video track stopping. Stop source
2021/04/20 16:45:01.899 21097 23123 Error MPEG4Writer Stop() called but track is not started or stopped
2021/04/20 16:45:01.899 21097 23123 Debug MPEG4Writer Audio track stopping. Stop source
2021/04/20 16:45:01.899 21097 23123 Error MPEG4Writer Stop() called but track is not started or stopped
2021/04/20 16:45:01.913 21097 21179 Info Unity Saved recording to: /storage/emulated/0/Android/data/com.spacears.space.android/files/recording_2021_04_20_16_44_54_342.mp4
2021/04/20 16:45:01.913 21097 21179 Info Unity <StopRecording>d__26:MoveNext()

EDIT I just tried Natcorder 1.8 without NCPX and NatRender, the frame is still black
EDIT2 Here is an example video with black frame

Spoiler

the result video