NatCorder - Video Recording API

Can you share the full (unfiltered) logs from logcat in a .txt file? I need to see everything from app start. I haven't had an error like this come up in over a year now.

 

Hi!

I'm having a trouble. All my videos in Android are awesome, but in iOS they are images not videos... I'm using MP4 recorder, I know that iOS records in M4A but, what can I do?

[EDIT]
Some of the videos dont appear in my camera roll... that was happening to me on Android and I moved the videos to my persistent data folder but in iOS I can't do that.

 

I think Unity 2019.4.0f1 LTE just introduced some new issues: replay recordings that were working properly until Unity 2019.3.15f1 now start consuming more and more RAM, killing the game on iOS in a few seconds, or compromising Unity Editor (on Windows, at least) by increasing its memory allocation by several GB in few hundreds of frames.

When analysing it using the profiler, the increase in memory usage is not related to textures/sprites/gameobjects, and even after stopping the game, the editor doesn't release this amount of memory.

I tried committing frames by providing both IntPtr and texture data. I think it should somehow be related to NativeArrays not being properly freed up.

I tested the project again with Unity 2019.3.15f1 and the issue didn't show up.
I tried to test the downgraded version on iOS as well but I'm facing a bug with incorrect asset versioning (I'll try to sort it out again tomorrow).

I strongly suspect it could be due to some regression introduced when fixing this: https://issuetracker.unity3d.com/is...a-copy-to-the-same-nativearray-multiple-times

 

Can you share your recording code, the logs from Xcode, and the recorded video file? You can get the video by sharing it to your computer or using iTunes (building your app with the ability for iTunes to browse its documents).

 

Yup definitely sounds like a regression. I'm surprised because the LTS should contain all the fixes till this point. I recommend sending a bug report to Unity.

 

Sorry, I wasted too many days creating and submitting repros to the Unity team, and despite being accepted as bugs none of them has been fixed.

Not going to waste anymore time on that.

 

I have:

var path = await recorder.FinishWriting();
Debug.Log(path);

#if UNITY_ANDROID
System.IO.File.Move(path, Application.persistentDataPath + "/" + System.IO.Path.GetFileName(path));
#endif

In Android works fine with moving the video to persistentDataPath, in iOS it takes the video like it's image and without that line of code, the video just doesnt appear in my camera roll.

There's no errors, with printing the path I can see it alright, but I cant get it into my camera roll.

[EDIT]
/var/mobile/Containers/Data/Application/D8B9A5E2-277A-4491-AC8C-1B5C789C3F15/Documents/recording_2020_06_17_16_15_20_368.mp4

Thats my path

 

That's fair. Consider rolling back to 2019.3.

 

You have to copy the image to the camera roll with a native gallery plugin. You can try using NatShare.

 

Hi @Lanre! Sorry for off topic. Can you check email please? I couldn't download your build with fixes.
https://forum.unity.com/threads/natcorder-video-recording-api.505146/page-61#post-5952728

 

Hey there I must have missed your email. NatDevice 1.0.2 is out.

 

I can confirm that iOS builds on Unity 2019.3.15f1 don't have any issues on freeing up allocated memory.

 

Hi @Lanre

We would like to run complex simulations in standalone builds on Windows, record them, and play them back at a constant 30 fps. We want to commit a frame precisely every .033 seconds in game time even if each frame takes five times as long to render in real time. Is NatCorder the best solution in this scenario? Can NatCorder express any control over the timestep for each frame in-game, or does this need to be handled by other Unity functions? If the answer to either question is no, could you offer us a pointer in the right direction? Many thanks.

 

NatCorder supports offline recording, along with multithreaded recording, so you should be able to do this with ease.

 

Hi Lanre, I am facing an issue in the audio of the recorded video. The audio of the recorded video plays like it is fast-forwarded. I checked the pitch value of the audio source in the scene and it is perfectly fine. The script for the recorder is much similar to the ReplayCam recorder script. Check that below,

Code (CSharp):

1. public class vrecorder : MonoBehaviour
2. {
3. private IMediaRecorder recorder;
4. private CameraInput cameraInput;
5. private AudioInput audioInput;
6. public AudioListener audioListener;
7. private UnityEvent savePath;
8.  
9. #if (UNITY_IPHONE && !UNITY_EDITOR)
10.     [DllImport("__Internal")]
11.     private static extern void on_video_record_complete(string path);
12. #endif
13.  
14.  
15. private void Start()
16.     {
17.         savePath = new UnityEvent();
18.         savePath.AddListener(RecordedPath);      
19.     }
20.  
21. public void StartRecording{
22. var clock = new RealtimeClock();
23. recorder = new MP4Recorder(720, 1280, 30, 44100, 2, bitrate: 6_000_000, keyframeInterval: 2);
24.             // Create recording inputs
25.             cameraInput = new CameraInput(recorder, clock, Camera.main);
26.             audioInput = new AudioInput(recorder, clock, audioListener);
27. }
28.  
29. public void StopRecording()
30.     {
31.         Debug.Log("StopRecording");
32.         audioInput.Dispose();
33.         Debug.Log("Audio input disposed");
34.         cameraInput.Dispose();
35.         Debug.Log("Camera Input Disposed");
36.         savePath.Invoke();      
37.     }
38.  
39.     public async void RecordedPath()
40.     {
41.         Debug.Log("Video file written to a path");
42.         recordedPath = await videoRecorder.FinishWriting();
43.         Debug.Log("Invoking OnReplay method");
44.         Invoke("CallOnReplay", 1f);
45.     }
46.  
47.     public void CallOnReplay(){
48.         Debug.Log("Recording path received. Path string:  "+recordedPath);    
49.         OnReplay(recordedPath);
50.     }
51.  
52. public void OnReplay(string path)
53.     {
54.         Debug.Log("Saved recording to: " + path);
55.         // iOS native callback
56. #if (UNITY_IPHONE && !UNITY_EDITOR)
57.         on_video_record_complete(path);
58. #endif
59. }
60. }

The recorded path is sent as a callback to the native iOS build. From there, the replay will be played. I have added a couple of videos, one is the screen record of the actual audio in the app and it is recorded playback audio. The other one is the actual recorded video file itself. I am only using scene audio.

The version details are given below.
Unity 2019.2.5f1
Xcode 11.3
Natcorder 1.7.2
iPhone 6S iOS version 12.4
Mac OS Catalina version 10.15.3

Also, I noted few points, which I assume the root cause would be the same

1. Sometimes recording preview screen delayed around 5-10 secs
2. The recording preview screen will not be displayed sometimes.

Need your help on this. If you need further information, please let me know.

 

You are hardcoding the audio format to 44100Hz with 2 channels. You should be using whatever format Unity's audio engine is using.

 

Thank you Lanre. Changing it to Unity audio settings fixed the issue.

 

Hi Lanre ,
i m facing issue of object shader, actully i want to recording shader on my way. but not happening..in android devices..
in PC working perfect.

 

Can you be more descriptive? I'm not sure what you are referring to at all. Please post as much information as you can: screenshots, your recording code, the logs, and anything else that might be useful.

 

Hello. I have few question.
Im using unity 2019.3.15. Also i use Natcorder(NatSuit) last version, And NatMic (old version 1.3.1)
Platform Android/IOS, App - AR application with ARfoundation
1)Does Natsuit work without NatMic? Should i delete old NatMic Plugin?
2) Recorded video has very low volume. how can i fix it? I read here to create AudioSource near the AudioListener. I tried it. but wherever i create AudioSource. result is same. Volume on AudioSorce is MAX.

Thanks.

 

Just to be clear of any confusion: NatCorder is the video recording API; NatDevice is the media device API (which replaces NatMic); and NatSuite (the NatSuite Framework) is the umbrella that all the API's are under. There is no NatSuite plugin.

I presume you are asking if NatCorder works without NatMic. Yes, it does. The ReplayCam example demonstrates how to record with microphone audio from Unity.

This would be from either the microphone itself or from Unity's microphone API. NatCorder will never touch the video samples.

 

ok Thanks.
Why i updated natcorder version - When i used old version of natcorder ~1.5v, i got blue color in my recorded video on IOS 13.5. on Android all was good(volume too, and i cant understand what is the volume problem with new version) .

Have you any suggestion how to solve that blue screen?

 

I'm not familiar with the blue screen issue. In any case I always recommend using the latest version of the API. As for the low volume, it must be coming from Unity's Microphone API. As I mentioned previously, NatCorder does not touch the audio samples in any way, and it certainly doesn't attenuate them.

 

I meant blue filter. recorded videos are under blue filter. there was few question about it in forum. but you asked to email you. so i cant find solution for it . old version working great on Android just that "blue filter" on ios. if i could fix it, that would be awesome.

 

Ah I see. Upgrade to the latest version of the API; the issue has been fixed.

 

ok. just last question. is there any way to fix blue filter issue on olde version. because i still can resolve microphone volume problem ))))

 

The only way is to update to the latest version.

 

hello there
is it possible to save recorded video to gallery in android and ios
or can I move it to gallery after save?

 

You'll have to use a native gallery plugin for this. I recommend NatShare.

 

in ios there is only one folder for pictures and videos but in android there are multiple folders can I create a folder with my apps name and save it in there with NatShare

 

You can't create folders anymore as of Android Q. NatShare uses Android's MediaStore API's for saving items to the camera roll; we don't manipulate files or folders.

 

yes, possible using android native gallery(available on assets store with doc.)

 

https://drive.google.com/file/d/11pqZ6drCEGksuwZT4bdmSrXNU0yBv966/view

https://drive.google.com/file/d/1J6OKs9tn2u5yUJlyW2FwX-2xuNNZ4yVs/view?usp=sharing

yes, 1st image for recording hair shade diffrence and 2nd image for start my recording..!!
in hair shader image you can see when mp4 record and play that video not as i set before

 

New version is totally different )))
May i ask your help. Where i can change code, so Natcorder will use NatMic.
Where i should put this code part AudioDevice.GetDevices()[0];

 

I have no clue what could be going on here. It seems to be specific to whatever your hair shader is doing. The `CameraInput` script does not affect what gets rendered in any way, and you can check the source to verify this. All it does is re-render the camera to a texture so that the pixel data can be sent to the video encoder.

 

I recommend upgrading to NatDevice. There is example code on GitHub.

 

Hey Lanre! Hope you're well man.

Me and another developer have spent hours trying to get pause recording/resume functionality on the EnoxSoftware/Natcorder example project.

I emailed you a pause/resume recording problem, but on the off chance someone else might need this functionality in that example project, you can write here if you'd like!

EnoxSoftware/NatCorder link: https://github.com/EnoxSoftware/NatCorderWithOpenCVForUnityExample
NatCorder Pause/Resume link: https://medium.com/natsuite/natcorder-and-natmic-a-crash-course-221e58bc3525

Sadly, we could not bring home the bacon, as I'm 100% absolutely sure you would say IRL.

Any thoughts on what object we're "freezing time" with and what to dispose/re-initialize to continue recording?

Your inbox has a script of ours in case you need more info. Thanks man!

 

Thanks for this extremely kind and informative reply.

I'm happy to discover this bug has already been fixed (I'm going to download 2019.4.1f1 right now and double-check it, but you're totally right, it really seems to be the same issue I faced).

I agree with everything you wrote, and I actually saw huge improvements in issue tracking/fixing in the last couple of years. Unfortunately I often faced bugs that are very difficult to spot or reproduce, and this makes things extremely complicated since it becomes extremely time-consuming to create a repro AND it's often considered a low priority bug due to its reduced number of affected projects.

If you still want to give it a try, the one I care the most since it's a real game-stopper for us is this: Issue tracker - Forum discussion.

Thanks again for your post!

 

Hi. i have this error on ios, when trying to record video.
Uncaught exception: NSInvalidArgumentException: *** -[AVAssetWriterInput initWithMediaTypeutputSettings:sourceFormatHint:] AVVideoCompressionPropertiesKey dictionary must specify a positive value for AVVideoMaxKeyFrameIntervalKey

Any idea how to solve?

Update: Fixed by changing keyframes from 0 to dafult(3), when creating MP4Recording.

 

When you pause recording, you are simply suspending sending frames to the recorder for encoding. This is why you'll want to delete any recorder inputs you have, because they continuously send video and audio frames to the recorder. When you stop sending frames to the recorder, you'll also need to pause your timestamps (so that there isn't a massive time gap in your final recording, corresponding to how long you were paused). This is why you'll also want to pause the clock (if you're using a realtime clock). I didn't get any code in the email you sent me.

 

@Lanre , @MartinTilo : Unfortunately, I tried building for iOS with Unity 2019.4.1f1 and the bug is still present.
I have the impression that the amount of RAM being allocated is slightly reduced if compared to 2019.4.0f1, and also that in the Editor now some memory gets actually de-allocated... but the core problem is still here: on iOS the app runs out of memory after a few seconds of video recording, and in the Windows Editor the amount of memory used (as shown in the profiler) is drastically increasing.

I think that the 2019.4.1f1 fix should be somehow related to this as well, since in this version, compared to 2019.4.0f1:
- on iOS the recording lasts a few seconds more before crashing due to memory issues
- on Windows Editor the profiler shows a slower memory increase (IE, same time frame: 0.9 GB -> 1.5 Gb instead of 0.9 -> 7 Gb)

 

I performed additional testing and I discovered that the bug affecting the recordings actually is the one @MartinTilo guessed: the only problem is that it isn't fixed in 2019.4.1f1.

As a workaround for anybody needing to use AsyncReadback in 2019.4.x, please use AsyncGPUReadback.RequestIntoNativeArray() instead of AsyncGPUReadback.Request().

 

I can't add this workaround in NatCorder because `RequestIntoNativeArray` is new, introduced in either 2019.2 or 2019.3. NatCorder needs to support 2018.3+.

 

Sure, I wasn't referring to you directly, my suggestion was merely for developers in the need of using 2019.4.x.

 

Hi @Lanre ,
Thanks for a great asset. Its been straightforward to use it - really plug-and-play.

Encoding MP4 would also be very useful for efficient streaming of video data, like writing into NetworkStream. Is this possible with NatCorder? If not, do you intend to make it a feature?

Thanks.

 

I'm glad you find it easy to use. Streaming is not as simple as writing the MP4 file data over a network. It requires specialized protocols/formats like MPEG-TS, HLS, or MPEG-DASH. All of these are far out of scope for NatCorder, so it is not on the roadmap.

 

Hi everyone, I have an Unity AR project using Placenote. I want to record video, audio and photo in it. But I need to integrate this project into native iOS app.

Could you tell me, has anyone had experience with this plugin in an integrated native iOS app?

Thanks.

 

Could you maybe help me out on how to implement this workaround?

 

I'm not familiar with Placenote. I'm not sure if there is anything to worry about. As far as you are using NatCorder properly from Unity, it shouldn't matter if Unity is being run as a module within an iOS app.