NatCorder - Video Recording API

Check the Changelog.md and README.md. The namespace was changed as part of a plan to consolidate the NatSuite API's.

 

Thats fine, but i have an app in the play store that is now broken because of the above crash it works on my old phone but crashes on my new phone for some reason when MP4Recorder is called.

I was trying to put out a fast fix, but seems i have to re-write a lot of code because of the upgrade to 1.7.1 now first because its all different.

MP4recorder used to have a call back function, now it async, CommitSamples is now different, so are the clocks, timestamp from the clocks is gone, "paused" from the clocks not there anymore which i was using to pause/resume the recording (how to fix that now?), videoRecorder.Dispose() dont exist anymore, so i have to rewrite all the code involved with those. I got all that code from your tutorial website, but its all invalid now. It going to take a really long time to re-write all that because the app is complicated.

A better way to do an upgrade would be to add new functions and methods, but keep the old ones so that every bodies code isnt broken and have to re-write everything. Have like a fixed interface, where that stays the same, so that we are not exposed to changes in internal implementation.

 

The logs always contain the reason for the crash. No need to speculate; collect the logs and share them.

It isn't all different. The only changes you need to make from 1.6.1 to 1.7.1 are:

1. Replace your `using NatCorder` with `using NatSuite.Recorders`
2. Instead of passing your `OnReplay` callback to the recorder's constructor, replace `recorder.Dispose()` with `await recorder.FinishWriting()` and provide the path to your callback.

These are the main changes.

 

OK i upgraded to 1.7.1 code compiled no errors, but when i run it hangs on MP4Recorder for a few seconds, then the app exists before it returns from that, no error or anything, here is the code and the log...

Code (CSharp):

1.         int w = Screen.width;
2.         int h = Screen.height;
3.         Debug.Log("START RECORDING setting up MP4 recorder: w="+w + " h="+h + " s="+sampleRate + " c="+channelCount);
4.         videoRecorder = new MP4Recorder(w, h, 30, sampleRate, channelCount);
5.         Debug.Log("START RECORDING setting camera");

It never returns from MP4Recorder, it stays in there for about 2 or 3 seconds then the app just exists without any error, here is the log

(Filename: ./Runtime/Export/Debug.bindings.h Line: 45)
2020-04-08 10:43:18.405 11399-11430/com.app.app I/Unity: START RECORDING setting up MP4 recorder: w=720 h=1507 s=44100 c=1
<DoActualStartRecording>d__233:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
UnityEngine.Events.UnityEvent:Invoke()
NatSuite.Examples.Components.<Countdown>d__10:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

(Filename: ./Runtime/Export/Debug.bindings.h Line: 45)

No idea why this is happenning, i feel so stressed now because this app is in the play store, and a potential employer will look at it before giving me a job, but its crashing because of Mp4Record, any ideas how to fix fast before he sees it?

Thanks

 

Im trying to test version 1.7 using your ReplyCam that comes with NatCorder, but the demo scene wont load becuase it says it was serialized for 2019.x but im using 2018.x so i cant even try your demo.

You should use the earliest version of Unity to publish your stuff because a lot of people not using 2019 yet. im still using 2018.3 i cant open your scene because it was serialized for 2019.x

 

Your height is an odd number (1507). There is a note in the README on this. I don't recommend recording at screen resolution. Use a fixed resolution like 1920x1080 or 1280x720.

 

ok i rounded the screen resolution to the nearest 10 pixels and it worked, im BACK IN ACTION!!!

 

I don't recommend this approach. It is best to record at a standard resolution. Unity will aspect scale the result.

 

Hi all.

I've been working on less hideous API documentation for NatCorder. Please check it out and give me any feedback you have.

Also, I'd like to add more media in some of the pages, so if you'd like some free promo for your app, send me a recording you'd like me to showcase--preferably a short video (3-4 seconds) and augmented reality gameplay.

 

I am attempting to check if a recording file exists on iOS and could use some guidance.

I cache the path after the recording is saved and turn on a Replay button in my UI if the file is there. This works on Android and in the editor, but I'm missing something on iOS. I realize in your demo for iOS you add the prefix file:// when passing the path to Handheld.PlayFullscreenMovie.

I was wondering if that path isn't available with System.io.File.Exists() on iOS?

My replay is saved at:

/var/mobile/Containers/Data/Application/C27B9676EA82-9823/Documents/recording_2020_04_08_23_45_27_717.mp4

I have tried to access it with: (File:///)
File.Exists(file:///var/mobile/Containers/Data/Application/C27B9676EA82-9823/Documents/recording_2020_04_08_23_45_27_717.mp4))

And with : (File://)
File.Exists(file://var/mobile/Containers/Data/Application/C27B9676EA82-9823/Documents/recording_2020_04_08_23_45_27_717.mp4))

But it feels that I shouldn't be able to access the file this way on iOS. Maybe if I use Application.PersistentDataPath?

I realize this might not be 100% NatCorder related, but any guidance you have is appreciated.

 

So I was finally able to update to the latest version and it just wouldn't work. Out of curiosity, as I had already tried 1.6 and 1.7, I tried 1.5 and for some reason, it works!

 

File handling API's differ depending on whether they accept a file path or a URI. A file path is what NatCorder will always give you. A URI on the other hand must include a protocol/scheme. This is the "file://" prefix that gets added before Handheld.PlayFullscreenMovie. In general:

• NatCorder will always return a file path, not a URI.
• Application.persistentDataPath will always return a path, not a URI.
• System.IO API's will only accept a path, not a URI.
• Handheld.PlayFullscreenMovie will only accept a URI on iOS (which is why we add the "file://" scheme).
• NatShare will only accept a path, not a URI.
• NatReader will only accept a URI, not a path.

These are a pain to keep track of, but you've got to keep track of them.

 

Thanks so much for the detailed explanation! This should help.

On a side note, I wanted to further emulate Everyplay with the ability to view and share the replay from the same screen. I ended up using Simple Movie Player (From the Asset Store) which handles the playback without leaving Unity. It's not a perfect solution, but it plays pretty well with NatCorder and was thinking you two could form some sort of partnership for an integrated solution.

 

Hello,
I just looked at new API documentation and it looks great!
Unfortunately for me, "Augmented Reality" section states that "...EasyAR SDK is currently not compatible with NatCorder...". Could you describe more what are the "incompatibilities" and if there are any workarounds?

I just implemented NatCorder in app which is using EasyAR SDK and... it works... on Android I'm able to record AR camera with trackers, on iOS it also works... but... camera background texture on video is rotated On iOS device is displayed correctly.



Thank you

 

This is exactly what I was referring to. EasyAR seems to hijack the game camera's rendering causing these weird rotations when you render the game camera to a texture. I'm glad you like the new docs .

 

I have it in the app store now where its rounding to nearest 10 pixels, will it crash?

 

You shouldn't get any recording failures when rounding to the nearest 10, but the safest you can be is to record at a standard resolution.

 

Hi Lanre,

I am experiencing audio timing issues when importing Natcorder into a new project on 2019.3.8 and using the ReplayCam example with no modifications to the code. When I hold record, the first recording is perfect, but in subsequent recordings, the recorded audio samples are taken from after I stopped the initial recording, much like if I was not using the same clock for both audio and video. As I say, I've not modified any of the code so the clock is shared on both audio and video recorders.

Any help would be really appreciated.

 

Can NatCorder record the video feed directly as a background process while also passing it to the app for use in a texture?

I would like to record audio/video on iOS directly from the camera while simultaneously presenting the camera video feed to the user with UI and other app content on top. I don't want to capture the iOS screen with all the overlay content, I want to record the video feed directly while also using it as a texture in the app.

Update: I found NatDevice API that looks like it might be what I need.
-- Does NatDevice record video merged with audio from the mic? The docs have a Video Recording section but it's incomplete.

 

I seems that I found solution for this issue: https://answers.easyar.com/15872/fix-found-easyar-unity-ios-camera-feed-upside-down-using-hdr

I used attached, updated EasyAR shaders and also changed Camera.forceIntoRenderTexture setting (in
CameraImageRenderer class) and now video with EasyAR view is recorded correctly.
I didn't wanted to change iOS graphics API to GLES3 but I wanted to stick with Metal so I applied described fix and it works for me.

Thank you

 

I think this happens because of how Unity buffers the microphone audio clip. I recommend using NatDevice for microphone recording instead. It isn't susceptible to audio glitches like Unity's Microphone API. Here's the ReplayCam example that uses NatDevice instead.

 

NatCorder doesn't record things for you. It is up to you to push frames to the recorder. As such, you would have to get the camera preview data and commit it to the recorder.

WebCamTexture provides the GetPixels32 method for getting the preview data, but on mobile apps this will appear rotated. And rotating it back to upright is not trivial.

NatDevice does provide the preview data upright all the time, so it can be passed to the recorder as is. You can also commit microphone audio from an audio device. See the ReplayCam example that uses NatDevice.

 

This looks good. Thanks for letting me know!

 

Bug report

Upgraded from 1.6.3 -> 1.7.1
Unity 2018.4.16
Device: iPhone X, iOS 13.2.3

1.6.3: worked well
1.7.1: recording video now freezes entire screen (viewfinder and UI don't update). Video is still being recorded tho, and when I tap my camera button to stop recording the app returns to a responsive state, and recorded video is saved. Can't find the cause.

For a sanity check I tried the ReplayCam sample on device, used Unity 2019.3.9 to build it and it works, but I can't try it in Unity 2018 because of the "serialized for 2019.x" issue mentioned earlier, so I don't know if this is a 2018 issue.

Thank you

 

This is a bug in Unity 2018. Unity fixed it in 2019.2. The serialization issue will be fixed in the next update. Sorry for the inconvenience.

 

Thanks for the quick reply! Shoot, too bad

 

Hello,

I have a question and an issue to report.

The question:
Is there a way of taking a screenshot with NatCorder instead of recording a video?

The issue :
I am experiencing an issue when recording video from an ARCore session with NatCorder.

Unity Version -> 2019.3.4f1
NatCorder version -> Latest
I am using an iPhone XS and a Pixel 4.

I am using ArCore on iOS as well as I am using the ArCore cloud anchors example on both devices.
I cannot use Vulkan on Android because ArCore doesn't support it, but multithreading is enabled.

I am using a modified version of the NatCorder ReplayCam example to save videos to memory only instead of playing them back. I record 10 seconds videos.

The issue is that when I record the first video I get about 1 second of frozen camera and stuttering audio. Then after that everything goes back to normal. The video is recorded fine. From the second time onwards I start the video recording everything is fine, no stuttering. If I restart the app I get the same problem.

I tried a resolution of 1280x720 but also up to the full screen resolution. No difference.

My code looks like this:

Code (CSharp):

1. /*
2. *   NatCorder
3. *   Copyright (c) 2020 Yusuf Olokoba
4. */
5.  
6. namespace NatSuite.Examples
7. {
8.  
9.     using UnityEngine;
10.     using System.Collections;
11.     using Recorders;
12.     using Recorders.Clocks;
13.     using Recorders.Inputs;
14.     using System.IO;
15.     using UnityEngine.UI;
16.  
17.     public class CameraGrabber : MonoBehaviour
18.     {
19.  
20.         public GameObject _ScreenshotButton;
21.  
22.         [Header("Recording")]
23.         public int videoWidth = 1280; //Screen.width;
24.         public int videoHeight = 720; //Screen.height;
25.         public bool recordMicrophone;
26.  
27.         private IMediaRecorder recorder;
28.         private CameraInput cameraInput;
29.         private AudioInput audioInput;
30.         public  AudioSource audioOut;
31.         public AudioListener _audioWeHear;
32.  
33.         public void StartRecording()
34.         {
35.  
36.             //Disable Screenshot button
37.             _ScreenshotButton.SetActive(false);
38.  
39.             // Start recording
40.             var frameRate = 30;
41.             var sampleRate = AudioSettings.outputSampleRate;
42.             var channelCount = 2;
43.             recorder = new HEVCRecorder(videoWidth, videoHeight, frameRate, sampleRate, channelCount);
44.             var clock = new RealtimeClock();
45.  
46.             // Create recording inputs
47.             cameraInput = new CameraInput(recorder, clock, Camera.main);
48.             audioInput = new AudioInput(recorder, clock, _audioWeHear);
49.         }
50.  
51.         public async void StopRecording()
52.         {
53.             audioInput.Dispose();
54.             cameraInput.Dispose();
55.             var path = await recorder.FinishWriting();
56.              Debug.Log($"Saved recording to: {path}");
57.             var info = new DirectoryInfo(Application.persistentDataPath);
58.             var fileInfo = info.GetFiles();
59.             if (fileInfo.Length > 0)
60.             {
61.                 foreach (FileInfo file in fileInfo)
62.                 {
63.                     if (file != null)
64.                     {
65.                         string full = file.DirectoryName + "/" + file.Name;
66.                         //text_field.text += "\n" + full + "\n";
67.                         NativeGallery.SaveVideoToGallery(full, "UMS", file.Name);
68.                         File.Delete(full);
69.                     }
70.                 }
71.             }
72.             _ScreenshotButton.SetActive(true);
73.         }
74.     }
75. }

My RecordButton script look like this:

Code (CSharp):

1. /*
2. *   NatCorder
3. *   Copyright (c) 2020 Yusuf Olokoba
4. */
5.  
6. namespace NatSuite.Examples.Components {
7.  
8.     using System.Collections;
9.     using UnityEngine;
10.     using UnityEngine.UI;
11.     using UnityEngine.Events;
12.     using UnityEngine.EventSystems;
13.  
14.     [RequireComponent(typeof(EventTrigger))]
15.     public class UMSRecordButton : MonoBehaviour, IPointerDownHandler, IPointerUpHandler
16.     {
17.  
18.         public Image button, countdown;
19.         public UnityEvent onTouchDown;
20.  
21.         public CameraGrabber _cameraGrabber;
22.  
23.         private bool pressed;
24.         private const float MaxRecordingTime = 10f; // seconds
25.  
26.         private void Start()
27.         {
28.             Reset();
29.         }
30.  
31.         private void Reset()
32.         {
33.             // Reset fill amounts
34.             if (button)
35.                 button.fillAmount = 1.0f;
36.             if (countdown)
37.                 countdown.fillAmount = 0.0f;
38.         }
39.  
40.         void IPointerDownHandler.OnPointerDown(PointerEventData eventData)
41.         {
42.             // Start counting
43.             if (!pressed)
44.             {
45.                 StartCoroutine(Countdown());
46.             }
47.         }
48.  
49.         void IPointerUpHandler.OnPointerUp(PointerEventData eventData)
50.         {
51.    
52.         }
53.  
54.         private IEnumerator Countdown()
55.         {
56.             pressed = true;
57.             // Start recording
58.             onTouchDown?.Invoke();
59.             // Animate the countdown
60.             float startTime = Time.time, ratio = 0f;
61.             while ((ratio = (Time.time - startTime) / MaxRecordingTime) < 1.0f)
62.             {
63.                 countdown.fillAmount = ratio;
64.                 button.fillAmount = 1f - ratio;
65.                 yield return null;
66.             }
67.             _cameraGrabber.StopRecording();
68.             // Reset
69.             Reset();
70.             pressed = false;
71.         }
72.     }
73. }

Is there anything I could try to solve this issue?

Many Thanks!

 

Unity provides an API for taking screenshots. NatCorder is only for video recording.

The native encoders usually undergo a warmup process the first time you record. We can't control it. The workaround is to add a blank scene where you create an empty recording, before proceeding to your app.

 

Hi Lanre,
I have a problem with pause & resume while record video
when I pause & resume 2 or 3 time , video result will be error, it still record sound but image is not

Code (CSharp):

1. using System;
2. using System.Collections;
3. using AniLipSync.VRM;
4. using NatCorder;
5. using NatCorder.Clocks;
6. using NatCorder.Inputs;
7. using NatMic;
8. using NatShareU;
9. using UnityEditor;
10. using UnityEngine;
11. using UnityEngine.UI;
12. using UnityEngine.Video;
13.  
14. public class RecordVideo : MonoBehaviour, IAudioProcessor
15. {
16.     public Camera[] camRecords;
17.     public GameObject btnStart, btnStop, btnPause, btnResume;
18.    
19.     [Header("Recording")]
20.     public int videoWidth = 1280;
21.     public int videoHeight = 720;
22.  
23.     public VideoPlayer VideoPlayer;
24.    
25.     private MP4Recorder videoRecorder;
26.     private CameraInput cameraInput;
27.     private RealtimeClock recordingClock;
28.  
29.     private bool recording;
30.    
31.     private IAudioDevice audioDevice;
32.     private bool mute;
33. #if UNITY_IOS
34.     private IEnumerator Start()
35.     {
36.         if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
37.         {
38.             yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
39.         }
40.        
41.         if (!Application.HasUserAuthorization(UserAuthorization.Microphone))
42.         {
43.             yield return Application.RequestUserAuthorization(UserAuthorization.Microphone);
44.         }
45.     }
46. #endif
47.  
48.     private void Update()
49.     {
50.         if (recording)
51.         {
52.             UIManager.Instance.RecordTime += Time.deltaTime;
53.         }
54.     }
55.    
56.     void OnApplicationFocus(bool hasFocus)
57.     {
58.         if (recording && !hasFocus)
59.         {
60.             Stop_OnClick();
61.         }
62.     }
63.  
64.     public void Start_OnClick()
65.     {
66.         btnStart.SetActive(false);
67.         btnStop.SetActive(true);
68.         btnPause.SetActive(true);
69.         btnResume.SetActive(false);
70.  
71.         UIManager.Instance.ShowRecordUI(false);
72.  
73.         var sampleRate = 44100;
74.         var channelCount = 1;
75.  
76.         recordingClock = new RealtimeClock();
77.         videoRecorder = new MP4Recorder(
78.             videoWidth,
79.             videoHeight,
80.             30,
81.             sampleRate,
82.             channelCount,
83.             OnReplay
84.         );
85.  
86.         // Create recording inputs
87.         cameraInput = new CameraInput(videoRecorder, recordingClock, camRecords);
88.  
89.         audioDevice = AudioDevice.GetDevices()[0];
90.         audioDevice.StartRecording(sampleRate, channelCount, this);
91.  
92.         UIManager.Instance.RecordTime = 0;
93.         recording = true;
94.  
95.         if (VideoPlayer.enabled)
96.         {
97.             VideoPlayer.frame = 0;
98.             VideoPlayer.Play();
99.         }
100.     }
101.  
102.     public void Stop_OnClick()
103.     {
104.         btnStart.SetActive(true);
105.         btnStop.SetActive(false);
106.         btnPause.SetActive(false);
107.         btnResume.SetActive(false);
108.  
109.         UIManager.Instance.ShowRecordUI(true);
110.  
111.         audioDevice.StopRecording();
112.  
113.         if (recording)
114.             cameraInput.Dispose();
115.  
116.         // Stop recording
117.         videoRecorder.Dispose();
118.        
119.         cameraInput = null;
120.         videoRecorder = null;
121.        
122.         recording = false;
123.        
124.         if(VideoPlayer.isPlaying)
125.             VideoPlayer.Pause();
126.     }
127.  
128.     public void Pause_OnClick()
129.     {
130.         btnPause.SetActive(false);
131.         btnResume.SetActive(true);
132.        
133.         recordingClock.Paused = true;
134.         cameraInput.Dispose();
135.         recording = false;
136.     }
137.  
138.     public void Resume_OnClick()
139.     {
140.         btnPause.SetActive(true);
141.         btnResume.SetActive(false);
142.  
143.         recordingClock.Paused = false;
144.         cameraInput = new CameraInput(videoRecorder, recordingClock, camRecords);
145.         recording = true;
146.     }
147.  
148.     private void OnReplay(string path)
149.     {
150.         Debuger.Log("Saved recording to: " + path);
151.        
152.         // Playback the video
153. #if UNITY_EDITOR
154.         EditorUtility.OpenWithDefaultApp(path);
155. #elif UNITY_IOS
156.         //Handheld.PlayFullScreenMovie("file://" + path);
157.         NatShare.SaveToCameraRoll(path, "jins");
158. #elif UNITY_ANDROID
159.         Handheld.PlayFullScreenMovie(path);
160. #endif
161.     }
162.  
163.     public void OnSampleBuffer(float[] sampleBuffer, int sampleRate, int channelCount, long timestamp)
164.     {
165.         if (mute)
166.             Array.Clear(sampleBuffer, 0, sampleBuffer.Length);
167.         videoRecorder.CommitSamples(sampleBuffer, recordingClock.Timestamp);
168.     }
169.  
170.     public void OnMute(Toggle toggle)
171.     {
172.         mute = toggle.isOn;
173.     }
174. }

I use NatCorder 1.6.3 & Unity2018.4.14f1

 

I am having some performance issues on my Android device when recording a replay. It causes the gameplay to become noticeably jerky. I don't feel like I noticed this before. If I toggle the replays off, it returns to running smooth. The device is a Google Pixel 1.

On iOS I have also been able to run with replays successfully with as low as an iPhone 6.

My settings for recording are
Width: 1280
Height: 720
FPS: 30 and have tried 15fps
Audio Recorder is set to AUdioSettings.outputSampleRate

Google Play Version
https://www.dropbox.com/s/b1i2o98719j15gm/Natcorder_Google1.mp4?dl=0
iPhone 6 version
https://www.dropbox.com/s/f3m8nwracu64h2o/Natcorder_iPhone6.mp4?dl=0

Any performance tips would be appreciated.

Thanks

 

Thank you so much for the prompt answer! I understand, I was hoping I didn't have to resort to doing that

I did some more testing and I achieved good results on iOS. On Android (Pixel 4) I still get a few milliseconds frozen camera feed at the start and ending of each recoding, even at 720p resolution. Portrait mode.

I cannot use Vulkan as ArCore does not support it atm. Do you think this is where the problem could be?

 

You don't seem to stop audio recording when you pause. This might be the problem. Also, you'll want to upgrade to NatDevice (NatMic has been deprecated) and NatCorder 1.7.1.

 

Here are some general performance tips, but you'll have to profile your app in the Editor to know where your frame time is being spent.

 

If you can't use Vulkan, then you can't use async GPU readbacks. So that's definitely where the problem lies.

 

thanks, I will try something with it

 

Hello,

I own NatDevice and NatCorder. I would like to take a photo of unity camera into a Texture2D so I can share it using NatShare. There are solutions on the internet that use RenderTexture & ReadPixels but I'm having difficulty implementing them because I'm using URP.

NatDevice can capture a photo, but I believe it takes a native photo, so it's not what I'm looking for.

NatCorder can take frames of Unity's camera and commit them to the recorder (MP4) using CameraInput then save it as a video in a path.

What I would like to do is just take one frame from CameraInput and have it as Texture2D or save in a path (PNG or JPEG).

Is that possible on iOS? If not, can it be implemented? NatSuite make's life easier by taking the complexity of media in Unity and NatCorder can takes videos of Unity's Camera, I'm sure it can be extended to take a photo. Maybe call it NatCapture, so it includes photos and not just video / audio.

Thanks!

 

Thanks, I just bought NatDevice, and this fixes the sync issue, but with the ReplayCam example code you linked to (and also with the HotMic example with NatDevice imported into a fresh unity scene), my first recording is always sped up by about 50%, and then when I press the record button again, the next recording is perfect. I've tried setting the ClipRecorder's sample rate to 48000:
recorder = new ClipRecorder(48000, device.channelCount);

I am on iOS and I was reading in the NatDevice forum that NatDevice currently ignores any request to set the sample rate of the audio device but there is a new asset store update pending? Would this pending update solve my problem or do you advise another fix?

 

I found the solution in CameraInput, I modified the code to take one Texture2D, here it is.

Code (CSharp):

1. private Texture2D cameraTexture;
2.  
3. public void CapturePhoto()
4. {
5.     StartCoroutine(OnCapturePhoto());
6. }
7.  
8. private IEnumerator OnCapturePhoto()
9. {
10.     var endOfFrame = new WaitForEndOfFrame();
11.     yield return endOfFrame;
12.  
13.     RenderTextureDescriptor frameDescriptor = new RenderTextureDescriptor(videoWidth, videoHeight, RenderTextureFormat.ARGB32, 24);
14.     frameDescriptor.sRGB = true;
15.     RenderTexture frameBuffer = RenderTexture.GetTemporary(frameDescriptor);
16.  
17.     RenderTexture prevTarget = mainCamera.targetTexture;
18.  
19.     mainCamera.targetTexture = frameBuffer;
20.     mainCamera.Render();
21.  
22.     cameraTexture = new Texture2D(frameBuffer.width, frameBuffer.height, TextureFormat.RGBA32, false, false);
23.     cameraTexture.ReadPixels(new Rect(0, 0, frameBuffer.width, frameBuffer.height), 0, 0, false);
24.     mainCamera.targetTexture = prevTarget;
25.  
26.     RenderTexture.ReleaseTemporary(frameBuffer);
27.  
28.     SharePhoto(cameraTexture);
29. }
30.  
31. private async void SharePhoto(Texture2D texture)
32. {
33.     SharePayload payload = new SharePayload();
34.     payload.AddImage(texture);
35.     await payload.Commit();
36. }

Not sure if it's completely correct or some parts can be omitted but it works for now. Also I have to check for memory leaks.

 

Unity already has a function for this. See the ScreenCapture class.

 

My only guess here is that iOS reroutes the system audio, causing the sample rate to change immediately `AudioDevice.StartRunning` is called. I recommend opening an issue on the examples repo.

 

ScreenCapture takes an image of the whole screen, you can't hide GUI or elements you don't want as you can do with Cameras culling masks. That's why I used the method above. I think it would be handy to provide a method that returns a Texture2D of a specific camera or cameras.
CameraInput.GetTetxure() for example, just a thought. I've achieved what I wanted from looking at your code in the first place so thanks!

 

Hey guys
got a problem which arose when testing the application, installed via Test Flight, on iPhone 8, iPhone XR, iPhone X.

When recording a video, the sound overlaps incorrectly.
In the recorded video, the audio track plays silence or the sounds that had been before the "Record" button was pressed. And only 10 seconds later (out of 30), the needed track begins.
However, after installing the application directly, the sound overlays perfectly.

 

Can you share your recording code?

 

Code (CSharp):

1. private IEnumerator Start ()
2.         {
3.             videoWidth = Screen.width;
4.             videoHeight = Screen.height;
5.             renderTexture.width = Screen.width;
6.             renderTexture.height = Screen.height;
7.             camera = FindObjectOfType<Camera>();
8.             // Start microphone
9.             microphoneSource = gameObject.AddComponent<AudioSource>();
10.             microphoneSource.mute =
11.             microphoneSource.loop = true;
12.             microphoneSource.bypassEffects =
13.             microphoneSource.bypassListenerEffects = false;
14.             microphoneSource.clip = Microphone.Start(null, true, audioLength, AudioSettings.outputSampleRate);
15.             yield return new WaitUntil(() => Microphone.GetPosition(null) > 0);
16.             microphoneSource.Play();
17.         }
18.  
19. public void StartRecording ()
20.         {
21.             isFinished = false;
22.             // Start recording
23.             var frameRate = 30;
24.             var sampleRate = recordMicrophone ? AudioSettings.outputSampleRate : 0;
25.             var channelCount = recordMicrophone ? (int)AudioSettings.speakerMode : 0;
26.             var clock = new RealtimeClock();
27.  
28.             recorder = new MP4Recorder(videoWidth, videoHeight, frameRate, sampleRate, channelCount);
29.             // Create recording inputs
30.             cameraInput = new CameraInput(recorder, clock, camera);
31.             audioInput = recordMicrophone ? new AudioInput(recorder, clock, microphoneSource, true) : null;
32.             // Unmute microphone
33.             microphoneSource.mute = audioInput == null;
34.         }
35.  
36.         public async void StopRecording() {
37.             // Mute microphone
38.             microphoneSource.mute = true;
39.             // Stop recording
40.             audioInput?.Dispose();
41.             cameraInput.Dispose();
42.             var path = await recorder.FinishWriting();
43.             // Playback recording
44.             Debug.Log($"Saved recording to: {path}");
45.             isFinished = true;
46.             var prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
47.           //  Handheld.PlayFullScreenMovie($"{prefix}{path}");
48.         }

 

This is a bug in Unity's Microphone API (the microphone clip is set to loop, so it can keep old samples). I recommend using our NatDevice API instead. Also, do not record at screen resolution.

 

Hello Lanre! Hello everybody!
I can see that pause option is possible. But how can I cut off part of recording? Is it possible to save timestamp on every time I press "pause" button and on press "remove" button remove all video and audio frames science last saved timestamp?
Thank you!

 

Recorders act as sinks; once you commit frames to them, you can't go back and modify or remove those frames. NatCorder doesn't have video editing functionality, so it does not support video cutting or other editing tasks.

 

Hi,

I just recently purchased NatDevice and I have NatCorder as well. I'm simply trying to display a rear facing camera on a Raw Image texture on my canvas. Then have functionality for Starting and Stopping recording of what the camera sees (with any overlayed UI) and as well as audio.

I think I have everything working though I did seem to get confused looking at outdated API and more recent API...I guess I just want to know if I am doing everything correctly.

There is one current issue I am facing, and that is the recorded audio's volume is really low.

Here's my code for opening camera, starting recording, stopping recording, and closing camera:

Code (CSharp):

1.  public async void OpenCamera()
2.         {
3.             outputImage.gameObject.SetActive(true);
4.  
5.             if (Application.isEditor)
6.             {
7.                 deviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.GenericCameraDevice);
8.             }
9.             else
10.             {
11.                 // Request camera permissions
12.                 if (!await MediaDeviceQuery.RequestPermissions<CameraDevice>())
13.                 {
14.                     Debug.LogError("User did not grant camera permissions");
15.                     return;
16.                 }
17.  
18.                 deviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.FrontFacing);
19.             }
20.  
21.             // Start camera preview
22.             nativeCamera = deviceQuery.currentDevice as CameraDevice;
23.             var previewTexture = await nativeCamera.StartRunning();
24.  
25.             Debug.Log($"Started camera preview with resolution {previewTexture.width}x{previewTexture.height}");
26.             // Display preview texture
27.             outputImage.texture = previewTexture;
28.             //  previewAspectFitter.aspectRatio = previewTexture.width / (float)previewTexture.height;
29.         }

Code (CSharp):

1. public void StartRecording()
2.         {
3.             // Start recording
4.             var frameRate = 30;
5.             var sampleRate = recordMicrophone ? AudioSettings.outputSampleRate : 0;
6.             var channelCount = recordMicrophone ? (int)AudioSettings.speakerMode : 0;
7.             var clock = new RealtimeClock();
8.             recorder = new MP4Recorder(720, 1280, frameRate, sampleRate, channelCount);
9.  
10.             // Create recording inputs
11.             cameraInput = new CameraInput(recorder, clock, Camera.main);
12.             audioInput = recordMicrophone ? new AudioInput(recorder, clock, microphoneSource, true) : null;
13.  
14.             // Unmute microphone
15.             microphoneSource.mute = audioInput == null;
16.         }

Code (CSharp):

1. public async void StopRecording()
2.         {
3.             // Mute microphone
4.             microphoneSource.mute = true;
5.  
6.             // Stop recording
7.             audioInput?.Dispose();
8.             cameraInput.Dispose();
9.             var path = await recorder.FinishWriting();
10.  
11.             if (debugText)
12.             {
13.                 debugText.text = path;
14.             }
15.  
16.             finishedRecording = true;
17.  
18.            // var prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
19.            // Handheld.PlayFullScreenMovie($"{prefix}{path}");
20.         }

Code (CSharp):

1. public void CloseCamera()
2.         {
3.             outputImage.gameObject.SetActive(false);
4.  
5.             if (nativeCamera != null)
6.             {
7.                 nativeCamera.StopRunning();
8.             }
9.         }

I have an audio source attached to my CameraController in the canvas hierarchy. Settings look to be correct. And Audio Listener on the MainCamera object. Both at same X and Y position. Main Camera at 0 Z pos, Canvas at 1 Z pos.

Appreciate any and all help!

 

You seem to still be using Unity's microphone API. If you already have NatDevice, use NatDevice to record the microphone.

EDIT: Issues relating to volume typically depend on the device. We don't touch the audio samples that are committed, so we have no control over the gain.

 

Appreciate the fast response!

So looking at your documentation, I tried to just copy the setup you have, so now my Start and Stop recording functions look like:

Code (CSharp):

1. public void StartRecording()
2.         {
3.  
4.             // With NatDevice...
5.             // Get a microphone
6.             var query = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
7.             var device = query.currentDevice as AudioDevice;
8.  
9.             // With NatCorder...
10.             // Create a recorder
11.             var recorder = new MP4Recorder(720, 1080, 30, device.sampleRate, device.channelCount);
12.             var clock = new RealtimeClock();
13.             // Start streaming frames to the recorder
14.             var cameraInput = new CameraInput(recorder, clock, Camera.main);
15.             device.StartRunning((sampleBuffer, _) => recorder.CommitSamples(sampleBuffer, clock.timestamp));
16.         }
17.  
18.         public async void StopRecording()
19.         {
20.             var query = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
21.             var device = query.currentDevice as AudioDevice;
22.  
23.             // Stop streaming frames to the recorder
24.             cameraInput.Dispose();
25.             device.StopRunning();
26.             // Finish writing
27.             var path = await recorder.FinishWriting();
28.         }

And I just tried it on my device. But it seems like the mp4 file is corrupt. I can't open it whereas before (with my old code I displayed) it opened and played fine. Anything I am doing wrong? Do I need to use a combination of your Recording The Camera and Recording the Microphone code? I just used the microphone code as that looks like it creates an mp4 recorder as well. Still using my original OpenCamera functionality.