NatDevice - Media Device API

This is weird. I use an iPhone 7+ and don't face this issue.

 

Oh I forgot to add. You also need to select the Front Camera as the device for this to reproduce. I wasn't able to reproduce this on an iPhone 7 if I selected the Rear Camera. When I'm able to snap an iPhone 7 again, I'll try to reproduce this with the MiniCam example.

 

No problem in the logs.

 

Did the proguard rules for Natsuite change from Natcam?
We've got com.yusufolokoba.** for Natcam.

 

Hello there,

I am facing issues after the latest upgrade of Natcam and Natcorder in my Unity project.
I am testing on OnePlus6T and Unity version 2019.2.5

Issues:
1. While opening the camera for the first time in-app, it always lags for first time, And if we record the video, it also records a laggy video.
2. While switching the scene, the Natcam doesn't open, it just shows the white color canvas.

Is anyone else also facing issues or it might be related to Unity version or something else. With older version of Natcam and Natcorder, I wasn't facing any such issues.

Looking forward for assistance.

Regards,
Paramjot

 

Hi, we are still facing the issue for StopPreview() freeze in the example. I tried with LG G5 and 2.3.1 version of NatCam.

Thanks

 

@Lanre : Can you please help us to fix the issues coming up with Lagging of video and StopPreview()

 

I can't reproduce this with MiniCam. Switching cameras back and forth works as expected.

 

Yup. All NatSuite API's now have just one bundle identifier:

Code (CSharp):

1. api.natsuite.*

 

The StopPreview fix is coming next week. As for lagging, I am not able to reproduce it, so please profile your app to see where CPU time is being spent.

 

Hi Lanre,

The current version of NatDevice 1.0.0 appears to be crashing 100% of Samsung Galaxy S10 5G, and Samsung Galaxy Note S10+ users.

This is a hard crash, with the app being killed.

In looking at the difference between these devices and the rest of the Samsung S10 lineup, it appears the crashing devices are unique in that they both have front and back 3D depth sensing cameras.

 

Can you send me full logs from logcat?

 

I'm sorry, I can't because we don't have one of these phones. I'm getting this info from Crashlytics where we have 208 crashes across 104 users, and I can tell from our own logging that the crash occurs where we initialize the camera.

I've instrumented the code in a new version to hopefully give more info.

 

And just to confirm, this is specific to the S10? What OS version?

 

More info:

The crash specifically happens on the following line:

natcamDeviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.CameraDevice);

The crash is only happening on the two specific devices I mentioned (which appear to be the only cameras in the S10 line that have Time Of Flight sensors in their camera arrays).

Samsung Galaxy Note S10+
Samsung Galaxy S10 5G

All phones appeared to be running Android 10.

 

Doesn't Crashlytics have a stack trace? Can you share any information logged by the Crashlytics crash reporter?

 

I have just updated to v2.3.1 of NatCam to try and solve a problem with the camera preview crashing my app on an Android 9.0 device. Latest NatCam version fixes this issue but now causes my app to hang when stopping the camera preview, as noted by some other people on this forum. Interestingly my app appears to remain running, but just becomes unresponsive.

Is there any update on when a fix for the StopPreview bug will become available on the Unity Asset Store?

Tested on Moto G4 plus, running Android 7.0, and Oukitel WP5 running Android 9.0

 

We are waiting for Unity to approve the update that fixes this issue. They are taking quite a long time, so I sent them a follow up yesterday.

 

I actually shared two callstacks with you via email a week ago, but I guess they slipped through.

The first crash affected 144 users 276 times in the past 7 days.
This is the Samsung S10+ one, and is giving this somewhat cryptic stack:

Caused by java.lang.Error: signal 6 (SIGABRT), code -1 (?), fault addr --------

at [vdso].()
at libc.abort(abort:156)
at libart._ZN3art7Runtime5AbortEPKc(_ZN3art7Runtime5AbortEPKc:2388)
at libbase._ZN7android4base10LogMessageD1Ev(_ZN7android4base10LogMessageD1Ev:580)
at libart._ZN3art9JavaVMExt8JniAbortEPKcS2_(_ZN3art9JavaVMExt8JniAbortEPKcS2_:1584)
at libart._ZN3art3JNI14GetArrayLengthEP7_JNIEnvP7_jarray(_ZN3art3JNI14GetArrayLengthEP7_JNIEnvP7_jarray:832)
at split_config.()
at split_config.()
at split_config.()
at split_config.()
at split_config.()


There is a second entry in the Crashlytics list which may or may not be related, but is happening to an eclectic mix of devices that are not Samsung Galaxy S10:

The second crashed 58 users 66 times in the past 7 days.

Caused by java.lang.NullPointerException: Attempt to invoke virtual method 'android.view.Surface api.natsuite.natdevice.utility.FrameRenderer.surface()' on a null object reference
at api.natsuite.natdevice.CameraDevice$2.onOpened16)
at android.hardware.camera2.impl.CameraDeviceImpl$1.run(CameraDeviceImpl.java:151)
at android.os.Handler.handleCallback(Handler.java:883)
at android.os.Handler.dispatchMessage(Handler.java:100)
at android.os.Looper.loop(Looper.java:214)
at android.app.ActivityThread.main(ActivityThread.java:7682)
at java.lang.reflect.Method.invoke(Method.java)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:516)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)

 

Sorry for the late response. The second crash can occur if you call `StopRunning` while the camera is starting. This should never happen if you `await` the call to `StartRunning`. As for the second crash, it seems more severe but the stack trace doesn't contain much useful info. I'll respond over email with anything I find.

 

Hi, Just wondering with Android. Can NatDevice change the value of white balance on camera?
if not, is there anyway to do it?

 

Hello! I'm trying to combine the work of NatCorder and NatDevice (microphone recording). On a redmi 7 phone, this works great when recording both audio and video. But on the galaxy S6 and honor 9x phones, only the first one frame with sound is recorded on the video. When removing scripts for recording sound, then on all devices, video recording becomes normal. Not sure what the problem might be? Here is the script used

Code (CSharp):

1. class NatMicCorder : MonoBehaviour
2. {
3.     [Header("Recording")]
4.     public int videoWidth = 1280;
5.     public int videoHeight = 720;
6.     public Vector2Int maxResolution = new Vector2Int(540, 960);
7.  
8.     RealtimeClock recordingClock;
9.     MP4Recorder videoRecorder;    
10.     CameraInput cameraInput;      
11.  
12.     IAudioDevice audioDevice;    
13.     ClipRecorder recorder;
14.  
15.     async void Start()
16.     {
17.         var deviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice);
18.         audioDevice = deviceQuery.currentDevice as AudioDevice;
19.     }
20.  
21.     void StartRecording()
22.     {
23.         var sampleRate = 44100;
24.         var channelCount = 1;
25.         recordingClock = new RealtimeClock();
26.  
27.         SetupRatio(out videoWidth, out videoHeight);
28.  
29.         videoRecorder = new MP4Recorder(videoWidth, videoHeight, 30, sampleRate, channelCount);
30.         cameraInput = new CameraInput(videoRecorder, recordingClock, Camera.main);
31.  
32.         audioDevice.StartRunning(videoRecorder.CommitSamples);
33.     }
34.  
35.  
36.     async void StopRecording()
37.     {
38.         audioDevice.StopRunning();
39.         cameraInput.Dispose();
40.         var recordingPath = await videoRecorder.FinishWriting();
41.  
42.         Debug.Log($"Saved recording to: {recordingPath}");
43.         var prefix = Application.platform == RuntimePlatform.IPhonePlayer ? "file://" : "";
44.         Handheld.PlayFullScreenMovie($"{prefix}{recordingPath}");
45.     }
46.  
47.     public void OnSampleBuffer(float[] sampleBuffer, long timestamp)
48.     {
49.         videoRecorder.CommitSamples(sampleBuffer, recordingClock.timestamp);
50.     }
51.  
52.     private void SetupRatio(out int width, out int height)
53.     {
54.         height = Screen.height;
55.         width = Screen.width;
56.  
57.         if (width > height)
58.         {
59.             height = (int)(height * maxResolution.x / (float)width);
60.             width = maxResolution.x;
61.         }
62.         else
63.         {
64.             width = (int)(width * maxResolution.y / (float)height);
65.             height = maxResolution.y;
66.         }
67.         width -= (width % 2 != 0) ? 1 : 0;
68.         height -= (height % 2 != 0) ? 1 : 0;
69.     }
70. }

 

NatDevice only exposes a way to lock the white balance. You can't modify the white balance coefficients.

 

It's very hard to notice what's going wrong, but the problem is in how you commit audio samples from the audio device to the recorder. All timestamps provided to the recorder have to start from zero; but the timestamps that come from `IAudioDevice` are system timestamps. So you have to commit the sample buffers from the audio device, but use the timestamps from the recording clock instead. Here's what I mean.

 

Hi,

I have a crash on my iPhone 6s but there are no problems with the iPhone XR.
The crash happened when I tried to access the camera:

// Create a device query for device cameras
var deviceQuery = new MediaDeviceQuery(MediaDeviceQuery.Criteria.RearFacing);
// Get current camera
var device = deviceQuery.currentDevice as ICameraDevice;

The CrashLog can be found in the attachment.

 

I've had another report of this, and I can confirm that it is specific to the iPhone 6/6S. I am working on it.

 

Ok, thank you

 

Hello! Is it possible to combine

Code (CSharp):

1. audioDevice?.StartRunning((sampleBuffer, timestamp) => videoRecorder.CommitSamples(sampleBuffer, recordingClock.timestamp));

and

Code (CSharp):

1. audioInput = new AudioInput(videoRecorder, recordingClock, masterSource, true);

because before use NatDevice we combined audioclip and microphon using

Code (CSharp):

1. masterSource.PlayOneShot(microphoneSource.clip);

but if we used audioDevice and AudioInput we getting incorrect playback

 

I'm a bit confused. If you want to mix microphone audio with audio from an AudioSource or AudioListener, you can use the MixerDevice. You can't commit audio to the recorder from two different places, so choose one.

 

Trying to use NatCorder and NatDevice together to record a video with microphone audio - seeing a consistent crash across iOS devices when I stop recording - it seems as if commitSamples is called on the MP4Recorder after finish recording has been called. Tried on multiple iOS devices on various versions of iOS 13, Unity versions 2019.2.15 and 2019.2.21 with Xcode 11.3 beta and Xcode 11.3.1.

In Xcode I see the following error when I finish recording (both in the Beta and official release)

Code (CSharp):

1. TarrotTalkAr`-[NCMP4Recorder commitSamples:sampleCount:timestamp:]:
2.     0x10143dfa4 <+0>:   sub    sp, sp, #0x80             ; =0x80
3.     0x10143dfa8 <+4>:   stp    x24, x23, [sp, #0x40]
4.     0x10143dfac <+8>:   stp    x22, x21, [sp, #0x50]
5.     0x10143dfb0 <+12>:  stp    x20, x19, [sp, #0x60]
6.     0x10143dfb4 <+16>:  stp    x29, x30, [sp, #0x70]
7.     0x10143dfb8 <+20>:  add    x29, sp, #0x70            ; =0x70
8.     0x10143dfbc <+24>:  mov    x19, x4
9.     0x10143dfc0 <+28>:  mov    x21, x3
10.     0x10143dfc4 <+32>:  mov    x22, x2
11.     0x10143dfc8 <+36>:  mov    x20, x0
12.     0x10143dfcc <+40>:  ldr    x0, [x0, #0x38]
13.     0x10143dfd0 <+44>:  adrp   x8, 7790
14.     0x10143dfd4 <+48>:  ldr    x1, [x8, #0xa68]
15.     0x10143dfd8 <+52>:  bl     0x1029b14d0               ; symbol stub for: objc_msgSend
16.     0x10143dfdc <+56>:  cmp    x0, #0x1                  ; =0x1
17.     0x10143dfe0 <+60>:  b.ne   0x10143e130               ; <+396> at NCMP4Recorder.m:157:1
18.     0x10143dfe4 <+64>:  adrp   x8, 7804
19.     0x10143dfe8 <+68>:  ldr    x23, [x8, #0x9c8]
20.     0x10143dfec <+72>:  ldr    x0, [x20, #0x10]
21.     0x10143dff0 <+76>:  mov    x1, x23
22.     0x10143dff4 <+80>:  bl     0x1029b14d0               ; symbol stub for: objc_msgSend
23. ->  0x10143dff8 <+84>:  cbz    w0, 0x10143dfec           ; <+72> at NCMP4Recorder.m:135:13
24.     0x10143dffc <+88>:  stp    xzr, xzr, [sp, #0x30]
25.     0x10143e000 <+92>:  sbfiz  x24, x21, #2, #32
26.     0x10143e004 <+96>:  adrp   x8, 7006
27.     0x10143e008 <+100>: ldr    x8, [x8, #0x9a0]
28.     0x10143e00c <+104>: ldr    x23, [x8]
29.     0x10143e010 <+108>: add    x8, sp, #0x38             ; =0x38
30.     0x10143e014 <+112>: str    x8, [sp]
31.     0x10143e018 <+116>: mov    w7, #0x0
32.     0x10143e01c <+120>: mov    x0, x23
33.     0x10143e020 <+124>: mov    x1, #0x0
34.     0x10143e024 <+128>: mov    x2, x24
35.     0x10143e028 <+132>: mov    x3, x23
36.     0x10143e02c <+136>: mov    x4, #0x0
37.     0x10143e030 <+140>: mov    x5, #0x0
38.     0x10143e034 <+144>: mov    x6, x24
39.     0x10143e038 <+148>: bl     0x1029af9dc               ; symbol stub for: CMBlockBufferCreateWithMemoryBlock
40.     0x10143e03c <+152>: cbz    w0, 0x10143e054           ; <+176> at NCMP4Recorder.m:145:36
41.     0x10143e040 <+156>: str    x0, [sp]
42.     0x10143e044 <+160>: adrp   x0, 7583
43.     0x10143e048 <+164>: add    x0, x0, #0x238            ; =0x238
44.     0x10143e04c <+168>: bl     0x1029afce8               ; symbol stub for: NSLog
45.     0x10143e050 <+172>: b      0x10143e130               ; <+396> at NCMP4Recorder.m:157:1
46.     0x10143e054 <+176>: ldr    x0, [sp, #0x38]
47.     0x10143e058 <+180>: bl     0x1029af9c4               ; symbol stub for: CMBlockBufferAssureBlockMemory
48.     0x10143e05c <+184>: ldr    x1, [sp, #0x38]
49.     0x10143e060 <+188>: mov    x0, x22
50.     0x10143e064 <+192>: mov    x2, #0x0
51.     0x10143e068 <+196>: mov    x3, x24
52.     0x10143e06c <+200>: bl     0x1029afa00               ; symbol stub for: CMBlockBufferReplaceDataBytes
53.     0x10143e070 <+204>: ldr    x0, [x20, #0x20]
54.     0x10143e074 <+208>: bl     0x1029af9ac               ; symbol stub for: CMAudioFormatDescriptionGetStreamBasicDescription
55.     0x10143e078 <+212>: ldr    x22, [sp, #0x38]
56.     0x10143e07c <+216>: ldr    x24, [x20, #0x20]
57.     0x10143e080 <+220>: ldr    w8, [x0, #0x1c]
58.     0x10143e084 <+224>: udiv   w21, w21, w8
59.     0x10143e088 <+228>: add    x8, sp, #0x18             ; =0x18
60.     0x10143e08c <+232>: mov    w1, #0xca00
61.     0x10143e090 <+236>: movk   w1, #0x3b9a, lsl #16
62.     0x10143e094 <+240>: mov    x0, x19
63.     0x10143e098 <+244>: bl     0x1029afa90               ; symbol stub for: CMTimeMake
64.     0x10143e09c <+248>: add    x4, sp, #0x18             ; =0x18
65.     0x10143e0a0 <+252>: add    x6, sp, #0x30             ; =0x30
66.     0x10143e0a4 <+256>: mov    x0, x23
67.     0x10143e0a8 <+260>: mov    x1, x22
68.     0x10143e0ac <+264>: mov    x2, x24
69.     0x10143e0b0 <+268>: mov    x3, x21
70.     0x10143e0b4 <+272>: mov    x5, #0x0
71.     0x10143e0b8 <+276>: bl     0x1029af9b8               ; symbol stub for: CMAudioSampleBufferCreateReadyWithPacketDescriptions
72.     0x10143e0bc <+280>: cbz    w0, 0x10143e0d4           ; <+304> at NCMP4Recorder.m:151:15
73.     0x10143e0c0 <+284>: str    x0, [sp]
74.     0x10143e0c4 <+288>: adrp   x0, 7583
75.     0x10143e0c8 <+292>: add    x0, x0, #0x278            ; =0x278
76.     0x10143e0cc <+296>: bl     0x1029afce8               ; symbol stub for: NSLog
77.     0x10143e0d0 <+300>: b      0x10143e128               ; <+388> at NCMP4Recorder.m:156:15
78.     0x10143e0d4 <+304>: ldr    x0, [x20, #0x10]
79.     0x10143e0d8 <+308>: ldr    x2, [sp, #0x30]
80.     0x10143e0dc <+312>: adrp   x8, 7803
81.     0x10143e0e0 <+316>: ldr    x1, [x8, #0x9e0]
82.     0x10143e0e4 <+320>: bl     0x1029b14d0               ; symbol stub for: objc_msgSend
83.     0x10143e0e8 <+324>: tbnz   w0, #0x0, 0x10143e120     ; <+380> at NCMP4Recorder.m:153:19
84.     0x10143e0ec <+328>: ldr    x0, [x20, #0x38]
85.     0x10143e0f0 <+332>: adrp   x8, 7790
86.     0x10143e0f4 <+336>: ldr    x1, [x8, #0x4d0]
87.     0x10143e0f8 <+340>: bl     0x1029b14d0               ; symbol stub for: objc_msgSend
88.     0x10143e0fc <+344>: mov    x29, x29
89.     0x10143e100 <+348>: bl     0x1029b1530               ; symbol stub for: objc_retainAutoreleasedReturnValue
90.     0x10143e104 <+352>: mov    x20, x0
91.     0x10143e108 <+356>: stp    x19, x0, [sp]
92.     0x10143e10c <+360>: adrp   x0, 7583
93.     0x10143e110 <+364>: add    x0, x0, #0x258            ; =0x258
94.     0x10143e114 <+368>: bl     0x1029afce8               ; symbol stub for: NSLog
95.     0x10143e118 <+372>: mov    x0, x20
96.     0x10143e11c <+376>: bl     0x1029b1500               ; symbol stub for: objc_release
97.     0x10143e120 <+380>: ldr    x0, [sp, #0x30]
98.     0x10143e124 <+384>: bl     0x1029af4d8               ; symbol stub for: CFRelease
99.     0x10143e128 <+388>: ldr    x0, [sp, #0x38]
100.     0x10143e12c <+392>: bl     0x1029af4d8               ; symbol stub for: CFRelease
101.     0x10143e130 <+396>: ldp    x29, x30, [sp, #0x70]
102.     0x10143e134 <+400>: ldp    x20, x19, [sp, #0x60]
103.     0x10143e138 <+404>: ldp    x22, x21, [sp, #0x50]
104.     0x10143e13c <+408>: ldp    x24, x23, [sp, #0x40]
105.     0x10143e140 <+412>: add    sp, sp, #0x80             ; =0x80
106.     0x10143e144 <+416>: ret  
107.  

which looks to me like it's failing on the native side, and not Unity's end.


Here is my recording code, which is almost the same as the sample found here:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using System.Linq;
4. using UnityEngine;
5. using UnityEngine.Events;
6. using NatCorder;
7. using NatCorder.Inputs;
8. using NatCorder.Clocks;
9. using System.IO;
10. using NatDevice;
11.  
12. public class CaptureController : MonoBehaviour
13. {
14.     #region Singleton
15.     private static CaptureController instance;
16.     public static CaptureController Instance {
17.         get { return instance; }
18.     }
19.     #endregion
20.  
21.     #region Events
22.     public class VideoCaptureComplete : UnityEvent <string> {};
23.     public VideoCaptureComplete OnVideoCaptureComplete = new VideoCaptureComplete();
24.  
25.     public class PhotoCaptureComplete : UnityEvent <string> {};
26.     public PhotoCaptureComplete OnPhotoCaptureComplete = new PhotoCaptureComplete();
27.     #endregion
28.  
29.     #region Variables
30.     [SerializeField] private float frameRate = 30.0f;
31.     /*
32.         We can play with the width/height values. Not sure if we can record full screen - I remember this making devices crash but
33.         we can do some testing.
34.     */
35.     [SerializeField] private int width = 1080;
36.     [SerializeField] private int height = 1920;
37.     [SerializeField] private Camera screenShotCamera;
38.     [SerializeField] private RenderTexture screenShotRenderTexture;
39.     [SerializeField] private AudioListener audioListener;
40.     private bool isRecording = false;
41.  
42.     #if UNITY_EDITOR
43.     [SerializeField] private bool deleteInEditor = false;
44.     #endif
45.  
46.     private IMediaRecorder videoRecorder;
47.     private CameraInput cameraInput;
48.     private AudioInput audioInput;
49.     private IAudioDevice microphoneAudio;
50.     #endregion
51.  
52.     #region Mono
53.     private void Awake() {
54.         if(instance != null){
55.             DestroyImmediate(gameObject);
56.             return;
57.         }
58.  
59.         instance = this;
60.     }
61.  
62.     private void OnDestroy() {
63.         if(instance == this){
64.             instance = null;
65.         }
66.     }
67.  
68.     private void Start(){
69.         screenShotRenderTexture.width = Screen.width;
70.         screenShotRenderTexture.height = Screen.height;
71.         RequestMicPermission();
72.     }
73.     #endregion
74.  
75.  
76.     #region Capture Methods
77.     async void RequestMicPermission(){
78.         if(!await MediaDeviceQuery.RequestPermissions<IAudioDevice>()){
79.             Debug.Log("No Audio Permission");
80.         }
81.     }
82.  
83.     public void StartRecording(){
84.         MediaDeviceQuery query = new MediaDeviceQuery();
85.         microphoneAudio = query.devices.FirstOrDefault(device => device is IAudioDevice) as IAudioDevice; //This is weird syntax but ok
86.         // Debug.Log(microphoneAudio.sampleRate);
87.         var clock = new RealtimeClock();
88.         videoRecorder = new MP4Recorder(width, height, frameRate, microphoneAudio.sampleRate, microphoneAudio.channelCount);
89.         cameraInput = new CameraInput(videoRecorder, clock, Camera.main);
90.         //audioInput = new AudioInput(videoRecorder, clock, audioListener);
91.         if(microphoneAudio != null){
92.             Debug.Log("recording audio");
93.             microphoneAudio.StartRunning((sampleBuffer, timestamp) => {
94.                 if(videoRecorder != null && isRecording){
95.                     videoRecorder.CommitSamples(sampleBuffer, clock.timestamp);
96.                 }
97.             });
98.         }
99.  
100.         isRecording = true;
101.     }
102.  
103.     public async void StopRecording(){
104.         if(microphoneAudio != null){
105.             microphoneAudio.StopRunning();
106.         }
107.         cameraInput.Dispose();
108.         Debug.Log("Calling Finish writing");
109.      
110.         string path = await videoRecorder.FinishWriting();
111.         isRecording = false;
112.         Debug.Log("Video Saved to Path: " + path);
113.         OnVideoCaptureComplete.Invoke(path);
114.  
115.         #if UNITY_EDITOR
116.             if(deleteInEditor){
117.                 File.Delete(path);
118.                 Debug.Log("Deleted: " + path);
119.             }
120.         #endif
121.  
122.     }
123.     #endregion
124. }
125.  

 

A few things you need to fix with your code:

1. Set `isRecording = false` before calling `FinishWriting`. `FinishWriting` is async, so your code likely tries to commit an extra audio sample while the recorder is finishing, hence the crash.
2. You can simplify how you get a microphone by using `new MediaDeviceQuery(MediaDeviceQuery.Criteria.AudioDevice)`. See the README for more info.
3. You get the audio device `microphoneAudio`, try to start recording with its sample rate and channel count, then later on check if it's not null before streaming its audio to the recorder. You should be doing that null check before you create the recorder.

 

Thanks for your quick response. Seems setting isRecording to false at the beginning of my StopRecording function has fixed the issue. I've also updated my microphoneAudio null checks and will create an MP4Recorder without audio if AudioDevice does not exist.

 

I am also seeing the issue where new MediaDeviceQuery(MediaDeviceQuery.Criteria.FrontFacing) does not find any camera devices. Seen on iphone 6 and 11.

Falling back to MediaDeviceQuery(MediaDeviceQuery.Criteria.CameraDevice) will find the rear camera.

 

Fix coming in the next update.

 

Unity 2019.3.3f1
platform: ios
xCode: 11.1
Empty project with NatCoder, NatDevice, we get error (image 1)
Project settings : (image 2 3)

script and scene from examples, only I add in scene and script ability to change input type

Code (CSharp):

1. /*
2. *   NatSuite Demo
3. *   Copyright (c) 2020 Yusuf Olokoba.
4. */
5.  
6. namespace NatSuite
7. {
8.  
9.     using UnityEngine;
10.     using UnityEngine.UI;
11.     using System.Linq;
12.     using NatCorder;
13.     using NatCorder.Clocks;
14.     using NatCorder.Inputs;
15.     using NatDevice;
16.  
17.     public class NatSuiteDemo : MonoBehaviour
18.     {
19.  
20.         #region --Op vars--
21.  
22.         [Header("UI")]
23.         public RawImage rawImage;
24.         public AspectRatioFitter aspectFitter;
25.  
26.         [Header("Sharing")]
27.         public bool shareRecording;
28.  
29.         private IAudioDevice audioDevice;
30.         private MP4Recorder recorder;
31.         private CameraInput cameraInput;
32.         [SerializeField] private AudioSource source;
33.         #endregion
34.  
35.         public async void Start()
36.         { // Invoked by Unity when the scene opens
37.             // Request permissions
38.             if (!await MediaDeviceQuery.RequestPermissions<IAudioDevice>())
39.             {
40.                 Debug.LogError("User did not grant microphone permission");
41.                 return;
42.             }
43.             if (!await MediaDeviceQuery.RequestPermissions<ICameraDevice>())
44.             {
45.                 Debug.LogError("User did not grant camera permission");
46.                 return;
47.             }
48.             // Get a microphone and camera
49.             var query = new MediaDeviceQuery();
50.             var microphone = query.devices.FirstOrDefault(device => device is IAudioDevice) as IAudioDevice;
51.             audioDevice = new MixerDevice(microphone, source);
52.  
53.             var cameraDevice = query.devices.FirstOrDefault(device => device is ICameraDevice) as ICameraDevice;
54.             // Start the camera preview
55.             var previewTexture = await cameraDevice.StartRunning();
56.             rawImage.texture = previewTexture;
57.             aspectFitter.aspectRatio = previewTexture.width / (float)previewTexture.height;
58.         }
59.  
60.         public void StartRecording()
61.         {
62.             // Create a recorder
63.             recorder = new MP4Recorder(720, 1280, 30, audioDevice.sampleRate, audioDevice.channelCount);
64.             // Start recording
65.             var clock = new RealtimeClock();
66.             cameraInput = new CameraInput(recorder, clock, Camera.main);
67.             audioDevice.StartRunning((sampleBuffer, timestamp) => recorder.CommitSamples(sampleBuffer, clock.timestamp));
68.         }
69.  
70.         public async void StopRecording()
71.         {
72.             // Stop the recorder inputs
73.             audioDevice.StopRunning();
74.             cameraInput.Dispose();
75.             // Stop recording and get the path
76.             var path = await recorder.FinishWriting();
77.             Debug.Log($"Saved recording to: {path}");
78.             // Share the recording
79.             // if (shareRecording)
80.             // {
81.             //     var payload = new SharePayload();
82.             //     payload.AddMedia(path);
83.             //     await payload.Commit();
84.             // }
85.             // Playback the recording
86.             if (Application.platform == RuntimePlatform.IPhonePlayer)
87.                 path = "file://" + path;
88.             Handheld.PlayFullScreenMovie(path);
89.         }
90.     }
91. }
92.  

Unity 2019.2.21f : (image 4)
Unity 2018.4.11 : also error

On Android all work.

 

When will the new update be released?

 

Hi Lanre,

When i try to build to iOS, it works fine directly from XCode, but my builds on Unity Cloud and if i try to archive in XCode i get this error:

ld: bitcode bundle could not be generated because '/Users/aawezz/Jabii/Builds/Jabii iOS/Libraries/Plugins/NatDevice/Plugins/iOS/libNatDevice.a(Bridge.o)' was built without full bitcode. All object files and libraries for bitcode must be generated from Xcode Archive or Install build for architecture arm64

clang: error: linker command failed with exit code 1 (use -v to see invocation)

Im using xcode version 11.3.1.
I tried building for Universal and armv64 on ios, both get this error.
Im using Unity version 2019.3.

Regards,
Anders

 

Hi Lanre,

Any news on when the StopPreview fix in NatCam v2.3.1 will become available on the Unity Asset Store?

I'm concerned that the NatCam package no longer appears in Asset Store searches and that it now shows with a Deprecated label when viewing in the My Assets list. I am unable to release my app with this bug and really could do with the fix becoming available ASAP.

 

Hi @Lanre we're also rather desperate for a fix for the cameraDevice.StopPreview() bug. Would be great a rough idea of when a fix will be available so we can plan. Many thanks!

 

@Lanre Hi, I'm using NatDevice to make a karaoke game. In my case, I need to hear my voice when recording. So is it possible to replay the clip on real-time when recording through microphone with NatDevice?
We could do that with the demo code from Unity's Demo, but with noticable latency:

Code (CSharp):

1. audioSource.clip = Microphone.Start(microphone, true, 1, audioSampleRate);
2. audioSource.loop = true;
3. audioSource.Play();

Could we do the similar thing with NatDevice but with less latency? Thank you.

 

Are you running on an iPhone 6? If so this is a known issue. I am looking into it.

 

Submitting this weekend, so should be approved early to mid next week.

 

Fix for this is coming in the next update. Sorry for the inconvenience.

 

NatCam and NatMic have been succeeded by NatDevice. Grab the update on the Asset Store.

 

You could just record the microphone audio, as normal. NatDevice exhibits very little latency compared to Unity's Microphone API.

 

Hi Lanre,

So NatCam is now deprecated? Is there an upgrade path from NatCam to NatDevice? Have only had NatCam for 6 months and didn't expect to be forced into buying a new asset so soon.

 

Ah, just checked and I can see I can purchase NatDevice for a reduced rate. Please disregard my previous message, should have looked first.

 

Started playing around with your plugin. Tried to run the Minicam example scene on my MacBook with Catalina and Unity 2019.2.21f1 and it crashes as soon as it starts. What would cause this and how would I able to debug functionalities without building for my phone every time?

 

You might need to manually grant Unity camera permissions. The permission dialog should pop up automatically, but maybe it didn't in your case.

 

Thanks, for the fast reply. Unity should have camera permission, at least I checked in the Security & Privacy settings.