NatDevice - Media Device API

I will need logs to know what is going on, but since it is a previous version, we don't offer support for it. Upgrade to the most recent version because it is most stable.

 

Hello Lanre,
I'm trying to limit the weight of the videos (in order to upload them).

However a 1 minute video = 124mb for both "medium (480X640)" and "HD (720X1280)" presets.
But... In medium quality it should be 2 times lighter like 62mb right ?

Other thing : in the NatCamTypes.cs script I'm not sure about the math behind the default bitrate var :
(11.4f * 1280 * 720)

What is the 11.4f number ? Should I set (11.4f * 640 * 480) instead for a medium video ?

Thank you very much for your help !
Seb

 

It is a ballpark estimate of the multiplier for a "standard" resolution. As you have suspected, the bitrate parameter will control the recording size (and also, quality).

 

Thank you for the explanation

But what seems strange to me is that if I less the same bitrate (Ex : 10506240)

• 1 minute video in 720X1280 => 124 mb
• 1 minute video in 480X640 => 124 mb

The medium quality should be lighter ?

 

Imagine the bitrate as a ceiling for how much data the encoder should output. The value used is very high (10 million+), and so the encoder might never need to produce that much data at all. If you reduce the value to be proportional to the resolution, then you will see a difference in the sizes.

 

Hi again Lanre, I updated the lastest version and it's working on android, but for IOS I got this new error :


Do you have any ideas ?
Seb

 

Apparently, this is not a NatCorder error. You might have to reinstall Xcode.

 

It's very strange because I just tried again : the previous version of natCam is compiling to the Iphone without any issues, and the new one shows me this error (with the miniCam exemple)...
I will dig and I will post if I find the answer.
Btw: The target minimum iOS version is still 7.0 right ?

Edit : Targeting the 9.3 is better, but I still have the "framework not found IOSurface for architecture armv7".
I have Xcode 9.3 (the last version)

 

Yes, the minimum iOS version hasn't changed.

I'm not sure why IOSurface.framework is being linked in the first place. As far as I can tell, it isn't a NatCam dependency; nor is it a Unity dependency.

 

Hi all!

Check out a free sharing API we created for iOS and Android. NatShare allows you to save photos and videos to the gallery, to share photos and videos, and to retrieve video thumbnails.

NatShare on GitHub

Make sure to check out the release article.
___

We have significant new developments coming later this week, along with a new release: NatCam 2.0.

 

Hi all!

Here's the big news I talked about in my post above:

Goodbye, NatCam Core!

In a nutshell, we are deprecating NatCam Core; deprecating video recording from NatCam Pro because we have developed a dedicated video recording API called NatCorder; reducing the price of NatCam Pro; and renaming NatCam Pro to NatCam.
___

If you have NatCam Core:
- We will be allowing free upgrades to NatCam Pro starting next week.
- We will be deprecating the package in 6-8 weeks from now. When the package is deprecated, you will not be able to upgrade. So make sure you upgrade sooner rather than later.

We will keep you updated as we go. Thank you!

 

Hi @Lanre ,

I just integrated NatCam 2.0f1 in an empty project with Unity 2017.3.1p1 and I am having an issue with the camera preview not working for the first time you start the camera. The preview starts working after capturing a photo in the MiniCam Scene.

This issue is replicated in all the sample scenes. Can you help with this?

 

As a follow up, I identified that the NatCam.OnStart callback is not being called.

 

What platform does this occur on?

 

In the Windows Editor, the NatCam.OnStart() is not called due to line 136 of NatCamLegacy:

if (!PreviewTexture.didUpdateThisFrame || PreviewTexture.width + PreviewTexture.height == 16 << 1) return;

I am also experiencing some issues with NatCam together with NatCorder on Android, will report back to you as soon as I dig a bit more.

 

@Lanre , when targetting Graphics API OpenGLES2 in Android 6 (haven't tried other Androids yet) the following errors occur for previewing the camera:

Spoiler

04-19 18:42:26.697 22238 22252 D Unity   : NatCam: Changed camera 0 resolution to 1280x720

04-19 18:42:26.698 22238 22252 D Unity   : NatCam: Changed camera 0 photo resolution to 1920x1080

04-19 18:42:26.839 22238 22345 D Unity   : NatCam Rendering: Created ES2 GLRenderContext

04-19 18:42:26.846 22238 22345 D Unity   : NatCam Rendering Error: Failed to load ES2 FastRead context

04-19 18:42:27.409 22238 22312 D Unity   : NatCam: Camera 0 started session

04-19 18:42:27.585 22238 22345 D Unity   : NatCam: Detected first frame flag with preview size: 720x1280 rotation: 1

04-19 18:42:27.591 22238 22345 D Unity   : NatCam Rendering Error: Performed blit with error 1286

04-19 18:42:27.650 22238 22345 D Unity   : NatCam Rendering Error: Performed blit with error 1286

04-19 18:42:27.706 22238 22345 D Unity   : NatCam Rendering Error: Performed blit with error 1286

04-19 18:42:27.763 22238 22345 D Unity   : NatCam Rendering Error: Performed blit with error 1286

04-19 18:42:27.825 22238 22345 D Unity   : NatCam Rendering Error: Performed blit with error 1286

Capturing the photo works despite this issue. Only the preview is not working.

 

Hi Henrique. This is not supported. When running on a device that supports OpenGL ES3, NatCam expects that your OpenGL ES context be an ES3 context. This constraint exists because Google decided to restrict access to system libraries in Android 7.

 

This line should be:

Code (CSharp):

1. if (!PreviewTexture.didUpdateThisFrame || PreviewTexture.width == 16 || PreviewTexture.height == 16) return;

 

Hi all!

NatCam 2.0 is out! If you use NatCam Core, please upgrade for free

Thank you!

 

Hi Lanre.

I merged the pull request to correspond to new NatCam 2.0f1 to my GitHub repository.
https://github.com/EnoxSoftware/NatCamWithOpenCVForUnityExample

Two problems occurred in NatCam 2.0f1.
The first one is the problem of line 136 of NatCamLegacy.cs as henriqueranj says.
Second, the screen orientation event does not occur when after moving scenes. I think that this problem is probably related to the new OrientationUtility.cs.

 

Thank you for the instructions, @Lanre. However, the previous version of NatCam did allow this to happen we were forcing OpenGLES2 to all our devices, was there an internal change to how this works?

Further, I have now found an issue with the NatCam preview on the Samsung Galaxy Note 10.1 (2014) tablet with Android 5.1. The camera preview is only showing the first frame and not updating anymore. Photo capture still works well.

This can be replicated with an empty Unity 2017.3.1p1 project with the NatCam samples Scenes. The log is the following:

D/Unity   (24603): NatCam: Changed camera 0 resolution to 1280x720

D/Unity   (24603): NatCam: Changed camera 0 photo resolution to 1920x1080

D/Unity   (24603): NatCam Rendering: Created ES3 GLRenderContext

D/Unity   (24603): NatCam: Camera 0 started session

D/Unity   (24603): NatCam: Detected first frame flag with preview size: 720x1280 rotation: 1

 

I have pushed an update with both of these fixes. It is waiting to be approved by Asset Store Review. I will let you know when it is released. Thanks for merging!

 

There wasn't a change. Technically, you can still use ES2. To do so, open NatCamAndroid.cs in NatCam > Plugins > Managed > Platforms > Android and disable the `PreviewData` flag. Just keep in mind that this will cause the `NatCam.CaptureFrame` function to not work on Android.

Can you email me the full (unfiltered) logs from logcat? I'm not sure what could be causing this.

 

Hi Lanre, I figured out that this was an issue with Unity not making builds with my latest Graphics API changes in the Player Settings (it got stuck in the old settings for OpenGLES3). Therefore, it was using OpenGLES3 in this Android Tablet. Today when I performed a build it started working well again.

It can be seen in the logs that he is now using OpenGLES2. No changes in the code or NatCam were needed.

D/Unity   (13670): NatCam: Changed camera 1 resolution to 1280x720

D/Unity   (13670): NatCam: Changed camera 1 photo resolution to 1280x720

D/Unity   (13670): NatCam: Changed camera 1 framerate to 30000

D/Unity   (13670): NatCam Rendering: Created ES2 GLRenderContext

D/Unity   (13670): NatCam: Camera 1 started session

D/Unity   (13670): NatCam: Detected first frame flag with preview size: 720x1280 rotation: 1

 

That's a weird bug on ES3. But I'm glad you figured it out!

 

Hi - I'm bidding on a project where I need to control both exposure and "shutter" speed. It looks like this plugin covers the aperture part, but can I specify the length of the exposure somehow? I didn't see anything in the documentation but thought I'd ask...

Thanks!

 

NatCam does not support setting the shutter speed. It only supports setting the exposure bias (in EV's).

 

Hi Lanre.

I updated my GitHub repository by incorporating the change points of new NatCam API.
https://github.com/EnoxSoftware/NatCamWithOpenCVForUnityExample

This works well on Android and iOS, but in the Windows Editor, the NatCam.Release () gets the error every time. (Line 142 of NatCamLegacy.cs)

 

Hei there,

I am trying to update to latest v2.0. The Asset Store tells that the current version is 2.0, I can import and it shows that only the scene will be updated and in the end I only get v1.6.

I have been trying to delete the existing plugin in the project, to no avail. I purchased the Pro just in case that helps.
I tried contacting the support for Asset Store but they seem a little slow and I really need to get this done asap...

Thanks

 

I have submitted a fix for this. Thanks for letting me know.

 

I have had multiple reports of this. I am also reaching out to them.

 

Hi, is there a demo APK for NatCam that is available for download to test out on an actual device? Just curious to see if this will meet my needs before I purchase.

 

Hello !
Bug running on iPhone 6s, preview size request 1280x720

libc++abi.dylib: terminating with uncaught exception of type Il2CppExceptionWrapper
SIGABRT received (abort() call) -- WHOA! I did something nasty. Dumping the stack trace:
0 encolor 0x00000001031a29ac __sig_handler + 180
1 libsystem_platform.dylib 0x0000000189a1d338 _sigtramp + 36
2 libsystem_pthread.dylib 0x0000000189a23450 pthread_kill + 112
3 libsystem_c.dylib 0x00000001898cf400 abort + 140
4 libc++abi.dylib 0x00000001893992d4 __cxa_bad_cast + 0
5 libc++abi.dylib 0x00000001893b6cc0 <redacted> + 0
6 libobjc.A.dylib 0x00000001893c4860 <redacted> + 152
7 libc++abi.dylib 0x00000001893b366c <redacted> + 16
8 libc++abi.dylib 0x00000001893b2f84 <redacted> + 0
9 encolor 0x00000001030e426c il2cpp::vm::Exception::RaiseOutOfMemoryException() + 0
10 encolor 0x0000000102a0fc54 Scripting::RaiseIfNull(ScriptingObjectPtr) + 0
11 encolor 0x0000000102c9f750 ScriptingObjectOfType<Object>::GetReference() const + 0
12 encolor 0x0000000102caa110 Texture_CUSTOM_Internal_GetWidth(Il2CppObject*) + 24
13 encolor 0x0000000100ffa088 NatCamiOS_onStart_m144070083 + 116
14 encolor 0x0000000100ff9fe4 ReversePInvokeWrapper_NatCamiOS_onStart_m144070083 + 56
15 encolor 0x000000010319ff4c -[NatCam captureOutput:didOutputSampleBuffer:fromConnection:] + 1324
16 AVFoundation 0x000000019217f6bc <redacted> + 308
17 AVFoundation 0x000000019217f4f8 <redacted> + 100
18 CoreMedia 0x000000018d2e4d94 <redacted> + 260
19 CoreMedia 0x000000018d303dcc <redacted> + 216
20 libdispatch.dylib 0x00000001898161bc <redacted> + 16
21 libdispatch.dylib 0x0000000189822f94 <redacted> + 576
22 libdispatch.dylib 0x000000018982f634 <redacted> + 204
23 libdispatch.dylib 0x0000000189818160 <redacted> + 820
24 libdispatch.dylib 0x0000000189824210 <redacted> + 468
25 libdispatch.dylib 0x00000001898199a4 <redacted> + 652
26 libdispatch.dylib 0x000000018981abbc _dispatch_main_queue_callback_4CF + 572
27 CoreFoundation 0x000000018a93a810 <redacted> + 12
28 CoreFoundation 0x000000018a9383fc <redacted> + 1660
29 CoreFoundation 0x000000018a8662b8 CFRunLoopRunSpecific + 444
30 GraphicsServices 0x000000018c31a198 GSEventRunModal + 180
31 UIKit 0x00000001908ad7fc <redacted> + 684
32 UIKit 0x00000001908a8534 UIApplicationMain + 208
33 encolor 0x00000001000ae3dc _rdc_main(int, char**) + 160
34 encolor 0x00000001031a2670 main + 764
35 libdyld.dylib 0x00000001898495b8 <redacted> + 4

 

Here is the MiniCam example APK.

 

Your app seems to have run out of memory:
9 encolor 0x00000001030e426c il2cpp::vm::Exception::RaiseOutOfMemoryException() + 0
What is your app's memory consumption right now?

 

I sent you a new email with more info, but if I had to guess the problem hdxpmc was having has to do with

NatCam.Camera.PhotoResolution = CameraResolution.Lowest; 

results in the camera being set to 3264x1836.

 

This is possible, but only if multiple photos have been taken, but not freed (causing a memory leak). Make sure to call Texture2D.Destroy on photos that are not being used any longer.

 

On iOS, if you launch the Mini Cam example again after going into app settings, and disabling the camera permission, the app crashes on launch.

Code (CSharp):

1.  
2. 2018-05-04 13:58:56.920278-0400 myApp[564:162986] NatCam: Changed camera format for photo resolution: (1920, 1080)
3. 2018-05-04 13:58:57.019102-0400 myApp[564:162986] Uncaught exception: NSInvalidArgumentException: *** -[AVCaptureSession addInput:] Can't add a nil AVCaptureInput
4. (
5.    0   CoreFoundation                      0x000000018354eda4 <redacted> + 252
6.    1   libobjc.A.dylib                     0x00000001827085ec objc_exception_throw + 56
7.    2   AVFoundation                        0x000000018903bbe4 <redacted> + 912
8.    3   myApp                                0x0000000101bdcbbc +[NatCam initializeSession] + 2536
9.    4   myApp                                0x0000000101bdadb4 +[NatCam play] + 40
10.    5   myApp                                0x0000000101bd802c NCCorePlay + 40
11.    6   myApp                                0x00000001003236a4 MiniCam_Start_m4217845606 + 320
12.    7   myApp                                0x0000000100f7c89c _Z31RuntimeInvoker_Void_t1185182177PFvvEPK10MethodInfoPvPS4_ + 20
13.    8   myApp                                0x0000000101af10c4 _ZN6il2cpp2vm7Runtime6InvokeEPK10MethodInfoPvPS5_PP15Il2CppException + 68
14.    9   myApp                                0x000000010146eb20 _Z23scripting_method_invoke18ScriptingMethodPtr18ScriptingObjectPtrR18ScriptingArgumentsP21ScriptingExceptionPtrb + 100
15.    10  myApp                                0x00000001014761f8 _ZN19ScriptingInvocation6InvokeEP21ScriptingExceptionPtrb + 60
16.    11  myApp                                0x000000010147bc98 _ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtrP21ScriptingExceptionPtr + 1180
17.    12  myApp                                0x000000010147be94 _ZN13MonoBehaviour30InvokeMethodOrCoroutineCheckedE18ScriptingMethodPtr18ScriptingObjectPtr + 84
18.    13  myApp                                0x000000010147af48 _ZN13MonoBehaviour16DelayedStartCallEP6ObjectPv + 80
19.    14  myApp                                0x00000001012ef3a0 _ZN18DelayedCallManager6UpdateEi + 636
20.    15  myApp                                0x00000001013aadac _Z10PlayerLoopv + 452
21.    16  myApp                                0x00000001011e564c _ZL19UnityPlayerLoopImplb + 36
22.    17  myApp                                0x00000001001e72d8 UnityRepaint + 140
23.    18  myApp                                0x00000001001e71c4 -[UnityAppController(Rendering) repaintDisplayLink] + 88
24.    19  GPUToolsCore                        0x0000000105c645f0 -[DYDisplayLinkInterposer forwardDisplayLinkCallback:] + 176
25.    20  QuartzCore                          0x00000001875ae3c0 <redacted> + 832
26.    21  IOKit                               0x00000001837b78a0 IODispatchCalloutFromCFMessage + 492
27.    22  CoreFoundation                      0x00000001834dcb20 <redacted> + 188
28.    23  CoreFoundation                      0x00000001834f7ae8 <redacted> + 56
29.    24  CoreFoundation                      0x00000001834f7230 <redacted> + 440
30.    25  CoreFoundation                      0x00000001834f4c80 <redacted> + 2456
31.    26  CoreFoundation                      0x0000000183414da8 CFRunLoopRunSpecific + 552
32.    27  GraphicsServices                    0x00000001853f7020 GSEventRunModal + 100
33.    28  UIKit                               0x000000018d3f578c UIApplicationMain + 236
34.    29  myApp                                0x00000001001e0020 main + 160
35.    30  libdyld.dylib                       0x0000000182ea5fc0 <redacted> + 4
36.  

 

Also, the GreyCam example crashes on launch on Android. The above issue is more important though.

Code (CSharp):

1. 05-04 13:26:31.401 9432-9463/? E/AndroidRuntime: FATAL EXCEPTION: Thread-11
2.                                                  Process: com.test.myApp, PID: 9432
3.                                                  java.lang.NullPointerException: Attempt to invoke virtual method 'int com.yusufolokoba.natcam.NatCamDevice.getPreviewHeight()' on a null object reference
4.                                                      at com.yusufolokoba.natcam.NatCam.previewWidth(NatCam.java:239)
5.                                                      at com.yusufolokoba.natcam.NatCam.access$100(NatCam.java:27)
6.                                                      at com.yusufolokoba.natcam.NatCam$1.invoke(NatCam.java:84)
7.                                                      at com.yusufolokoba.natcam.rendering.RenderDispatch.invoke(RenderDispatch.java:90)
8.  

 

NatCam does not check for camera permission as it expects that the user has granted it. Hence you can prevent this crash by checking:

Code (CSharp):

1. if (NatCam.Implementation.HasPermissions) {
2.     // Start camera...
3. }

 

This is actually a bug in our GreyCam code. In the Start function of GreyCam, change `NatCam.Play();` to `NatCam.Play(camera);`.

 

Awesome, thank you very much for the fast reply and great help!

 

Another question: Is there anyway to simulate or test the Nat Cam from within the Unity Editor using a web cam?

 

Yes, it is possible. Just have a webcam hooked up to your pc and make your default cam settings use the Front camera. You should get an error message if this is not working.

 

Great, thank you!

 

I'm seeing the below error pop up a lot with NatCam. At line 603, (in my update function), I'm just checking NatCam.IsPlaying. Has anyone encountered and solved this issue? I'm using Unity Editor 2017.4.212.

Thanks in advance for any thoughts...
D


UnityException: Internal_CreateGameObject is not allowed to be called from a MonoBehaviour constructor (or instance field initializer), call it in Awake or Start instead. Called from MonoBehaviour 'DispatchUtility' on game object 'NatCam Dispatch Utility'.
See "Script Serialization" page in the Unity Manual for further details.
UnityEngine.GameObject..ctor (System.String name) (at /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/GameObjectBindings.gen.cs:392)
NatCamU.Dispatch.DispatchUtility..cctor () (at Assets/NatCam/Core/Plugins/Managed/Dispatch/DispatchUtility.cs:28)
Rethrow as TypeInitializationException: An exception was thrown by the type initializer for NatCamU.Dispatch.DispatchUtility
NatCamU.Core.NatCam..cctor () (at Assets/NatCam/Core/Plugins/Managed/NatCam.cs:140)
Rethrow as TypeInitializationException: An exception was thrown by the type initializer for NatCamU.Core.NatCam
CmrdCam.WebCam.Update () (at Assets/Scripts/WebCam.cs:603)

 

Can you share your CmrdCam.WebCam code? I need to know how early NatCam is being accessed.

 

Actually, there will be a bunch of errors when running in editor.
Resolution, flash and focus are shown as errors but are more likely just red warnings. They used to be yellow but v2 turned them red.

 

I will revert this change. You can ignore them as they are completely innocuous.

 

Hi Lanre,
version 2.0f1 still have an issue with the Preview on my Android 8.0 (Nexus 6P). Basically is that the preview is vertically mirrored (or horizontally mirrored I never know the difference).
Is there a way to easily fix this, should I handle each Orientation case myself?

Thanks