NatDevice - Media Device API

Yeah, you are correct. I finally found a solution: Go to player settings, turn off Auto Graphics API, leave only OpenGLES2, and then the Vuforia plugin works. I don't need to do this if I only have Vuforia in the project, but somehow this solves the problem.

 

I'm using latest version. What kind of bug was fixed in 1.3f4? That memory is clean while minigram gameObject inactive, or Release method crash?

 

Please submit this bug report to Vuforia so that they can address it.

 

The error when Release() is called was fixed in 1.3f4. As for Minigram, you must manually call Release on NatCam. The camera will not automatically stop running when you delete the Minigram game object.

 

I am building an Android application. I have an external webcam plugged into my Android (using USB on the go) and the webcam does not show up in Unity using the built in WebCamTexture. I am hoping that NatCam will allow me to access this external camera. Do you know if this will work?

 

Understood, than crashesh only applied on Unity editor. Thanks for help!

 

No. NatCam only supports cameras that are compatible with the android.hardware.camera API. You might want to look into UVCCamera for UVC camera access.

 

Can you post your code or email it to me? I have just verified that there is no memory leak when photos are captured in 1.3f4. What do you do with the captured photo you receive?

 

Does it support Vuforia?

 

This is a 'feature' of the .NET CLR. When a large array is allocated (calling Texture2D.GetPixels) and subsequently freed, the CLR does not surrender the memory back to the OS because it decides to pool the memory just in case you make an allocation of that size again. This causes memory usage to grow and then normalize after a large amount of memory has been consumed (100s of MB). It is either this happens, or the application runs out of memory (which is your case).

This issue has nothing to do with NatCam. Create a huge Texture2D and call GetPixels on it; the result is the same. This is why image processing in the managed layer is not advisable. You are much better off modifying the image in a C-based layer where you can deterministically free allocated memory to the OS (free(void*) or delete [] void*).

Also, if you don't comment out ALLOCATE_NEW_PHOTO_TEXTURE, then you must manually call MonoBehaviour.Destroy(...) on the texture.

 

No. Vuforia never got back to me about support.

 

I'm back!

I have an updated NatCam 1.3f4 build:
+ Added FocusMode.SoftFocus for soft autofocus especially in VR applications on Android.
+ Fixed bug where face callback was invoked multiple times with wrong coordinates.

This build will NOT be updated to the Asset Store (because it's merely a patch), so email me with your invoice number and I will send it to you. Thank you!

 

We are seeing an issue on Android 7 devices. Here is the log cat we are getting. Any thoughts.


W/Unity: Failed to get permission info for android.permission.WRITE_INTERNAL_STORAGE, manifest likely missing custom permission declaration
W/Unity: Permission android.permission.WRITE_INTERNAL_STORAGE ignored
D/NvOsDebugPrintf: NvRmPrivGetChipPlatform: Could not read platform information
D/NvOsDebugPrintf: Expected on kernels without fuse support, using silicon
D/Unity: GL_EXT_debug_marker GL_EXT_base_instance GL_EXT_blend_func_extended GL_EXT_blend_minmax GL_EXT_buffer_storage GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_copy_image GL_EXT_debug_label GL_EXT_discard_framebuffer GL_EXT_disjoint_timer_query GL_EXT_draw_buffers_indexed GL_EXT_draw_elements_base_vertex GL_EXT_float_blend GL_EXT_frag_depth GL_EXT_geometry_point_size GL_EXT_geometry_shader GL_EXT_gpu_shader5 GL_EXT_map_buffer_range GL_EXT_multi_draw_indirect GL_EXT_multisample_compatibility GL_EXT_occlusion_query_boolean GL_EXT_post_depth_coverage GL_EXT_primitive_bounding_box GL_EXT_raster_multisample GL_EXT_render_snorm GL_EXT_robustness GL_EXT_separate_shader_objects GL_EXT_shader_implicit_conversions GL_EXT_shader_integer_mix GL_EXT_shader_io_blocks GL_EXT_shader_non_constant_global_initializers GL_EXT_shader_texture_lod GL_EXT_shadow_samplers GL_EXT_sparse_texture GL_EXT_sparse_texture2 GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_tessellation_point_size GL_EXT_tessellation_shader GL_EXT_
D/Unity: texture_border_clamp GL_EXT_texture_buffer GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map_array GL_EXT_texture_filter_anisotropic GL_EXT_texture_filter_minmax GL_EXT_texture_format_BGRA8888 GL_EXT_texture_norm16 GL_EXT_texture_rg GL_EXT_texture_sRGB_decode GL_EXT_texture_storage GL_EXT_texture_view GL_EXT_unpack_subimage GL_KHR_context_flush_control GL_KHR_debug GL_KHR_no_error GL_KHR_robust_buffer_access_behavior GL_KHR_robustness GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_sliced_3d GL_NV_bgr GL_NV_bindless_texture GL_NV_blend_equation_advanced GL_NV_blend_equation_advanced_coherent GL_NV_conditional_render GL_NV_conservative_raster GL_NV_copy_buffer GL_NV_copy_image GL_NV_draw_buffers GL_NV_draw_instanced GL_NV_draw_texture GL_NV_draw_vulkan_image GL_NV_EGL_stream_consumer_external GL_NV_explicit_attrib_location GL_NV_fbo_color_attachments GL_NV_fill_rectangle GL_NV_fragment_coverage_to_color GL_NV_fragment_shader_interlock GL_NV_framebuffer
D/Unity: _blit GL_NV_framebuffer_mixed_samples GL_NV_framebuffer_multisample GL_NV_generate_mipmap_sRGB GL_NV_geometry_shader_passthrough GL_NV_instanced_arrays GL_NV_internalformat_sample_query GL_NV_gpu_shader5 GL_NV_image_formats GL_NV_occlusion_query_samples GL_NV_non_square_matrices GL_NV_pack_subimage GL_NV_packed_float GL_NV_packed_float_linear GL_NV_path_rendering GL_NV_path_rendering_shared_edge GL_NV_pixel_buffer_object GL_NV_polygon_mode GL_NV_read_buffer GL_NV_read_depth GL_NV_read_depth_stencil GL_NV_read_stencil GL_NV_sample_locations GL_NV_sample_mask_override_coverage GL_NV_shader_atomic_fp16_vector GL_NV_shader_noperspective_interpolation GL_NV_shadow_samplers_array GL_NV_shadow_samplers_cube GL_NV_sRGB_formats GL_NV_texture_array GL_NV_texture_border_clamp GL_NV_texture_compression_latc GL_NV_texture_compression_s3tc GL_NV_texture_compression_s3tc_update GL_NV_timer_query GL_NV_viewport_array GL_NV_viewport_array2 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_compress
D/Unity: ed_ETC1_RGB8_texture GL_OES_depth24 GL_OES_depth32 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_copy_image GL_OES_draw_buffers_indexed GL_OES_draw_elements_base_vertex GL_OES_texture_border_clamp GL_OES_tessellation_point_size GL_OES_tessellation_shader GL_OES_texture_buffer GL_OES_geometry_point_size GL_OES_geometry_shader GL_OES_gpu_shader5 GL_OES_shader_io_blocks GL_OES_texture_view GL_OES_primitive_bounding_box GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_get_program_binary GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_OES_sample_shading GL_OES_sample_variables GL_OES_shader_image_atomic GL_OES_shader_multisample_interpolation GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_texture_cube_map_array GL_OES_texture_npot GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_stencil8 GL_OES_tex
D/Unity: ture_storage_multisample_2d_array GL_OES_vertex_array_object GL_OES_vertex_half_float GL_ANDROID_extension_pack_es31a
I/libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x3, using default mask 0x3 based on channel count of 2)
D/AudioTrack: Client defaulted notificationFrames to 512 for frameCount 2048
D/NetworkSecurityConfig: No Network Security Config specified, using platform default
W/libc: pthread_create sched_setscheduler call failed: Operation not permitted
W/libc: pthread_create sched_setscheduler call failed: Operation not permitted
V/MediaRouter: Adding route: RouteInfo{ name=Tablet, description=null, status=null, category=RouteCategory{ name=null types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
V/MediaRouter: Updating audio routes: AudioRoutesInfo{ type=SPEAKER }
V/MediaRouter: Selecting route: RouteInfo{ name=Tablet, description=null, status=null, category=RouteCategory{ name=null types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
W/libc: pthread_create sched_setscheduler call failed: Operation not permitted
I/Unity: Access Token: 2JgIcfx1mCU0c+6v7fdj7s6jWfrl2IEttA0+eE9EgdUSmkzsksVJhrXQAPBaW+H9FWGPV0fogaYGp17p3QLInlqs3hRkSpIxjHBCsxwlEqE=
I/Unity:
I/Unity: (Filename: ./artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)
E/Unity: Unable to find NatCam
I/Unity: NatCam: Initialized Synchronous Dispatcher
I/Unity:
I/Unity: (Filename: ./artifacts/generated/common/runtime/UnityEngineDebugBindings.gen.cpp Line: 37)
I/Unity: DllNotFoundException: NatCam
I/Unity: at (wrapper managed-to-native) NatCamU.Internals.NatCamNativeInterface:RegisterCallbacks (NatCamU.Internals.NatCamNativeInterface/RenderCallback,NatCamU.Internals.NatCamNativeInterface/UpdateCallback,NatCamU.Internals.NatCamNativeInterface/UpdatePhotoCallback,NatCamU.Internals.NatCamNativeInterface/UpdateCodeCallback,NatCamU.Internals.NatCamNativeInterface/UpdateFaceCallback)
I/Unity: at NatCamU.Internals.NatCamNativeInterface.Initialize (PreviewType previewType, Boolean metadataDetection) [0x00000] in <filename unknown>:0
I/Unity: at NatCamU.NatCam.Initialize (NatCamInterface NatCamInterface, PreviewType PreviewType, Switch MetadataDetection) [0x00000] in <filename unknown>:0
I/Unity: at NatCamU.UnitygramBase.Start () [0x00000] in <filename unknown>:0
I/Unity: at com.goldenratstudios.panels.ScanPanel.Start () [0x00000] in <filename unknown>:0
I/Unity:
I/Unity: (Filename: Line: -1)

 

It seems that you have an incomplete assembly. Try redownloading NatCam from the Asset Store. Also, NatCam 1.3 is not compatible with Android N. We are working to release another beta that has this issue fixed. Please use a spoiler instead of copying the code inside.

 

Hey guys. I will be writing memos as I develop NatCam. If you are interested in seeing where I am with NatCam and having insight, check out the post on Medium.

 

Where can I download a version before 1.3? As this is android N which it is happening on.

Don't worry found an old version

Thanks

 

Email me your invoice number and I will email you 1.4b2. Previous versions of NatCam will not work with Android N.

 

Email me your invoice number and I will email you 1.4b2. We fixed the issue in 1.4b2.

 

Hello,

Is there any way to stop camera from rotating, to have it fixed in landscape mode

 

Yep this is one of the issues reported by my QA team, and would be nice to solve.

 

It is currently completely stable apart from video recording.

 

Are you using 1.3f4? If not, delete and reimport (from Asset Store) NatCam to your project.

 

Does your app use a locked rotation or autorotation? NatCam will not work with autorotation because it won't have a 'reference' orientation to use.

 

I am using locked orientation in Landscape mode, but wold like to have camera in portrait mode, because I have to tilt preview for 90º.

 

Can you post a screenshot of what you mean? Does the preview display incorrectly?

 

Little update:

- NatCam 1.4b3 is going well. Working on finalizing video recording on iOS then moving on to (rebuilding it on) Android.
- NatCam 1.5 is in development. I can't wait till it enters beta; it's a very very exciting release.

Also, we considered creating a website forum specifically for NatCam but due to the relatively small size of the ecosystem, and my ability to quickly communicate individually, we are shelving the effort for now. This could change with the release of either 1.5f1 or 2.0f1.

 

Thanks, had it on auto rotate even though I only have one rotation selected.

 

Having some problems with 1.3f4 on Android when the app is sent to the background, and then resumed.

The issue is that the video feed is stuck on the last frame it captured and will not resume.

Do you know what might cause this?

My Setup Script:

Code (CSharp):

1.  
2. void SetupNatCam()
3.     {
4.         NatCam.Initialize(NatCamInterface.NativeInterface);
5. #if UNITY_EDITOR
6.         NatCam.ActiveCamera = DeviceCamera.Cameras[0];
7. #else
8.         NatCam.ActiveCamera = DeviceCamera.RearCamera;
9. #endif
10.         NatCam.Verbose = Switch.On;
11.         NatCam.ActiveCamera.FocusMode = FocusMode.HybridFocus;
12.         NatCam.ActiveCamera.SetResolution(ResolutionPreset.HD);
13.         NatCam.ActiveCamera.SetFramerate(FrameratePreset.Smooth);
14.         NatCam.Play();
15.         NatCam.ExecuteOnPreviewStart(OnPreviewStart);
16.         NatCam.OnPreviewUpdate += OnPreviewUpdate;
17.     }
18.  

 

Trying to get admob integrated into my project. I had to get rid of the Google Play Services folder when I added NatCam or any build would just end with the message "couldn't convert java into dex format" or something like that. I'm finally getting around to getting ads integrated and want to tackle this. Could you help me understand how to make Natcam and Google Play services play well together or at least explain why they don't work together in the first place? Thx!

 

We also have a bug with the latest beta 1.4b2 where the app will crash right away on showing the camera on an iPhone 5, OS 7.0.4

Thanks

Phil

 

This issue was fixed in NatCam 1.4b2. Email me your invoice number and I will send you the beta.

 

You are probably getting this error because of duplicate copies of google-play-services.jar. Remove one from your project (like the one that comes with NatCam) and the issue should go away.

 

It sounds like you are running the camera preview with ResolutionPreset.HighestResolution which will default to the highest possible resolution the camera can reach (including when performing photo capture) which is 8MP on an iPhone 5. If this is why, then do not use ResolutionPreset.HighestResolution. If it isn't, then email me the logs with NatCam's verbose mode on.

 

Hi guys. I thought I'd post a little update:

NatCam 1.4 will be skipped. Instead, we will be pushing out NatCam 1.5. The 1.5 betas are coming out soon; we are working around the clock to complete the release. I will post more about 1.5 in a post on Medium.

 

HI, i am using unity 5.1.3p2 for a project . I know it's written in asset store that it supports unity 5.3 and above.
is it still possible to use this plugin on unity 5.1.3 ?

 

I seem to have some issues with saving photo's to the gallery on iOS (haven't tested android yet)

1. NatCam.PhotoSaveMode doesn't seem to do anything since NatCam.SaveToPhotos just overrides it.
   
2. SaveMode is not in sync with the modes in the native code
   
   Code (CSharp):

   1. [NCDoc(14)] [Flags] public enum SaveMode : byte { //Update native mappings
   2.         [NCDoc(45)] DoNotSave = 0,
   3.         [NCDoc(184)] SaveToAppDocuments = 1,
   4.         [NCDoc(46)] SaveToPhotoGallery = 2,
   5.         [NCDoc(47)] SaveToPhotoAlbum = 4
   6.     }
   Code (CSharp):

   1. void SaveToGallery (const char * imageBytes, int mode) {
   2.     NSString* image64 = [NSString stringWithUTF8String:imageBytes];
   3.     NSData* imageData = [[NSData alloc] initWithBase64EncodedString:image64 options:NSDataBase64DecodingIgnoreUnknownCharacters];
   4.     UIImage* image = [UIImage imageWithData:imageData];
   5.     if (mode == 1) { //Save to Photos
   6.         UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil);
   7.         Log(@"NatCam: Saved photo to photo album");
   8.     }
   9.     if (mode == 2) { //Save to App Album
   10.         SaveImageToAppAlbum(image, NatCamInstance()->library);
   11.     }
   12. }
3. When saving to the gallery, the texture seems to be rotated
4. An album seems to be created in my photos, but when trying to save them to the app album, it always returns the error: NatCam Error: Cannot save to app album because library is null

I also have another question:
Due to the design my app is locked at portrait and portrait upside down rotation. If a user takes a screenshot holding their phone in a landscape position, then the resulting photo has a wrong rotation. Anything I can do about this?

 

NatCam currently requires at least Unity 5.2 because of the new native rendering API that was introduced in 5.2.

 

How do you mean SaveToPhotos overrides it?

Ahh, good catch. The C# mappings were changed to support composite saves (using bitwise OR to toggle bits). We decided to deprecate the SavePhoto API in 1.4 and so we didn't bother checking 1.3f4 when releasing it.

I have fixed this issue before. Email me your NatCamExtensions.mm file and I will add the fix. We have revamped the SavePhoto API in 1.5 so this has already been taken care of.

Can you send me the logs? This is perplexing.

What rotation is it? Can you share screenshots? Also, you mention that your app is locked to portrait and portrait upside down; are you using autorotation with only these two in Player Settings? If so, this has been problematic for NatCam and we will be addressing it in 1.5. For now, consider locking to only portrait.

 

Hey guys!

Here is all you need to know about NatCam 1.5.

 

[help]I use the latest 1.3 natcam version. and I run correctly on unity editor. but has some exception on actual android phone.
how can I solve the AndroidJavaException? the follow is info on phone by adb console.


11-24 17:14:27.618 23255 23276 D Unity : NatCam: Generated Camera Texture
11-24 17:14:27.880 23255 23276 D Unity : NatCam: Inspected Device: Found 2 cameras
11-24 17:14:28.027 23255 23276 I Unity : AndroidJavaException: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/google/android/gms/R$string;
11-24 17:14:28.027 23255 23276 I Unity : java.lang.NoClassDefFoundError: Failed resolution of: Lcom/google/android/gms/R$string;
11-24 17:14:28.027 23255 23276 I Unity : at com.google.android.gms.common.internal.zzg.zzi(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.google.android.gms.common.GooglePlayServicesUtil.zza(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.google.android.gms.common.GooglePlayServicesUtil.zza(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.google.android.gms.common.GooglePlayServicesUtil.showErrorNotification(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.google.android.gms.common.GoogleApiAvailability.showErrorNotification(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.yusufolokoba.natcam.NatCam.MetadataAvailability(NatCam.java:658)
11-24 17:14:28.027 23255 23276 I Unity : at com.yusufolokoba.natcam.NatCam.InspectDevice(NatCam.java:86)
11-24 17:14:28.027 23255 23276 I Unity : at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
11-24 17:14:28.027 23255 23276 I Unity : at com.unity3d.player.UnityPlayer.a(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at com.unity3d.player.UnityPlayer$b$1.handleMessage(Unknown Source)
11-24 17:14:28.027 23255 23276 I Unity : at android.os.Handler.dispatchMessage(Handler.java:107)
11-24 17:14:28.027 23255 23276 I Unity : at android.os.Looper.loop(Looper.j

 

Hi guys, im just started using NatCam and it's awesome. My issue is a classic one: when the app goes suspended and resumed, the cam is frozen and still no longer works. This happens on three different android devices. Please gimme a hint or post a code to restart the cam so i could use it in OnApplicationPause event.
Thank you so much!

 

Do you have google-play-services.jar? And is it updated on your device?

 

Email me your invoice number and I will email you NatCam 1.4b2 that fixes the issue.

 

Sent now, thanks for being so fast.

 

Replied.

 

I'd like to update on these issues in case any developer using 1.3f4 encounters them.

PhotoSaveMode is meant only for pictures that are taken with NatCam.CapturePhoto (so that NatCam can automatically save them for you). When using NatCam.SaveToPhotos, you can specify your own save mode.

Fixed in 1.5

Also fixed in 1.5. The fix is really easy though. In the SaveImageToAppAlbum in NatCamExtensions.mm, change this line:

Code (CSharp):

1. [library writeImageToSavedPhotosAlbum:img orientation:ALAssetOrientationRight completionBlock:ImageSaveCallback];

to this:

Code (CSharp):

1. [library writeImageToSavedPhotosA
2. [QUOTE="ThomasVandenberghe, post: 2861526, member: 566784"]An album seems to be created in my photos, but when trying to save them to the app album, it always returns the error: NatCam Error: Cannot save to app album because library is null[/QUOTE]
3. This is because you call NatCam.Release() before saving your photo. You must not call any NatCam functions after Release() is called.

 

thanks for your so fast reply. I found google-play-services.jar in NatCam plugins. I can run on unity editor normally too.
but we have no google service in china. how can I solve it?

 

NatCam needs google play services for barcode and face detection. You will have to turn off metadata detection and find a workaround.

 

Hey, been using Natcam with an app we've been working on over here and it's real nice. Having some trouble with it though, recently I've tried to do a merge with Firebase Cloud Messaging and it seems to be some sort of google play services conflict. I've read the readme and looked for the jar file in my project, however the only instance of google-play-services.jar is in the StagingArea folder for Natcam. Any idea why it's (apparently) appearing out of thin air?

POST-FACTO-EDIT: Apparently I had no idea what an .AAR file was. Found out it was basically a zip file, checked the contents, found the file. There's me with egg on my face.

 

The documentation has been updated to NatCam 1.5.