NatDevice - Media Device API

Logcat, ok I see it's part of Android studio... never used it yet...
I don't have Android Studio installed yet (just the sdk tools). Downloading now...

Anyway, I tried the pause() suggestion... but... it looks like something goes wrong there too... After going from Pause(), back to Play(), the NatCam.CapturePhoto(...) function does not seem to work anymore (or the callback does not trigger).

Also, when I switch to another app while the app is running, it crashes too (only when Pause() has not been called earlier.
When pause() has previously been called, it only crashes when camera is running while switching apps).

Edit: just did a build for iOS too... also disables capture after pause.

 

Actually you don't need Android Studio. You have the Android SDK on your system, so invoke adb.exe (or adb on OSX) from the platform-tools directory in your CLI (Command Line Interface) and pass in the logcat argument:

Code (CSharp):

1. adb.exe logcat

Ahh, a minor bug that was addressed in 1.4b0. Open NatCamNativeInterface.cs and go to the Pause function. Change the line that is:

Code (CSharp):

1. isPlaying = false;

to this:

Code (CSharp):

1. FirstFrameReceived = isPlaying = false;

I need to see the logs for this one.

The fix above would take care of this.

 

NatCam 1.4b0 is complete. This came much sooner than I expected. Please email me if you want to test it out. Below is the full changelog.

NatCam 1.4b0:
+ Greatly improved speed of NatCam and Unity on Android.
+ Greatly imrpoved speed of capturing photos especially on Android.
+ Added NatCamPreview component for preview scaling, focusing, zooming and panning.
+ Added ScaleMode.FillView for NatCamPreviewScaler.
+ Added timeStamp integer field to Barcode and Face.
+ Added SaveMode enum.
+ Added SaveMode.SaveToAppDocuments to save photos and videos privately.
+ Added NatCam.OnPhotoSave and NatCam.OnVideoSave events to get the path of saved media.
+ Added ALLOCATE_NEW_OPENCV_MATRIX macro for memory management.
+ Added limited support for SaveMode on Fallback Interface.
+ Added a whitelist and blacklist to keep track of devices that work and don't work.
+ Deprecated NatCamPreviewScaler component.
+ Deprecated NatCamPreviewZoomer component.
+ Deprecated NatCamPreviewGestures component.
+ Deprecated NatCamPreviewBehaviour base class.
+ Deprecated ZoomMode.ZoomSpeedOverrideOnly.
+ Deprecated NatCam.RequestFace. Use NatCam.OnFaceDetect event instead.
+ Deprecated PhotoSaveMode enum.
+ Dropped captured photo correction for app orientation. Now all captured photos are in landscape left orientation.
+ Fixed Android N compatibility.
+ Fixed rare crash immediately app is suspended on Android.
+ Fixed EXC_BAD_ACCESS crash when switching cameras on iOS.
+ Fixed preview incorrectly rotating when app is using fixed orientation on iOS.
+ Fixed compiler errors in NatCamNativeInterface.UpdatePhoto().
+ Fixed build error on Android because of targetSDKVersion.
+ Fixed OnPreviewStart/ExecuteOnPreviewStart not being invoked when camera was played after being paused.
+ Fixed OnPreviewStart/ExecuteOnPreviewStart being invoked too soon after camera was played after being paused.
+ Fixed faces not being detected on Android.
+ Fixed preview flipping momentarily when switching cameras on Android.
+ Fixed bug on Android where photo resolution could only be set before NatCam.Play() is called.
+ Fixed bug on iOS where setting exposure mode to Auto did not work properly.
+ Fixed faces not being detected properly by front camera on Android.
+ Fixed face positions not being in viewport coordinates on Android.
+ Fixed face positions being inverted on the Y axis.
+ Fixed null reference exception when NatCam.Release is called on the fallback interface.
+ Reimplemented NatCam.PreviewMatrix to be on demand like NatCam.PreviewFrame.
+ Reimplemented control pipeline on Android to be cleaner and more efficient.
+ Reduced the amount of text that NatCam logs.
+ Changed Android API minimum requirement to API level 15.
+ Exposed google-play-services.jar to prevent build errors on duplicate copies.

The big feature will be coming in 1.4b1: Video Capture.

 

Thank you Lanre to give me a helping hands!

However, I have another issue about how to find the path of the image, which is captured by Natcam. On the other hands, How can the facebook api work with Natcam? As I would like to upload the image to Facebook album. Thanks for your help!

 

I'm considering buying NatCam for a barcode scanning project. QR or ideally UPC-A if it's robust enough. Is there a preview .apk where I can see this functionality in action? The Minigram example app link is broken.

How does compatibility compare to just using WebCamTexture?

 

Hi Larne,

I just sent you a mail

Thanks, the "isPlaying"-fix worked well... Still have the problem that app crashes when switching or closing.
Sent you the logcat dump via mail...

 

Prior to 1.4, this isn't possible. Email me and I can send you the 1.4b0 beta with the SaveCallback feature.

 

I fixed the link, but here it is. I should note that NatCam doesn't support UPC codes, but you could use ZXing (included in the NatCam package as a dependency) to scan for UPC codes.

NatCam is completely stable on iOS. Android on the other hand is not nearly as uniform. I get a number of bug reports for a variety of devices for a variety of reasons, but nothing fatal. I am in the process of compiling a blacklist (currently with one device so far, the Galaxy Tab 4) and whitelist of devices. Overall though, NatCam is very stable on Android.

 

I haven't seen you email. Can you resend it to me?

 

Getting some strange results on iOS too... On some iPhones the camera turns landscape/portrait although I have default app orientation set to Portrait. Only seems to happen on newer devices (iPhone SE and iPhone6) using iOS version 9.3.4

 

Are you using the NatCamExtensions.mm fix? I mentioned it here.

 

Hey, the camera doesn't seem to load in the editor. How would I go about to get that working?

The plugin is great otherwise 5/5.

 

Is it possible to access both front and rear facing cameras at the same time? Thanks.

 

Hi Larne,

I don't know if you receives my second mail attempt

I think I isolated the problem:

Code (CSharp):

1. E/AndroidRuntime( 2271): FATAL EXCEPTION: UnityMain
2. E/AndroidRuntime( 2271): Process: com.alligence.poc.ViessmannDealer, PID: 2271
3. E/AndroidRuntime( 2271): java.lang.Error: FATAL EXCEPTION [UnityMain]
4. E/AndroidRuntime( 2271): Unity version     : 5.4.0f3
5. E/AndroidRuntime( 2271): Device model      : samsung SAMSUNG-SM-N920A
6. E/AndroidRuntime( 2271): Device fingerprint: samsung/noblelteuc/noblelteatt:5.1.1/LMY47X/N920AUCU2AOJ1:user/release-keys
7. E/AndroidRuntime( 2271):
8. E/AndroidRuntime( 2271): Caused by: java.lang.IllegalStateException: Unable to update texture contents (see logcat for details)
9. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture.nativeUpdateTexImage(Native Method)
10. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture.updateTexImage(SurfaceTexture.java:240)
11. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.UpdateSurfaceTexture(NatCam.java:555)
12. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCamRendering.UpdatePipeline(NatCamRendering.java:31)
13. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.UpdatePipeline(NatCam.java:720)
14. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.RequestRenderCallback(Native Method)
15. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam$3.onFrameAvailable(NatCam.java:499)
16. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture$1.handleMessage(SurfaceTexture.java:203)
17. E/AndroidRuntime( 2271):        at android.os.Handler.dispatchMessage(Handler.java:102)
18. E/AndroidRuntime( 2271):        at android.os.Looper.loop(Looper.java:145)
19. E/AndroidRuntime( 2271):        at com.unity3d.player.UnityPlayer$b.run(Unknown Source)

I'm having the same problem on all android devices (galaxy note5 (android 5.1), acer liquid s1 (android 4.2.2), huawei y530 (4.3)

I hope this helps!

Kind regards,

Wim

 

It should log an error like so:

Code (CSharp):

1. "NatCam Error: Unitygram: Active camera is null. Consider changing the facing of the camera. Terminating"

If so, change the facing of the camera. If not, email me and we will investigate further.

 

Not at all. Both iOS and Android prohibit having access to more than one camera at a time.

 

Are you sure you have the correct email address? My email address is olokobayusuf@gmail.com.

This issue was fixed in the past, but I lost track of which final version it debuted in. I thought it was in 1.3f2, but apparently, it was done in 1.3f3 (canceled version). Now, we have 1.3f4 which we might be releasing to the Asset Store--we probably wouldn't if we finish 1.4f1 in time. I can email you 1.3f4 with the fix.

 

Here's a short State of NatCam address:

We are gearing up for testing NatCam 1.4b0 and fixing all the bugs that developers find. We will be starting 1.4b1 in a couple of days, adding video capture functionality. Depending on the amount of bugs uncovered, there is a chance that we will make 1.4b2, a bug fix beta. Once we decide that the version is stable enough, we will be releasing 1.4f1 onto the Asset Store.

Since all of the core features of a camera API like NatCam would have been completed by then, we will be moving onto our next goal for the NatCam API: supporting all platforms natively. We intend to start the NatCam 2.x release cycle with this in mind. We might also start and publish a 1.5f version depending on any more-than-insignificant changes or fixes that need to be made to the 1.4f major version. Here is a map (very much subject to change) of how we will be proceeding with the 2.x release cycle:

- NatCam 2.0: Native Windows, Linux and macOS support.
- NatCam 2.1: Native Windows Phone support.
- NatCam 2.2: Native WebGL support (I'm personally excited about this one).

Adding native support for these platforms means that developer would have full access to camera controls like flash, focus, exposure, zoom, and so on; it means that NatCam wouldn't have to fall back to WebCamTexture on these platforms; and it also means much more speed.

With that in mind, we can't wait to get 1.4 out of the way!

 

Hi Larne,

Great news!
)

Just tried to send you a mail again (took out the log file).
Hope it works this time... strange that you didn't get previous mails.
I used following e-mail addresses: wim.wouters@alligence.com & hello@wimwouters.com

Cheers,

wim

 

Ahh I actually received your email, but it was in my spam folder. That's why I didn't see it. I've replied to your email.

 

ALL FINE NOW!
YOU ROCK!

review submitted

 

Great! Thank you!

 

Hi Lanre - I want to use Natcam to capture and render 1080 webcam in Unity on Windows - is the current version ready to do that? TIA

 

Hi. Currently, NatCam falls back to WebCamTexture on Windows because we haven't added native support for Windows yet.

 

Hi Lanre,

A while back you said you were going to make an example and Snippet video for integrating Natcam with OpenCVForUnity and, ideally, the FaceTracker sample. Any progress on this? I've purchased Natcam but I can't for the life of me get it to integrate happily. I'm basically replacing the Mat from the with the NatCam.PreviewMatrix, but I'm getting a ton of errors.

Any help would be greatly appreciated!

 

This is great news. I have been using NatCam on a project for NASA, specifically Barcode Scanning of assets, and they have huge interest in furthering development of our project with the Hololens. I have two of them and they are by far the best Business Cards anyone could have right now. I now have numerous projects going on in various industries using them and they will certainly need camera and barcode/QR scanning support especially in a new medical project I am gearing up for.

This is huge.....

I would be more then happy to beta test Windows 10 support, specifically Hololens, if possible. Will need to get this capability going rapidly especially now that Microsoft has gone all in on the Hololens as a commercial product and not just a developer platform.

Can barcode scanning work in any way on Windows 10 at this time? Have not merged my NASA project in and tested yet.

 

Hi @shawww , we are currently busy with 1.4 so we aren't working on examples right now. I do have a NatCamToMatHelper script (modeled after WebCamTextureToMatHelper from DlibFaceTrackerSample) and a corresponding WebCamTextureARSample script to use the former. Email me if you want me to send it to you. What ton of errors are you seeing?

 

Hi @Vern_Shurtz , that sounds amazing. Once 1.4 is done, we will start working on 2.0. Currently, NatCam does scan barcodes on the fallback interface (using WebCamTexture). We added the ZXing.NET dependency in 1.3. Use the same code, no difference.

 

Great, e-mailed you. Thanks!

 

Seen and replied.

 

Hello, I would like to use NatCam for barcode reading, but on my table Samsung Galaxy Tab A, it seems that auto focus is not working, on my phone LG G3 it working ok.
I this known issue with this brand of tablet?

Thaks

 

I seems that auto focus works on front camera, but don't on rear one.

 

Hi, I'm porting my project over to Windows 8.1 and 10 and I get the following error with NatCam

Assets\NatCam\Plugins\Managed\NatCamDispatch.cs(22,21): error CS0246: The type or namespace name 'Thread' could not be found (are you missing a using directive or an assembly reference?)

I've tried to use

Code (CSharp):

1. #if NETFX_CORE
2. using Windows.System.Threading;
3. #else
4. using System.Threading;
5. #endif

and

Code (CSharp):

1.  
2. #if NETFX_CORE
3. private Windows.System.Threading.Thread targetThread, mainThread, workerThread;
4. #else
5. privateThread targetThread, mainThread, workerThread;
6.  
7. #endif

but get that the namespace thread is still not found... any ideas how to approach this?

 

This sounds like a device-specific issue. There isn't anything I can do to make it work on the Tab A.

 

From here, the Windows.System.Threading namespace doesn't have a Thread class. Remove the NETFX_CORE macro so that it uses the default System.Threading namespace. Why are you using the NETFX_CORE macro/Windows.System.Threading namespace?

 

Threads aren't available in WSA through System.Threading... hence the original error and why I'm referencing Windows.system.threading... I saw someone reference needing to use Windows.system.threading... I'm not sure if threads are doable on windows? Also, I'm pretty green when it comes to coding... very... very... green... lol

edit... just to clarify
the first error shows up with Natcam code untouched...

 

It seems that you would have to use the Windows.System.Threading.ThreadPool class, specifically the ThreadPool.RunAsync method. I would also advise you to peruse the MSDN documentation on the Windows.System.Threading.ThreadPool class and the NatCamU.NatCamDispatch class source.

 

[everything you said to do]
../\
./.|\
.. |
.. |
.. |
...O[my head}

Sadly I don't think I'm up to the task of converting this. I may have to eliminate the windows 8.1 and 10 as build platforms... :/

 

I don't think an issue like this should cause you to eliminate entire platforms. Read up on multithreading, look at MSDN, and the NatCamDispatch source. It might take you a short while, but it would not take much.

 

Hi Lanre,

I'm having the same issue as l-keustermans. I've sent you an email about it. If you could also send me the fix it'd be greatly appreciated. Your plugin is exactly what we need, but we can't use it because of this issue.

Thanks!

 

So...from what I can tell natcam only uses threading on the supported devices and instead defaults back to the Platform's native approach so by sprinkling a bunch of #if NETFX_CORE 's around several scripts it looks like it should work. Unfortunately I'm working through some other build errors not related to Natcam before I can see if it works...

WSA is a PIA

 

Great! Will be trying it out shortly.

~V~

 

Sent.

 

Not quite. NatCam uses threads on all platforms (except WSA of course). Threads are a mono implementation of the platform's low-level threading API's. I am not familiar with WSA because I haven't worked with it before.

 

Let me know how it goes.

 

Is NatCam ready to provide footage from the iPhone 7+ Dual Camera to OpenCV?

 

I'm experiencing a similar issue with Dlib face tracking

 

Actually, we don't know whether iOS would allow running the two cameras simultaneously. Besides, developers do not yet have access to the device for development and testing, so I would say that the current answer is no.

 

By similar, do you mean performance? If so, please profile the DlibFaceTracker example. You will see that the slowdown is caused by the OpenCV processing, not NatCam. You will have to optimize it by yourself to make your app run smoothly. As a starting point, I will advise you to look into threading.

 

Looks like NATCAM.dll is simply not compatible with Windows Store APIs, no workaround other then the DLL being rewritten from what I can see and am being told. At build time I get about 39 errors referencing the NATCAM.dll mostly in regards to threading. I know this is something you never advertised Natcam for but if in the future if you added UAS support that would be great. As is anothuer plugin (RAIN AI) has the same problem and looks to be broke even though it worked fine on 5.1.4 with Windows phone 8.0/8.1. Support for RAIN is dead so it looks like there is no point in pursuing UAS for this project. Shame cause the only reason I updated my project from 5.1.4 was for Natcam. :/
Thx for your support.
Chris

Edit. So after further testing it looks like RAIN never would have worked on the Windows store even though I could build to the phone, so my project has been broken from the get go for UAS.