Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

NatCam - WebCam API

Discussion in 'Assets and Asset Store' started by Lanre, Dec 17, 2015.

?

Should we add exposure controls in v1.3? This means dropping support for iOS 7

Poll closed Jun 10, 2016.
  1. Yes

    9 vote(s)
    75.0%
  2. No

    3 vote(s)
    25.0%
  1. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    Logcat, ok I see it's part of Android studio... never used it yet...
    I don't have Android Studio installed yet (just the sdk tools). Downloading now...

    Anyway, I tried the pause() suggestion... but... it looks like something goes wrong there too... After going from Pause(), back to Play(), the NatCam.CapturePhoto(...) function does not seem to work anymore (or the callback does not trigger).

    Also, when I switch to another app while the app is running, it crashes too (only when Pause() has not been called earlier.
    When pause() has previously been called, it only crashes when camera is running while switching apps).

    Edit: just did a build for iOS too... also disables capture after pause.
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Actually you don't need Android Studio. You have the Android SDK on your system, so invoke adb.exe (or adb on OSX) from the platform-tools directory in your CLI (Command Line Interface) and pass in the logcat argument:
    Code (CSharp):
    1. adb.exe logcat
    Ahh, a minor bug that was addressed in 1.4b0. Open NatCamNativeInterface.cs and go to the Pause function. Change the line that is:
    Code (CSharp):
    1. isPlaying = false;
    to this:
    Code (CSharp):
    1. FirstFrameReceived = isPlaying = false;
    I need to see the logs for this one.
    The fix above would take care of this.
     
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    NatCam 1.4b0 is complete. This came much sooner than I expected. Please email me if you want to test it out. Below is the full changelog.

    NatCam 1.4b0:
    + Greatly improved speed of NatCam and Unity on Android.
    + Greatly imrpoved speed of capturing photos especially on Android.
    + Added NatCamPreview component for preview scaling, focusing, zooming and panning.
    + Added ScaleMode.FillView for NatCamPreviewScaler.
    + Added timeStamp integer field to Barcode and Face.
    + Added SaveMode enum.
    + Added SaveMode.SaveToAppDocuments to save photos and videos privately.
    + Added NatCam.OnPhotoSave and NatCam.OnVideoSave events to get the path of saved media.
    + Added ALLOCATE_NEW_OPENCV_MATRIX macro for memory management.
    + Added limited support for SaveMode on Fallback Interface.
    + Added a whitelist and blacklist to keep track of devices that work and don't work.
    + Deprecated NatCamPreviewScaler component.
    + Deprecated NatCamPreviewZoomer component.
    + Deprecated NatCamPreviewGestures component.
    + Deprecated NatCamPreviewBehaviour base class.
    + Deprecated ZoomMode.ZoomSpeedOverrideOnly.
    + Deprecated NatCam.RequestFace. Use NatCam.OnFaceDetect event instead.
    + Deprecated PhotoSaveMode enum.
    + Dropped captured photo correction for app orientation. Now all captured photos are in landscape left orientation.
    + Fixed Android N compatibility.
    + Fixed rare crash immediately app is suspended on Android.
    + Fixed EXC_BAD_ACCESS crash when switching cameras on iOS.
    + Fixed preview incorrectly rotating when app is using fixed orientation on iOS.
    + Fixed compiler errors in NatCamNativeInterface.UpdatePhoto().
    + Fixed build error on Android because of targetSDKVersion.
    + Fixed OnPreviewStart/ExecuteOnPreviewStart not being invoked when camera was played after being paused.
    + Fixed OnPreviewStart/ExecuteOnPreviewStart being invoked too soon after camera was played after being paused.
    + Fixed faces not being detected on Android.
    + Fixed preview flipping momentarily when switching cameras on Android.
    + Fixed bug on Android where photo resolution could only be set before NatCam.Play() is called.
    + Fixed bug on iOS where setting exposure mode to Auto did not work properly.
    + Fixed faces not being detected properly by front camera on Android.
    + Fixed face positions not being in viewport coordinates on Android.
    + Fixed face positions being inverted on the Y axis.
    + Fixed null reference exception when NatCam.Release is called on the fallback interface.
    + Reimplemented NatCam.PreviewMatrix to be on demand like NatCam.PreviewFrame.
    + Reimplemented control pipeline on Android to be cleaner and more efficient.
    + Reduced the amount of text that NatCam logs.
    + Changed Android API minimum requirement to API level 15.
    + Exposed google-play-services.jar to prevent build errors on duplicate copies.

    The big feature will be coming in 1.4b1: Video Capture.
     
  4. Wongyan

    Wongyan

    Joined:
    Aug 25, 2016
    Posts:
    2
    Thank you Lanre to give me a helping hands!

    However, I have another issue about how to find the path of the image, which is captured by Natcam. On the other hands, How can the facebook api work with Natcam? As I would like to upload the image to Facebook album. Thanks for your help!
     
    Last edited: Aug 28, 2016
  5. Zazery

    Zazery

    Joined:
    Jul 9, 2013
    Posts:
    3
    I'm considering buying NatCam for a barcode scanning project. QR or ideally UPC-A if it's robust enough. Is there a preview .apk where I can see this functionality in action? The Minigram example app link is broken.

    How does compatibility compare to just using WebCamTexture?
     
  6. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    Hi Larne,

    I just sent you a mail
    Thanks, the "isPlaying"-fix worked well... Still have the problem that app crashes when switching or closing.
    Sent you the logcat dump via mail...
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Prior to 1.4, this isn't possible. Email me and I can send you the 1.4b0 beta with the SaveCallback feature.
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    I fixed the link, but here it is. I should note that NatCam doesn't support UPC codes, but you could use ZXing (included in the NatCam package as a dependency) to scan for UPC codes.

    NatCam is completely stable on iOS. Android on the other hand is not nearly as uniform. I get a number of bug reports for a variety of devices for a variety of reasons, but nothing fatal. I am in the process of compiling a blacklist (currently with one device so far, the Galaxy Tab 4) and whitelist of devices. Overall though, NatCam is very stable on Android.
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    I haven't seen you email. Can you resend it to me?
     
  10. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    Getting some strange results on iOS too... On some iPhones the camera turns landscape/portrait although I have default app orientation set to Portrait. Only seems to happen on newer devices (iPhone SE and iPhone6) using iOS version 9.3.4
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Are you using the NatCamExtensions.mm fix? I mentioned it here.
     
    l-keustermans likes this.
  12. Skeledurr

    Skeledurr

    Joined:
    Jan 5, 2015
    Posts:
    23
    Hey, the camera doesn't seem to load in the editor. How would I go about to get that working?

    The plugin is great otherwise 5/5.
     
  13. contempt

    contempt

    Joined:
    Jul 31, 2012
    Posts:
    84
    Is it possible to access both front and rear facing cameras at the same time? Thanks.
     
  14. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    Hi Larne,

    I don't know if you receives my second mail attempt :)

    I think I isolated the problem:

    Code (CSharp):
    1. E/AndroidRuntime( 2271): FATAL EXCEPTION: UnityMain
    2. E/AndroidRuntime( 2271): Process: com.alligence.poc.ViessmannDealer, PID: 2271
    3. E/AndroidRuntime( 2271): java.lang.Error: FATAL EXCEPTION [UnityMain]
    4. E/AndroidRuntime( 2271): Unity version     : 5.4.0f3
    5. E/AndroidRuntime( 2271): Device model      : samsung SAMSUNG-SM-N920A
    6. E/AndroidRuntime( 2271): Device fingerprint: samsung/noblelteuc/noblelteatt:5.1.1/LMY47X/N920AUCU2AOJ1:user/release-keys
    7. E/AndroidRuntime( 2271):
    8. E/AndroidRuntime( 2271): Caused by: java.lang.IllegalStateException: Unable to update texture contents (see logcat for details)
    9. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture.nativeUpdateTexImage(Native Method)
    10. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture.updateTexImage(SurfaceTexture.java:240)
    11. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.UpdateSurfaceTexture(NatCam.java:555)
    12. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCamRendering.UpdatePipeline(NatCamRendering.java:31)
    13. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.UpdatePipeline(NatCam.java:720)
    14. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam.RequestRenderCallback(Native Method)
    15. E/AndroidRuntime( 2271):        at com.yusufolokoba.natcam.NatCam$3.onFrameAvailable(NatCam.java:499)
    16. E/AndroidRuntime( 2271):        at android.graphics.SurfaceTexture$1.handleMessage(SurfaceTexture.java:203)
    17. E/AndroidRuntime( 2271):        at android.os.Handler.dispatchMessage(Handler.java:102)
    18. E/AndroidRuntime( 2271):        at android.os.Looper.loop(Looper.java:145)
    19. E/AndroidRuntime( 2271):        at com.unity3d.player.UnityPlayer$b.run(Unknown Source)
    I'm having the same problem on all android devices (galaxy note5 (android 5.1), acer liquid s1 (android 4.2.2), huawei y530 (4.3)

    I hope this helps!

    Kind regards,

    Wim
     
    Last edited: Aug 30, 2016
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    It should log an error like so:
    Code (CSharp):
    1. "NatCam Error: Unitygram: Active camera is null. Consider changing the facing of the camera. Terminating"
    If so, change the facing of the camera. If not, email me and we will investigate further.
     
    l-keustermans likes this.
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Not at all. Both iOS and Android prohibit having access to more than one camera at a time.
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Are you sure you have the correct email address? My email address is olokobayusuf@gmail.com.

    This issue was fixed in the past, but I lost track of which final version it debuted in. I thought it was in 1.3f2, but apparently, it was done in 1.3f3 (canceled version). Now, we have 1.3f4 which we might be releasing to the Asset Store--we probably wouldn't if we finish 1.4f1 in time. I can email you 1.3f4 with the fix.
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Here's a short State of NatCam address:

    We are gearing up for testing NatCam 1.4b0 and fixing all the bugs that developers find. We will be starting 1.4b1 in a couple of days, adding video capture functionality. Depending on the amount of bugs uncovered, there is a chance that we will make 1.4b2, a bug fix beta. Once we decide that the version is stable enough, we will be releasing 1.4f1 onto the Asset Store.

    Since all of the core features of a camera API like NatCam would have been completed by then, we will be moving onto our next goal for the NatCam API: supporting all platforms natively. We intend to start the NatCam 2.x release cycle with this in mind. We might also start and publish a 1.5f version depending on any more-than-insignificant changes or fixes that need to be made to the 1.4f major version. Here is a map (very much subject to change) of how we will be proceeding with the 2.x release cycle:

    - NatCam 2.0: Native Windows, Linux and macOS support.
    - NatCam 2.1: Native Windows Phone support.
    - NatCam 2.2: Native WebGL support (I'm personally excited about this one).

    Adding native support for these platforms means that developer would have full access to camera controls like flash, focus, exposure, zoom, and so on; it means that NatCam wouldn't have to fall back to WebCamTexture on these platforms; and it also means much more speed.

    With that in mind, we can't wait to get 1.4 out of the way!
     
    Last edited: Sep 4, 2016
    l-keustermans likes this.
  19. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    Hi Larne,

    Great news!
    :))

    Just tried to send you a mail again (took out the log file).
    Hope it works this time... strange that you didn't get previous mails.
    I used following e-mail addresses: wim.wouters@alligence.com & hello@wimwouters.com

    Cheers,

    wim
     
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Ahh I actually received your email, but it was in my spam folder. That's why I didn't see it. I've replied to your email.
     
  21. l-keustermans

    l-keustermans

    Joined:
    Sep 23, 2015
    Posts:
    10
    ALL FINE NOW!
    YOU ROCK!

    review submitted :p
     
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Great! Thank you!
     
  23. pandigital

    pandigital

    Joined:
    Mar 20, 2009
    Posts:
    15
    Hi Lanre - I want to use Natcam to capture and render 1080 webcam in Unity on Windows - is the current version ready to do that? TIA
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Hi. Currently, NatCam falls back to WebCamTexture on Windows because we haven't added native support for Windows yet.
     
  25. shawww

    shawww

    Joined:
    Sep 30, 2014
    Posts:
    42
    Hi Lanre,

    A while back you said you were going to make an example and Snippet video for integrating Natcam with OpenCVForUnity and, ideally, the FaceTracker sample. Any progress on this? I've purchased Natcam but I can't for the life of me get it to integrate happily. I'm basically replacing the Mat from the with the NatCam.PreviewMatrix, but I'm getting a ton of errors.

    Any help would be greatly appreciated!
     
  26. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    263
    This is great news. I have been using NatCam on a project for NASA, specifically Barcode Scanning of assets, and they have huge interest in furthering development of our project with the Hololens. I have two of them and they are by far the best Business Cards anyone could have right now. I now have numerous projects going on in various industries using them and they will certainly need camera and barcode/QR scanning support especially in a new medical project I am gearing up for.

    This is huge.....

    I would be more then happy to beta test Windows 10 support, specifically Hololens, if possible. Will need to get this capability going rapidly especially now that Microsoft has gone all in on the Hololens as a commercial product and not just a developer platform.

    Can barcode scanning work in any way on Windows 10 at this time? Have not merged my NASA project in and tested yet.
     
    Lanre likes this.
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Hi @shawww , we are currently busy with 1.4 so we aren't working on examples right now. I do have a NatCamToMatHelper script (modeled after WebCamTextureToMatHelper from DlibFaceTrackerSample) and a corresponding WebCamTextureARSample script to use the former. Email me if you want me to send it to you. What ton of errors are you seeing?
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Hi @Vern_Shurtz , that sounds amazing. Once 1.4 is done, we will start working on 2.0. Currently, NatCam does scan barcodes on the fallback interface (using WebCamTexture). We added the ZXing.NET dependency in 1.3. Use the same code, no difference.
     
  29. shawww

    shawww

    Joined:
    Sep 30, 2014
    Posts:
    42
    Great, e-mailed you. Thanks!
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Seen and replied.
     
  31. tomihr2

    tomihr2

    Joined:
    Oct 25, 2010
    Posts:
    26
    Hello, I would like to use NatCam for barcode reading, but on my table Samsung Galaxy Tab A, it seems that auto focus is not working, on my phone LG G3 it working ok.
    I this known issue with this brand of tablet?

    Thaks
     
  32. tomihr2

    tomihr2

    Joined:
    Oct 25, 2010
    Posts:
    26
    I seems that auto focus works on front camera, but don't on rear one.
     
  33. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    428
    Hi, I'm porting my project over to Windows 8.1 and 10 and I get the following error with NatCam

    Assets\NatCam\Plugins\Managed\NatCamDispatch.cs(22,21): error CS0246: The type or namespace name 'Thread' could not be found (are you missing a using directive or an assembly reference?)

    I've tried to use
    Code (CSharp):
    1. #if NETFX_CORE
    2. using Windows.System.Threading;
    3. #else
    4. using System.Threading;
    5. #endif
    and

    Code (CSharp):
    1.  
    2. #if NETFX_CORE
    3. private Windows.System.Threading.Thread targetThread, mainThread, workerThread;
    4. #else
    5. privateThread targetThread, mainThread, workerThread;
    6.  
    7. #endif
    but get that the namespace thread is still not found... any ideas how to approach this?
     
    Last edited: Sep 5, 2016
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    This sounds like a device-specific issue. There isn't anything I can do to make it work on the Tab A.
     
  35. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    From here, the Windows.System.Threading namespace doesn't have a Thread class. Remove the NETFX_CORE macro so that it uses the default System.Threading namespace. Why are you using the NETFX_CORE macro/Windows.System.Threading namespace?
     
  36. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    428
    Threads aren't available in WSA through System.Threading... hence the original error and why I'm referencing Windows.system.threading... I saw someone reference needing to use Windows.system.threading... I'm not sure if threads are doable on windows? Also, I'm pretty green when it comes to coding... very... very... green... lol

    edit... just to clarify
    the first error shows up with Natcam code untouched...
     
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    It seems that you would have to use the Windows.System.Threading.ThreadPool class, specifically the ThreadPool.RunAsync method. I would also advise you to peruse the MSDN documentation on the Windows.System.Threading.ThreadPool class and the NatCamU.NatCamDispatch class source.
     
  38. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    428
    [everything you said to do]
    ../\
    ./.|\
    .. |
    .. |
    .. |
    ...O[my head}

    Sadly I don't think I'm up to the task of converting this. I may have to eliminate the windows 8.1 and 10 as build platforms... :/
     
  39. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    I don't think an issue like this should cause you to eliminate entire platforms. Read up on multithreading, look at MSDN, and the NatCamDispatch source. It might take you a short while, but it would not take much.
     
  40. latamakuchi

    latamakuchi

    Joined:
    Jan 16, 2013
    Posts:
    2
    Hi Lanre,

    I'm having the same issue as l-keustermans. I've sent you an email about it. If you could also send me the fix it'd be greatly appreciated. Your plugin is exactly what we need, but we can't use it because of this issue.

    Thanks!
     
  41. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    428
    So...from what I can tell natcam only uses threading on the supported devices and instead defaults back to the Platform's native approach so by sprinkling a bunch of #if NETFX_CORE 's around several scripts it looks like it should work. Unfortunately I'm working through some other build errors not related to Natcam before I can see if it works...

    WSA is a PIA
     
  42. Vern_Shurtz

    Vern_Shurtz

    Joined:
    Mar 6, 2009
    Posts:
    263
    Great! Will be trying it out shortly.

    ~V~
     
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Sent.
     
  44. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Not quite. NatCam uses threads on all platforms (except WSA of course). Threads are a mono implementation of the platform's low-level threading API's. I am not familiar with WSA because I haven't worked with it before.
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Let me know how it goes.
     
  46. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    831
    Is NatCam ready to provide footage from the iPhone 7+ Dual Camera to OpenCV?
     
  47. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    831
    I'm experiencing a similar issue with Dlib face tracking
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    Actually, we don't know whether iOS would allow running the two cameras simultaneously. Besides, developers do not yet have access to the device for development and testing, so I would say that the current answer is no.
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,661
    By similar, do you mean performance? If so, please profile the DlibFaceTracker example. You will see that the slowdown is caused by the OpenCV processing, not NatCam. You will have to optimize it by yourself to make your app run smoothly. As a starting point, I will advise you to look into threading.
     
  50. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    428
    Looks like NATCAM.dll is simply not compatible with Windows Store APIs, no workaround other then the DLL being rewritten from what I can see and am being told. At build time I get about 39 errors referencing the NATCAM.dll mostly in regards to threading. I know this is something you never advertised Natcam for but if in the future if you added UAS support that would be great. As is anothuer plugin (RAIN AI) has the same problem and looks to be broke even though it worked fine on 5.1.4 with Windows phone 8.0/8.1. Support for RAIN is dead so it looks like there is no point in pursuing UAS for this project. Shame cause the only reason I updated my project from 5.1.4 was for Natcam. :/
    Thx for your support.
    Chris

    Edit. So after further testing it looks like RAIN never would have worked on the Windows store even though I could build to the phone, so my project has been broken from the get go for UAS.
     
    Last edited: Sep 8, 2016