Search Unity

NatDevice - Media Device API

Discussion in 'Assets and Asset Store' started by Lanre, Dec 17, 2015.

?

Should we add exposure controls in v1.3? This means dropping support for iOS 7

Poll closed Jun 10, 2016.
  1. Yes

    9 vote(s)
    75.0%
  2. No

    3 vote(s)
    25.0%
  1. galent

    galent

    Joined:
    Jan 7, 2008
    Posts:
    1,070
    I've just purchased Natcam, and the example scene isn't working correctly in the editor. The video screen just creates a coloured flicker. If i press the picture button it does show a complete image. Is there something I can try to get video working correctly?

    Thanks
     
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi @galent. Can you post a screenshot? I am unfamiliar with this issue. I should note that in the editor, NatCam uses WebCamTexture.
     
  3. ozawachev

    ozawachev

    Joined:
    Feb 8, 2014
    Posts:
    1
    Do you have face detection sample code for OpenCV?
     
  4. galent

    galent

    Joined:
    Jan 7, 2008
    Posts:
    1,070
    Here's a screen shot

    it seems to be related to a UV issue (described here)

    That said, when I build and deploy to my Galaxy S5 (Android OS 6.0.1), the image shows but auto-focus doesn't seem to be working, the snapshot button seems to stretch the image a little and barcode detection... not sure if that's working either... hard to tell since it won't autofocus.

    Any thoughts?
     
  5. ericalan

    ericalan

    Joined:
    Jun 10, 2016
    Posts:
    17
    I had issues with the focus too, I initialized NatCam with Verbose mode and sent Lanre my logcat logs, might be useful to send your logs too?
     
  6. galent

    galent

    Joined:
    Jan 7, 2008
    Posts:
    1,070
    I emailed the logcat logs. Still not able to use the autofocus, which I'm hoping is the reason the natcam also can't parse the barcodes I'm putting in front of it.
     
  7. robbgray

    robbgray

    Joined:
    Feb 4, 2014
    Posts:
    3
    I'm also seeing nothing in the editor when running the exmple. I'm using it just for testing but having it work in editor would be immensely useful.

    This error seems relevant:
    NatCam Error: Camera is already active
    UnityEngine.Debug:LogWarning(Object)
    NatCamU.Internals.NatCamExtensions:Warn(String) (at Assets/NatCam/Plugins/Managed/NatCamExtensions.cs:170)
    NatCamU.NatCam:set_ActiveCamera(DeviceCamera) (at Assets/NatCam/Plugins/Managed/NatCam.cs:89)
    NatCamU.UnitygramBase:Start() (at Assets/NatCam/Scripts/UnitygramBase.cs:31)
    Unitygram:Start() (at Assets/NatCam/Examples/Unitygram/Unitygram.cs:25)

    And can't seem to press the button on the device (but I do see a preview). Switching from front to back causes a crash 50% of the time. I'll send logcats later today.
     
  8. mrbroshkin

    mrbroshkin

    Joined:
    Aug 14, 2012
    Posts:
    53
    Hi, I set FOV from NatCam.ActiveCamera.HorizontalFOV and is 56.7 on iPhone 5, but close to real is 32. Why?

    Also autofocus is not working on Samsung Galaxy Tab-S.

    SONY Experia Z3 Compact - auftofocus is not working!
     
    Last edited: Jun 28, 2016
  9. friuns

    friuns

    Joined:
    Jan 14, 2016
    Posts:
    23

    on samsung xcover 3 i get this bug, please help!

    SM-G388F
    android 5.1.1
     
    Last edited: Jun 28, 2016
  10. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi guys! Sorry I have been silent for the past few days. I have just gotten back from a trip. NatCam 1.3b1 is on track now featuring universal barcode detection, which means you can now scan barcodes in the editor.
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Not yet. I'll work on it once NatCam 1.3 is released.
     
  12. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    I would have to agree with you that it a UV issue. NatCam uses WebCamTexture when running in the editor, so it must be related to that. I have a suspicion though: try disabling NatCamPreviewScaler if you have it enabled and let me know if it shows properly. I replied to your email concerning the S5 focusing issue.
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    It could be a focusing issue. It could also be that Google Play Services is not available. We've added a check in 1.3 to make sure that it is available for use. I would email you the latest 1.3 beta once it is completed.
     
  14. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi. I don't quite understand. The horizontal FOV from the camera depends on what format it is set to. This is specific to the AVFoundation framework.

    As for autofocus, can you send me the logs? Email me at olokobayusuf@gmail.com. NatCam 1.2 has a bug where it sometimes ignores setting autofocus when you set it before the preview starts. You can workaround it by calling this:
    Code (CSharp):
    1. NatCam.ExecuteOnPreviewStart(() => NatCam.ActiveCamera.FocusMode = FocusMode.AutoFocus);
    This has been fixed in NatCam 1.3.
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi @fruins. Email me at olokobayusuf@gmail.com. I will send you a new APK to test. This sort of issue should be inherently fixed in NatCam 1.3.
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Fixed!
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    NatCam 1.3 is very close to submission. It will probably be sent in tomorrow.
     
  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    We are starting a YouTube Tutorial Series for NatCam. Here is the current list of tutorials we plan to release:
    1. Starting Off. UnitygramBase, custom initialization, and switching cameras.
    2. Scaling the Preview. Using NatCamPreviewScaler.
    3. Focusing and Focus Modes. Using NatCamPreviewGestures for easy focusing.
    4. Zooming. Using NatCamPreviewZoomer with NatCamPreviewGestures.
    5. Capturing Photos. Supplying PhotoCallbacks and using PhotoSaveMode.
    6. Detecting Barcodes. Using BarcodeRequests and detecting multiple formats at once.
    7. Detecting Faces. Creating a simple rectangle that follows a face.
    8. Using Shaders. Creating a black-and-white preview.
    9. Advanced: Realtime Instagram-Style Color Grading using Look Up Textures.
    10. Unitygram. Creating a simple camera app that combines these lessons.
    Please let me know of any requests or suggestions you have for these tutorials. Thank you!
     
    D3m0n and pixelsteam like this.
  19. galent

    galent

    Joined:
    Jan 7, 2008
    Posts:
    1,070
    Hi Lanre, I just downloaded and tried the new 1.3. the screen is working fine in the editor (unity 5.3.5), but the PDF417 specific scanner isn't working (editor or on my samsung), I can scan the code if I set it to all detection formats, but only if I designate the fallback interface (native won't scan QR or PDF417 in editor or on mobile.

    Any idea what's happening there?
     
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi @galent can you send me logs your samsung? I know that PDF417 takes a little longer to scan than regular barcodes, but QR should get picked up immediately. The native interface doesn't run in the editor--only the fallback does. Try scanning QR when you generate the logs (and remember to put verbose mode on).
     
  21. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    This might only be an iOS 10 beta error, but just a heads up.

    Using Metal as the graphics API results in quite severe stuttering on Metal supported devices. Works great on OpenGLES 3, so this could very well be the case of a beta issue. Hitting a constant 60fps while accessing the camera and rendering items on top feels super nice, all while the camera feed being of great quality.

    This is without a doubt one of the best purchases i have made on the app store, the efficiency, ease of use and support is all top notch.
     
    Lanre likes this.
  22. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Thanks for the notice. I will keep it in mind. NatCam 1.3 should not be affected since it doesn't perform any drawing; it just uploads preview data to the GPU. Thanks for the great review! :D
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi guys.

    I just noticed a bug in NatCam 1.3.

    When running a readable preview on iOS, switching cameras might cause a hard crash (EXC_BAD_ACCESS). The fix is trivial: In NatCam.mm, add this RGBA32 nullifying line at the end of -(void) SwitchCamera:
    Code (CSharp):
    1. ...
    2. [session commitConfiguration];
    3. if (RGBA32 != NULL) delete [] RGBA32; RGBA32 = NULL; //Add this line

    Sorry, thank you, and happy coding!
     
  24. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    The stuttering only seems to affect the preview texture coming from NatCam when using Metal, but it's an iOS beta, then again i might have broken something somewhere. ¯\_(ツ)_/¯.
     
  25. admob

    admob

    Joined:
    Jun 2, 2015
    Posts:
    3
    Dear,

    I just bought NatCam & Vuforia for 550$,
    I put them together in my project but it looks like conflicts dex files.
    Is there any solutions to combine them?
     
  26. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Hi, just bought your asset, looks to be great! I'm hoping I can prevent updating my Unity Version to use it though as I have several reasons not to update my project to 5.2 and up. Any chance this will work on 5.1.4? I get this error upon import:
    Assets/NatCam/Scripts/NatCamPreviewScaler.cs(19,64): error CS0246: The type or namespace name `IMeshModifier' could not be found. Are you missing a using directive or an assembly reference?

    Obviously that is part of the updated UnityEngine.UI but not sure if anything can be done to add that functionality to 5.1.4... any help would be appreciated. Perhaps even an older version of NatCam would be fine as my main interest is to have the camera feed as a background for my AR project.

    Edit: this error shows as well if I simply comment out the IMeshModifier bit

    Assets/NatCam/Scripts/NatCamPreviewScaler.cs(79,33): error CS0246: The type or namespace name `VertexHelper' could not be found. Are you missing a using directive or an assembly reference?
     
    Last edited: Jul 14, 2016
  27. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    You do not need the scaler to make NatCam work. I'm fairly sure you could just delete that script and make your own scaling method like what i'm doing, you do not need to draw it to a RawImage component.

    And just me being curious, why aren't you able to go above version 5.1.4?
     
  28. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Hi @admob . NatCam and Vuforia are incompatible. I emailed Vuforia a long time ago so that we could make them work together, but I never got a reply. What are you trying to achieve? Depending on what you want to do, I can advise you on which package to use and how.

    EDIT: I just emailed them again. I'll let you know if I hear back.
     
    Last edited: Jul 14, 2016
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Good point! @christougher, the scaler modifies individual vertices, but the logic stays the same:
    Code (CSharp):
    1. Find the aspect ratio of the texture applied (the dim vector). Calculated as: float ar = dim.x / dim.y.
    2. Depending on the scale mode, multiply or divide the height or width (respectively) of the rect transform by the aspect ratio calculated in (1).
    3. Set your new width or height.
    This can easily be translated to using rectTransform.localScale instead of the vertices.
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Does NatCam work on Android in 5.1.4? I would think not because Android depends on the new Unity rendering LLAPI introduced in 5.2. If you aren't using NatCam on Android, then you're going to be fine.

    Another error you should encounter is the Texture2D.LoadRawTextureData(IntPtr, int) overload that also wasn't introduced till 5.3 (because double-buffering the preview data in native and managed memory meant 2x memory cost). This is why the minimum recommended Unity version is 5.3. To get around this, replace every instance of:
    Code (CSharp):
    1. Texture2D.LoadRawTextureData(IntPtr, int);
    with
    Code (CSharp):
    1. Texture2D.LoadRawTextureData(Ext.MarshalNativeBuffer(IntPtr, int)); //Where Ext is an alias for NatCamExtensions
     
  31. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Using it for both IOS and android. Had quite a few incompatability issues with 5.2 which led me to rollback to 5.1.4 a while back. Trying wating to recreate anything here. I'm not very good at scripting.

    Also I get this when NatCam is first imported:

    System.IO.FileNotFoundException: Could not find file "C:\Users\Chris\Documents\Turret\Assets\NatCam\Plugins\Dependencies\ZXing.NET\zxing.unity.dll.mdb".

    Deleting the scaler script just leads to more errors with other scripts.... NatCamNativeInterface throws multiple, and I imagine it all snowballs from there... :/

    Maybe it's time to just update to 5.3 after all. Gonna break something for sure lol.
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    The errors don't surprise me. I personally advise you to upgrade; the issues you faced in 5.2 might have been solved.
     
  33. spxyz

    spxyz

    Joined:
    Feb 7, 2014
    Posts:
    9
    Hello,
    I have a question about NatCam.PreviewFrame
    It always returns the texture in a horizontal orientation and stretches if necessary.
    Am i right?
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    Yes, the texture returned is always in landscape orientation. But no stretching is applied. This is because the natural orientation of the camera is always landscape--regardless of the device orientation. This plugs in directly to the rendering pipeline, so the rendering pipeline creates a texture that is always landscape. But the preview is always oriented correctly, only that its height and width are not swapped in portrait mode. This means that if you save it with EncodeToPNG and read it from the device gallery, it will be stretched. If you load it back into a texture in Unity however, it will be correctly displayed. Note that this only occurs when the app is in portrait mode.

    One workaround I can think of is creating a RenderTexture with the width and height swapped then blitting from NatCam.PreviewFrame and performing a readback from the GPU (Texture2D.ReadPixels).
    Code (CSharp):
    1. if (appIsPortrait) {
    2. Texture2D frame = NatCam.PreviewFrame;
    3. RenderTexture correctRenderTexture = new RenderTexture(frame.height, frame.width, 0); //Swap width and height
    4. correctRenderTexture.Create();
    5. Graphics.Blit(frame, correctRenderTexture);
    6. Texture2D correctFrame = new Texture2D(correctRenderTexture.width, correctRenderTexture.height, frame.format, false);
    7. RenderTexture.active = correctRenderTexture;
    8. correctFrame.ReadPixels(new Rect(0, 0, correctRenderTexture.width, correctRenderTexture.height), 0, 0);
    9. RenderTexture.active = null;
    10. correctRenderTexture.Discard();
    11. correctFrame.Apply();
    12. }
     
    Last edited: Jul 15, 2016
  35. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    I did manage to reach Vuforia and have started preliminary talks with their developers. NatCam-Vuforia interoperability might become a reality. Let's keep our fingers crossed.
     
    D3m0n likes this.
  36. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    K, I updated to 5.3.4 (which I already had installed for another project) and now I get this error on import... would this be something fixed with 5.3.5?

    Assets/NatCam/Plugins/Managed/NatCamNativeInterface.cs(465,45): error CS1674: `AndroidJavaClass': type used in a using statement must be implicitly convertible to `System.IDisposable'

    Edit... Android SDK needed updating... Doing that right now...

    Edit: Didn't help.... same error, but it works fine in another 5.3.4 project... any guidance what the problem could be?

    Edit (again) anything I can find online references adding an Entity Framework dll (EF5 or EF6) to the project? But this seems to have more to do with Visual Studio?????
     
    Last edited: Jul 15, 2016
  37. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    I would love for that to happen. We use Vuforia for some marker based tracking solutions, but the speed and quality of the camera feed seems much better with NatCam. I know it's early, but would the integration be that NatCam would pass the captured frames to the Vuforia tracking algorith and bypass their camera capture layer or would NatCam just extend the Vuforia camera capture layer?
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    This might be a lot to ask, but try reinstalling Unity. AndroidJavaClass does implement IDisposable. Some NatCam developers use 5.3.4 and have not run into this issue, so I think there might be something wrong with Unity's assemblies. See the Dispose function here.
     
  39. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Yeah, I'm installing 5.3.5 right now. Natcam works fine on 5.3.4 on another computer, I'm wondering if it has to do with my Visual Studio 2013...
     
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    The way I would want it to work--and I'm sure other developers would want this too--is that NatCam will pass the camera preview to Vuforia for processing. The goal of combining NatCam and Vuforia is so that developers have access to NatCam's broad, customizable control API and Vuforia's specialized AR functionality. I am yet to hear back from their developers, but I will try to get as close to such integration as possible.
     
  41. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    I doubt that it's because of Visual Studio; it sounds more like incomplete/corrupted assemblies. Let me know how it goes.
     
  42. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Still get the same error... fresh download of 5.3.5. How can I fix something like this? It's obviously a problem with something other then Natcam so extra thanks for the guidance.
     
    Lanre likes this.
  43. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    So, looking at some similar problems online I got rid of the error with these changes:

    Code (CSharp):
    1.                         //using (AndroidJavaClass NCNAClass = new AndroidJavaClass("com.yusufolokoba.natcam.NatCam")) {
    2.                         AndroidJavaClass NCNAClass = new AndroidJavaClass("com.yusufolokoba.natcam.NatCam");
    3.                        
    4.                             if (NCNAClass != null) {
    5.                                 NCNA = NCNAClass.CallStatic<AndroidJavaObject>("instance");
    6.                             }
    7.                        
    Hope it doesn't break anything...
     
  44. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Now when I try to build I get a "cannot merge android manifest files..."
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    You have to manually add this at the end:
    Code (CSharp):
    1. NCNAClass.Dispose();
    This is what IDisposable and the 'using' block automatically do for you.
     
    christougher likes this.
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    If it is complaining about the minSdkVersion, you must change the minimum API level in Player Settings to at least API level 16.
     
  47. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    K android files not merging was fixed by changing the minimum API level for Android to 16, but now I get two errors when building that it cannot convert java class into dex format.... I've even updated the Java Development Kit (this error shows the older one) and Android SDK but that doesn't work. Everything builds fine when Natcam is removed...

    Edit: Would this be caused by conflicting android manifests? I have several from google play services, Unity Ads and a mystery one. I'll weed them out and see if that works...

    Edit: Working!

     
    Last edited: Jul 16, 2016
  48. christougher

    christougher

    Joined:
    Mar 6, 2015
    Posts:
    487
    Oh thanks, got that figured out, lol
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    2,979
    It seems that it is because of google ads. Nice that you have it working!
     
  50. Denis-Shevchenko

    Denis-Shevchenko

    Joined:
    Jul 17, 2016
    Posts:
    17
    I have the same problem, but this time, changing of API level to 16 or any others levels doesn't work for me:



    CommandInvokationFailure: Unable to merge android manifests. See the Console for more details.
    /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java -Xmx2048M -Dcom.android.sdkmanager.toolsdir="/Projects/Android/sdk/tools" -Dfile.encoding=UTF8 -jar "/Applications/Unity/PlaybackEngines/AndroidPlayer/Tools/sdktools.jar" -

    stderr[]
    stdout[
    Warning: [Temp/StagingArea/AndroidManifest-main.xml:14, /Projects/GhostAR/Temp/StagingArea/android-libraries/NatCam/AndroidManifest.xml:3] Main manifest has <uses-sdk android:targetSdkVersion='22'> but library uses targetSdkVersion='23' settings.png
     
    Last edited: Jul 17, 2016
unityunity