Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

NatDevice - Media Device API

Discussion in 'Assets and Asset Store' started by Lanre, Dec 17, 2015.

?

Should we add exposure controls in v1.3? This means dropping support for iOS 7

Poll closed Jun 10, 2016.
  1. Yes

    9 vote(s)
    75.0%
  2. No

    3 vote(s)
    25.0%
  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    What device does this occur on? And what is the framerate of your application?
     
    pontus_unity892 likes this.
  2. pontus_unity892

    pontus_unity892

    Joined:
    Nov 7, 2018
    Posts:
    7
    Samsung A5 and Samsung Galaxy S8+, and we run at 30 fps, thanks for the quick answer! Also, there seems to be some delay on CapturePhoto. Meaning i still have to hold the camera still for a short while after pressing the button. Any way to fix this?

    Thanks!
     
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Reducing the preview resolution should help with this. Maybe 1280x720.
    Only way to improve this is to reduce the photo resolution. On Android, the camera HAL only provides frames in the YUV format. Also, frames are always in the 'natural orientation' of the camera. So NatCam has to convert to RGBA and reorient the image, which takes longer for larger images.
     
    pontus_unity892 likes this.
  4. Luis-Disla-AppetizerMobile

    Luis-Disla-AppetizerMobile

    Joined:
    Sep 24, 2015
    Posts:
    2
    Hi Natcam & everyone,

    Great app! We've used it for a long time. We were making updates to an application and noticed we were still using v1.3 and decided to update to the latest version 2.1f2 . So we deleted the previous library folder, and downloaded the newest one into our project. There were many changes made to the API since 1.3 so we reimplemented the use of the library and it works well. However, when we try to build the Xcode project on the simulator, or even export the ipa for Release, it runs into this error every time.

    WARNING:
    -undefined dynamic_lookup is deprecated on iOS Simulator


    ERROR:

    error: Invalid bitcode signature
    clang: error: linker command failed with exit code 1 (use -v to see invocation)



    We've retraced our steps from deleting the old library to putting in the new one, and it seems even with commenting out all the Natcam code from the project and just including the Natcam plugin folder into the project triggers this bug.

    TLDR; Only just including the newest library of Natcam in our project is causing the above error.

    Any advice or guidance?

    Thanks.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Hey there! We've had one other report of this issue. Can you email me your invoice number? I'll share a rebuilt library with you.
     
  6. fafase

    fafase

    Joined:
    Jul 3, 2012
    Posts:
    156
    We faced the same and it was resolved by disabling arm7. I guess it has to do with the dropping of OpenGL support on iOS.

    I sent a mail to dev if there would be a workaround we missed or it would mean no support for iPhone 5 and iPad 4th gen.
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    I should probably divulge some insider information. Technically, we deprecated support for OpenGL ES on iOS. But in reality, nothing actually changed. The OpenGL ES deprecation on iOS happened in the NatRender support library...but the library isn't used on iOS. NatCam only uses NatRender on Android. I also suspect that the bitcode issue came from NatRender since we didn't explicitly set the bitcode generation marker when compiling. I've resolved that. If you'd like to get your hands on it ASAP, email me your invoice number.
     
  8. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    921
    Makes a very big difference for android!

    When will the official example be updated to 2.1?
     
  9. Bazookas

    Bazookas

    Joined:
    Jul 29, 2013
    Posts:
    7
    Hello,

    We are using Natcam Core - Webcam API 1.6f2 and we have 2 issues.

    -On android there is a significant delay between calling NatCam.CapturePhoto and the execution of the callback. During this delay the live camera input is still displayed. Is there a way to speed this up? Or should we disable live input manually?

    -Is there a fast/easy way to set the exposure bias? For now we take a picture and get the value from that picture, but this is not ideal as it will use flash automatically. We tried getting the value from the live input texture and this worked by casting it to a WebcamTexture in editor, but on mobile the type is a Texture2D and casting it to Texture2D results in a texture with no data.

    Updating the plugin is not an option. We are at the highest update possible for the Unity version we are using and because of other plugins, we can't update the Unity version. (This is a pretty old project at this point)
     
  10. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    I'm almost done with upgrading the example to 2.1. Once I complete the Pull Request, it'll be up to EnoxSoftware (dev of OpenCV) to review it and merge it. I'll keep you posted ;)
     
    SMR83 and ina like this.
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    This process is much faster on NatCam 2.1f2. I recommend upgrading.
    Oh, it seems like you are trying to estimate the current exposure before setting a new value (is this correct)? Because setting exposure bias is as easy as setting the DeviceCamera.ExposureBias property. If my guess is right, then you'll need to do a ReadPixels on the preview texture to get the texture data in system memory then look through that data. This is another place where you'll benefit greatly from using NatCam 2.1. There is now a CaptureFrame function that gives you access to the preview data in a byte[]. And it takes practically 0ms (since NatCam already keeps the data), unlike ReadPixels which has to synchronize the GPU before performing the readback.
    I understand. I still do recommend exploring an update; Unity and NatCam are only getting more performant over time.
     
  12. Luis-Disla-AppetizerMobile

    Luis-Disla-AppetizerMobile

    Joined:
    Sep 24, 2015
    Posts:
    2
    This updated library he sent us worked! Solved the issue completely. Thanks!
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Great!
     
  14. funkenschlag

    funkenschlag

    Joined:
    Dec 12, 2016
    Posts:
    3
    Thank you for your answer, i bought your asset...you did a good job with Natcam....iam happy :)
     
    Lanre likes this.
  15. Empu

    Empu

    Joined:
    Nov 27, 2012
    Posts:
    20
    I've been using NatCam on the Note 8 and it works perfectly. I've been requested to support the Lenovo Yoga 3 but the framerate is very choppy and crashes the device often. If I use WebCamTexture on the Lenovo I don't get these issues but I'm left without the ability to control exposure. Is there a workaround for something like this? Thanks.

    Preview resolution is 1280x720, 30fps.
     
    Last edited: Dec 20, 2018
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    The device runs Windows RT right? On this platform, NatCam falls back to WebCamTexture so it should work all the same. Can you upload the crash logs in a .txt file?
     
  17. Empu

    Empu

    Joined:
    Nov 27, 2012
    Posts:
    20
    Thanks for the reply. I've attached a crash log. The device is a Lenovo YT3-850F running Android 6.0.1.
    I've also uploaded a video testing the camera framerate:


    Is there a way to force it to use webcamtexture but still retain the exposure and focus controls?
     

    Attached Files:

  18. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Thanks for the video. From what I can tell, the Lenovo tablet doesn't accelerate conversion of the preview data from YUV to RGBA, which is why the framerate is so low. The workaround is to use a lower preview resolution. There is no way to use WebCamTexture and NatCam at the same time. As for the crash, it should be implicitly fixed in the next update.
     
  19. moyado

    moyado

    Joined:
    Jan 2, 2013
    Posts:
    6
    Any update with this? Thanks.
     
  20. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    I've submitted a PR. Waiting for EnoxSoftware to approve.
     
    SMR83 likes this.
  21. Empu

    Empu

    Joined:
    Nov 27, 2012
    Posts:
    20
    Thanks, I've reduced the preview resolution down to 380 x 240 and the photo resolution set as 1920 x 1080.
    As expected the preview refresh rate is a lot faster, however, when I try to take a photo using "NatCam.CapturePhoto(OnPhoto);" the texture resolution that comes out of "OnPhoto(Texture2D photo)" is only 380 x 240. Is this how it should function, I had assumed it should output at the specified photo resolution of 1920 x 1080.
     
  22. funkenschlag

    funkenschlag

    Joined:
    Dec 12, 2016
    Posts:
    3
    Hello Lanre, i am quite happy with your asset, the only problem is, when i try to implement a zoom slider, my app crashes on ipad (model: mtxn2fd/a) .

    i added to your minicam script in OnStart()

    zoomSlide.minValue = 1f; // the type of zoomSlide is Slider
    zoomSlide.maxValue = NatCam.Camera.MaxZoomRatio;

    and i added to Update()
    if (NatCam.IsRunning)
    {

    NatCam.Camera.ZoomRatio = zoomSlide.value;

    }

    without these lines everything runs fine, but with this lines it always crashes. thanks in advance.

    EDIT: when i set zoomSlide.maxValue in OnStart() to 2.0f without using NatCam.Camera.MaxZoomRatio everything works fine. Looks like NatCam.Camera.MaxZoomRatio returns a value wich causes the app to crash.
     
    Last edited: Jan 7, 2019
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    This depends entirely on the device and the camera being used. So even though you might request a preview or photo resolution, it is up to the device camera to decide what photo resolution actually gets used (cameras have a fixed set of supported preview and photo resolutions). You can see this by checking what the value of camera.PhotoResolution is immediately after setting it.
     
  24. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Can you upload the logs from Xcode in a .txt file? That's weird.
     
  25. karmatha

    karmatha

    Joined:
    Aug 25, 2012
    Posts:
    46
    Is the deprecated `NatCam.Camera.AutofocusEnabled = true;` the same as `NatCam.Camera.FocusLock = false`?

    Or what is the recommended migration strategy?
     
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Yup. Autofocus is naturally enabled, so is autoexposure and auto white balance. So instead, we inverted the control. So no need to explicitly set FocusLock to false when starting the camera.
     
  27. karmatha

    karmatha

    Joined:
    Aug 25, 2012
    Posts:
    46
    Okay thanks!
     
    Lanre likes this.
  28. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,282
    SMR83 and Lanre like this.
  29. shallwaycn

    shallwaycn

    Joined:
    Apr 3, 2012
    Posts:
    201
    Hello Lanre,

    Does NatCam support Vulkan? It seems only works with OpenGL Renderer.

    We need Vulkan for compute shaders, is there any options to support it ?

    Thanks

    [Edit]

    Never mind, seems my test device doesn't support Vulkan..
     
    Last edited: Jan 19, 2019
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    NatCam doesn't support Vulkan. Android itself doesn't fully support it.
     
  31. rattlesnake

    rattlesnake

    Joined:
    Jul 18, 2013
    Posts:
    138
    Hello Lanre,
    I send you a mail with some Logs.

    I'm using the lastest natCam 2.1.3 and my OnePlus5 on the MiniCam exemple.

    Portrait mode is working but when doing landscape-right it's crashing the app and when doing landscape left it's doing some crazy pixel art :


    Are you aware of this really annoying bug ?

    Thanks,
    Seb
     
  32. roadley

    roadley

    Joined:
    Mar 1, 2018
    Posts:
    5
    I have a very similar issue when working in Landscape. I believe in my case that there is a mismatch between the photo resolution and the preview resolution. I've tried to set my camera up like this:

    Code (CSharp):
    1. var camera = useFrontCamera ? DeviceCamera.FrontCamera : DeviceCamera.RearCamera;
    2.  
    3. camera.PreviewResolution = new Vector2Int(1920, 1080);
    4. camera.PhotoResolution = new Vector2Int(1920, 1080);
    5. camera.Framerate = 60;
    6.  
    7. NatCam.StartPreview(camera, OnStart);
    8.  
    But for some reason it does not stick, as my logging later reveals that it is set to:
    Changed camera format for photo resolution: (3264, 1836)
    Changed camera 0 framerate to 30.000000

    Device:
    iPad Air 2 - ios 11 - tested with the latest version of Natcam

    Am I doing it wrong?
     
    Last edited: Jan 23, 2019
    novaVision likes this.
  33. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    352
    Same problem as @roadley mentioned - can't change resolution.
    Furthermore, testing on iPhone7 and using memory profiler in XCode I have found that on Preview setup memory raised to +100mb, and each photo taken raised to another +100mb... After couple of shots I get app crash.

    What is wrong?
     
  34. roadley

    roadley

    Joined:
    Mar 1, 2018
    Posts:
    5
    So I managed to change the resolution now, by lowering it to 1280x720. I am now trying a new approach, where I will scale the full resolution photo down to 1920x1080 using a different method.
     
  35. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    352
    @roadley how did you change the resolution? How to get, what camera resolutions are suitable for our case?
     
  36. roadley

    roadley

    Joined:
    Mar 1, 2018
    Posts:
    5
    So that is what I am unsure about, I simply tried to lower the resolution, and noticed it changing once I put it low enough. As if it is expecting certain resolutions, but I am not sure how to expose those for each camera.
     
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Hey Seb, sorry for the late response. I'm aware of this issue (from here). I haven't gotten a chance to investigate since it only seems to happen on some devices, but I have a few ideas. I'll shoot you an email with more questions pretty soon.
     
    rattle-snake23 likes this.
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    This is expected behaviour on iOS. On iOS, the camera is configured with a format, of which there are a preset number to choose from (you can't create your own format). Each format has exactly one preview size and a few photo sizes to choose from (as few as one). As a result, when you set the preview resolution on a device camera, NatCam is really just picking the camera format with the closest preview resolution to what you asked for. On the other hand, when you set the photo resolution on a device camera, NatCam has to choose the closest matching photo resolution from those on the current format (if not, changing the photo resolution would inadvertently change the preview resolution). This is why your photo resolution setting doesn't seem to be setting.
     
  39. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    On iOS 12, apps have been reported to use a lot more memory. What's happening is that the profiler is exposing kernel-space resources (created in some part as a result of the current app) as the app's memory--something that wasn't done previously. As for photos, it sounds like you have a memory leak. You must explicitly destroy photo textures (Texture2D.Destroy); they don't get automatically cleaned up when they are dereferenced in C# code.
     
  40. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    352
    So, what is the exact way to lower down default resolution? I tried different approaches (setting up smaller resolution before camera start, changing it after initialisation etc) but nothing works
     
  41. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    This all depends on the device and camera. If you really need the photo to be at some resolution, you have to manually resize the photo texture.
     
  42. daleKaio

    daleKaio

    Joined:
    Jan 28, 2019
    Posts:
    3
    I think I am getting a similar issue with regards to photo resolution. I'm setting our photo resoulation to deviceCamera.PhotoResolution = new Vector2Int(9000,9000) to make sure we get the highest resolution possible, which when I Debug.Log(deviceCamera.PhotoResolution) I am getting 3264x2448 which is what we are looking for. When I use NatCam.CapturePhoto() I'm getting a resolution of 1280x720 (which is our preview resolution) from the texture received in the callback function. Is there something I am missing to being able to get a 3264x2448 resolution photo?

    Relevant Code:
    Code (CSharp):
    1.  void Start()
    2.     {
    3.         var camera = useFrontCamera ? DeviceCamera.FrontCamera : DeviceCamera.RearCamera;
    4.         if (!camera)
    5.         {
    6.             Debug.LogError("Camera is null. Consider using " + (useFrontCamera ? "rear" : "front") + " camera");
    7.             return;
    8.         }
    9.         //camera.PreviewResolution = new Vector2Int(4000, 4000);
    10.    
    11.         camera.PhotoResolution = new Vector2Int(4000, 4000);
    12.    
    13.         NatCam.StartPreview(camera, OnStart);
    14.         Debug.Log("Photo res: " + camera.PhotoResolution.x + " " + camera.PhotoResolution.y);
    15.  
    16.     }
    Code (CSharp):
    1.  
    2. void getPhoto()
    3.     {
    4.         waitingForCallback = true;
    5.         NatCam.CapturePhoto(returnTexture);
    6.     }
    7.     bool waitingForCallback;
    8.     void returnTexture(Texture2D photo)
    9.     {
    10.         waitingForCallback = false;
    11.         Texture2D myTexture = photo;
    12.         Debug.Log("Recieved photo res: " + photo.height + " x " + photo.width);
    13.     }
     
    Last edited: Jan 30, 2019
  43. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Apparently, this is a bug on Android. I'm looking into it.
     
  44. rattle-snake23

    rattle-snake23

    Joined:
    Jul 12, 2016
    Posts:
    5
    Hello Lanre, is this bug close to be fixed ?
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Yes I'm working on an update that should implicitly fix this. I'm trying to have a build by tomorrow or Thursday.
     
    rattle-snake23 likes this.
  46. mschwarz1

    mschwarz1

    Joined:
    Oct 22, 2013
    Posts:
    4
    Just picked this up today and ran into a couple of issues. I'm running a galaxy s7 edge and experiencing the orientation bug. Other then that the flash doesn't seem to work when taking pictures. Adding

    Code (CSharp):
    1. if (camera.IsTorchSupported) {
    2.         // Put on the torch
    3. camera.TorchEnabled = true;
    4. }
    5.  
    also doesn't cause the flash to activate. It does seem to cause my preview to become super blurry for some reason.
    When running the app, the toolbar I can pull down from the top of the screen does show that the flash is being controlled by the app so there is some level of communication going on. I can't post more info then that atm, but if you could redirect me to some instructions on submitting diagnostic data I could do that sometime tomorrow.
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    This is a known issue with camera2. Once I get the orientation stuff figured out, I'll get back to this.
     
  48. rattle-snake23

    rattle-snake23

    Joined:
    Jul 12, 2016
    Posts:
    5
    Hey I know you are very busy but just one little question :)
    With Camera2 do you think it is possible to record at the same time the front and the back camera ?

    Have a nice day :)
     
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,391
    Even though the camera2 API is advertised as such, most devices fall back to a legacy implementation (think the original camera API), so this is not supported.
     
  50. DongHweLee

    DongHweLee

    Joined:
    Aug 1, 2018
    Posts:
    9
    Hello, I am a user who is using NatCam.

    NatCam is used to create applications on IOS-based platforms.

    I am using Play (), pause () to play NatCamios as needed.

    Each time you use Play and Pause alternately, about 14 to 24 memories are accumulated.

    Can you give me the answer to this?

    thank you.
     
unityunity