Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NatDevice - Media Device API

Discussion in 'Assets and Asset Store' started by Lanre, Dec 17, 2015.

?

Should we add exposure controls in v1.3? This means dropping support for iOS 7

Poll closed Jun 10, 2016.
  1. Yes

    9 vote(s)
    75.0%
  2. No

    3 vote(s)
    25.0%
  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This question belongs on the NatCorder forum thread, not here. Please post this question there.
     
    ina likes this.
  2. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Yup. Use the CaptureFrame function. It gives you the preview buffer into a byte[].
     
    karmatha likes this.
  3. SniperED007

    SniperED007

    Joined:
    Sep 29, 2013
    Posts:
    345
    It seems to get worse each time you record a new video (using the ReplayCam sample scene on Android) after about the 4th or 5th go the voice breaks-up/repeats/echoes etc.

    Yet if I create a simple script in Unity to just repeat back everything on the mic it doesn't experience the same issue?

    AudioSource aud = GetComponent<AudioSource>();
    aud.clip = Microphone.Start(null, true, 1, 44100);
    aud.Play();
     
  4. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This sounds like NatCorder, not NatCam. Please be respectful and post on the correct forum thread.
    This sounds unlikely. If you play an AudioSource that is playing the microphone clip, you will hear the Larsen effect because the microphone is picking back what the AudioSource is playing, which is the microphone itself (a closed loop).
     
  5. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    Okay - just checking if it isn't just something I'm setting up wrong on my end..
     
    Lanre likes this.
  6. AlienFreak

    AlienFreak

    Joined:
    Jul 2, 2012
    Posts:
    40
    Is it possible to have rear and front cameras doing previews at the same time for iOS and/or Android? (Purpose is to do Picture in Picture of the front camera to record peoples reactions to things they see through the rear camera)
     
  7. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This is not possible. Mobile OS'es have a restriction that only one camera can be active at any given time (and in most cases, this stems from hardware constraints).
     
  8. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Hi,

    On iPhone X or iPhone 8 when i take a photo (with camera and minicam example) result is a purple image. All it's ok on device < iPhone 8.
    Minicam Preview Resolution : Highest
    Minicam Photo Resolution : Medium

    Any idea plz ?
     
  9. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Change the preview resolution to something like 1280x720 or 1920x1080. What's going on is that because the preview resolution is so high, the photo resolution is also at least as high (even though you are requesting Medium). When the photo gets taken, its resolution is too big to be uploaded to a GPU texture, resulting in a purple screen.
     
  10. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    It is not possible to enter a resolution, I only have HD, Full HD, Highest, Medium and Lowest
     
  11. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Use HD or FullHD.
     
  12. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Are you sure ? I have this problem with Highest...
     
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Yes absolutely. On iOS, the active preview resolution determines what the photo resolution is. As a result, you have to change the preview resolution to change the photo resolution.
     
  14. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Same thing for Android ?
     
  15. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    No, this only applies to iOS.
     
  16. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Ok but it's possible to let HD and Medium for Android so ?
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Yes, it is.
     
  18. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Thx a lot for your quick help !
     
    Lanre likes this.
  19. AlienFreak

    AlienFreak

    Joined:
    Jul 2, 2012
    Posts:
    40
    It appears some phones (Android ones) can like the Samsung S7
     
  20. AlienFreak

    AlienFreak

    Joined:
    Jul 2, 2012
    Posts:
    40
    Is there a way to do a live preview stream to another device (video chat) with NatCam or NatCorder?
     
  21. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This is a Samsung-specific extension. It is not officially supported by Android.
    No, this is not possible.
     
  22. Marco_Joao

    Marco_Joao

    Joined:
    Apr 4, 2016
    Posts:
    4
    Hello im planing to buy this assent, here is any demo.apk that I can try?
    And the main reason is to use this asset for AR, so direct acess to OpenCV Mat or color32(without gpu flush) could be nice.
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Here is the NatMicCorder Demo, using all of our API's:
    - NatCam for the camera preview
    - NatCorder for recording the screen
    - NatMic for recording the microphone audio (NatCorder puts this audio in the recording)
    - NatShare for social sharing the recording

    NatCam is built with performance in mind, and is designed for seamless integration for OpenCVForUnity. Check out the official NatCam-OpenCV project, maintained by the authors of OpenCVForUnity.
     
  24. BrodeyPison

    BrodeyPison

    Joined:
    Jun 12, 2018
    Posts:
    4
    I've been working with NatCam for a while now and just made a new project and noticed a weird glitch. I have simple logic using an if statement to switch between front and rear cameras. I noticed only in my new project, which uses the exact same code as a previous working camera app, that whenever I try to switch cameras the preview freezes on the current camera. I've tried doing a few work arounds, like stopping and restarting the preview with the opposite facing camera, but nothing seems to work. Any idea as to why this could be happening?
     
  25. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Hey there. What device are you running on? What OS version? Can you share your code and a screenshot or video?
     
  26. BrodeyPison

    BrodeyPison

    Joined:
    Jun 12, 2018
    Posts:
    4
    I'm running the app on both a Samsung S8+ and a Samsung S9+ both on Android 8. If I build my old project the camera switching works perfectly fine, it's only in this one specific project that the preview freezes, but not the app itself.

    public void SwitchCameras()
    {
    if(NatCam.Camera == DeviceCamera.RearCamera)
    {
    NatCam.Camera = DeviceCamera.FrontCamera;
    }
    else if (NatCam.Camera == DeviceCamera.FrontCamera)
    {
    NatCam.Camera = DeviceCamera.RearCamera;
    }
    }
     
  27. BrodeyPison

    BrodeyPison

    Joined:
    Jun 12, 2018
    Posts:
    4
    Just noticed something while attempting to fix the issue. Once I call that function, it freezes the preview, but if I switch to another app and then come back into the Unity app it actually starts with the switched camera
     
  28. karmatha

    karmatha

    Joined:
    Aug 25, 2012
    Posts:
    50
    Lanre I just wanted to tell you that on Android, calling `NatCam.Release()` after *denying camera permissions* results in an error. Also, subsequent attempts to call `Play` will now fail. I have been able to work around this issue by only calling `NatCam.Release()` when a camera is actually started, but still I would not expect a crash in this case.
     
  29. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This looks like a bug we've seen. We are working on an update that implicitly fixes this bug. If you'd like early access, shoot me an email.
     
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Noted. Would it be useful to have `NatCam.Play` stop and log an error if the camera permissions aren't granted? We can easily check whether permissions have been granted with `NatCam.Implementation.HasPermissions`.
     
  31. karmatha

    karmatha

    Joined:
    Aug 25, 2012
    Posts:
    50
    Well I don't know know about that. The thing is I called `NatCam.Release()` in my OnDestroy, (regardless if permissions were granted or not..). I didn't expect a crash when trying to release a camera that wasn't started in the first place. In my opinion `Release()` should just silently continue if there is nothing to release...
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    This issue will implicitly be fixed in the next version, 2.1. We have improved the frontend API, replacing the Play, Pause, and Release functions with StartPreview and StopPreview.
     
  33. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    I'm back for the same problem :

    with HD (preview) and Medium (photo) always a purple image on iPhone X... :(
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Can you email me the full logs from Xcode. Thanks.
     
  35. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    No i can't sorry because i don't have iPhone X. Tested on iPhone X of a friend with Testflight.

    Customers reviewers all says they are this problem with iPhone X so.

    All it's ok on iPhone 7 or lower.
     
  36. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    In Player Settings, do you have 'Auto Graphics API' enabled?
     
  37. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Okay since this is becoming more of an issue now that iOS cameras are getting better, I will add a flag in the new version that gives priority to the photo resolution when configuring the camera. This will allow for explicitly picking a photo resolution that is less than the maximum supported texture size on Metal.

    EDIT: I'm doing some reading online and apparently Metal on devices like the iPhone X can support textures as large as 8192x8192. The maximum iPhone X camera photo resolution is less than this. Please try to get your hands on an iPhone X and send me the logs.
     
  39. KaviAR_Tech

    KaviAR_Tech

    Joined:
    Aug 23, 2018
    Posts:
    4
    Hello,
    Thank you for NatCam. I have two problems please:
    1- In the MiniCam scene, the example scene, the switch camera button does not work, and my app is totally frozen.
    2- I've tried to integrate MiniCam scene in my project. Without switch camera, everything is great unless i quit the scene and come back (Loading another scene and going back). When doing so, i have no preview mode... everything is white and only the captured pic is displayed (when i try to capture a pic of course by tapping everywhere in my screen).

    My phone : OnePlus 6
    OS: Android 8.1.0

    Thank you very much
     
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    We have a fix for this coming in the new version. The new version should be submitted to the Asset Store this coming week.
     
    KaviAR_Tech likes this.
  41. unitycodergp

    unitycodergp

    Joined:
    Feb 1, 2017
    Posts:
    47
    Problem of purple image is on iPhone 8 and upper
     
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Noted. I'm still working on the 2.1 update.
     
  43. aRayon

    aRayon

    Joined:
    Oct 27, 2016
    Posts:
    4
    Hello!

    I'm a bit confused with NatCam and OpenCV integration... What would be the most correct way to achieve this?

    We bought this asset several months ago with this purpose and by then was done using OpenCVBehaviour, which, as I remember, did not work satisfactorily and apart I’m reading that it is deprecated…

    On the other hand, using the code of EnoxSoftware (NatCamWithOpenCVForUnityExample), in any of the examples, as soon as you have access to the camera the App crashes : https://github.com/EnoxSoftware/NatCamWithOpenCVForUnityExample/issues/2#issuecomment-432622050

    What would be the correct (and most efficient most efficient regarding the performance-cost) of integrating both libraries?

    Thank you!
     
  44. Brian_Dean

    Brian_Dean

    Joined:
    Aug 29, 2018
    Posts:
    2
    Hey,
    when the PreviewResolution and PhotoResolution is set to Highest, the photos get cropped on the right and left.
    But works fine on 1920x1080p.
    Is there any way to resolve this?
     
  45. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    I recommend simply fetching the preview data from NatCam using the NatCam.CaptureFrame function, then putting the data into a Mat using Utils.copyToMat. The OpenCV example uses `NatCamPreviewToMatHelper`, but this is to preserve compatibility with the WebCamTexture workflow. I do not recommend using the helper script for an actual project.
    This only occurs on a few Android devices. We have a fix coming in the next version, 2.1.
    Simply follow the NatCam event pattern, so having a dedicated OnStart and OnFrame function that get registered to NatCam.OnStart and NatCam.OnFrame respectively. Then in OnFrame, get the preview data into a byte[], then copy the byte[] into an OpenCV Mat, and use the Mat. Here is a gist outlining all this.

    The reason why I recommend this approach as opposed to using NatCamPreviewToMatHelper is that the NatCam-only approach is event-driven, with the OnFrame function being called only after the camera actually has a new frame available. NatCamPreviewToMatHelper on the other hand updates the preview matrix on every frame GetMat is called which in many cases is wasteful.
     
  46. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Chances are that your display plane or RawImage is being stretched to fill the screen. This is not a problem with the NatCam preview texture itself. Check your UI setup.
     
  47. fafase

    fafase

    Joined:
    Jul 3, 2012
    Posts:
    161
    Hi,

    I guess it has to have been mentioned already but just in case, starting/stopping the camera on Samsung Galaxy S6 will break. First time works, next attempt will show a still blurry image.

    Tried same on J7 and worked fine so I guess there is something about Galaxy (I have a feeling I have seen it before).

    Regards
     
  48. karmatha

    karmatha

    Joined:
    Aug 25, 2012
    Posts:
    50
    Hi Lanre, could you give us an estimation about when 2.1 will be available?
     
    KaviAR_Tech likes this.
  49. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Can you upload the logs from running on the S6? I haven't had reports of this before.
     
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,970
    Late next week. It's pretty close to being done.