Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

NatDevice - Media Device API

Discussion in 'Assets and Asset Store' started by Lanre, Dec 17, 2015.

?

Should we add exposure controls in v1.3? This means dropping support for iOS 7

Poll closed Jun 10, 2016.
  1. Yes

    9 vote(s)
    75.0%
  2. No

    3 vote(s)
    25.0%
  1. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Yup, you're gonna get the latency right back if you have to pipe the sample buffers through an AudioSource first. You'll need to find an implementation of the FFT. There are open source implementations in the wild.
    I only issue refunds for technical issues (bugs) that I can't fix in a reasonable amount of time (1wk). I'd recommend going through the online docs so you have a very complete understanding of what the API offers. Also feel free to ask me any questions directly.
     
  2. asa989

    asa989

    Joined:
    Dec 18, 2015
    Posts:
    52
    Hey there, Thank you for the plugin. I noticed some quality issues when using
    device.CapturePhoto()
    for back cameras. I hope you can help me. I am using Nat Device 1.0.2 in Unity 2020.2 IOS platform and I'm using iPhone XS.

    I have two problems:
    1- I cannot capture photo with same resolution as the iPhone native camera app. I always get something a bit lower and has lower quality. Which results to washed out photo in details. For example: I can take a photo on iPhone camera app with 3024x4032 resolution. if I do
    device.photoResolution = (3024,4032);
    then I'll get a 2376x4224 photo. Why I cant get same resolution as the native app camera? (same thing happens with the front camera but details are not that much washed out)

    Note: I put the iPhone camera app on video mode and I toke a screenshot from captured video. The quality is same as using
    device.CapturePhoto()
    . I hope it helps. all of my tests are on face.

    2- I dont have access to 60FPS or higher even if I set the FPS for both unity and the plugin.

    Thanks
     
    Lanre likes this.
  3. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    I can't speak to photos being washed out; NatDevice doesn't modify the pixel data it receives from AVFoundation. As for the resolution being lower, on iOS you'll need to understand how camera devices work:

    On iOS, every camera device is configurable with distinct "formats". A format has exactly one preview resolution and a small set (usually anywhere between 1-3) of photo resolutions. So when you set the preview or photo resolution, NatDevice has to choose a format to set, but must give precedence to either the preview or photo resolution to do so. NatDevice gives the preview resolution precedence, which means that it will only switch formats when the client (you) sets a preview resolution. When a photo resolution is set, NatDevice will keep the current format as-is (so that the preview resolution doesn't change) and will choose a photo resolution from the small set of available photo resolutions in the current format.

    What's happening for you is that with the current format (as dictated by the preview resolution), the photo resolution is 2376x4224, not 3024x4032. Oh also, when setting either resolution, it should always be in landscape (bigger number first).

    How are you measuring the frame rate? And have you profiled your app to see that your app can in fact run at 60FPS? If Unity isn't running at 60FPS, then that wouldn't have anything to do with NatDevice.
     
  4. DangerosoDavo

    DangerosoDavo

    Joined:
    Jan 18, 2014
    Posts:
    2
    Hi there

    I have emailed but I shall try here just incase this is where I'm meant to ask for any support.

    I have come into an issue where I'm trying to take a photo and use the flashMode setting, this setting does not seem to be respected on my android device.

    How I'm getting the camera: (works fine)
    Code (CSharp):
    1.  var criterion = MediaDeviceQuery.Criteria.RearFacing;
    2.         var query = new MediaDeviceQuery(criterion);
    3.         cameraDevice = query.currentDevice as CameraDevice;
    4.         // Start the camera preview
    5.         cameraDevice.flashMode = FlashMode.On;
    6.         previewTexture = await cameraDevice.StartRunning();
    7.         // Display the preview in our UI
    8.         rawImage.texture = previewTexture;
    Upon changing the flash mode I have set a text label to:
    Code (CSharp):
    1. DebugText.SetText("Flashmode Supported - "+cameraDevice.flashSupported + "\nFlashMode is now:"+System.Enum.GetName(typeof(FlashMode), cameraDevice.flashMode));
    This reads as expected, flash supported is true and the value is correct on the flashmode.

    How I take the photo:
    Code (CSharp):
    1. cameraDevice.flashMode = flashMode;
    2.         var tx2d = await cameraDevice.CapturePhoto();
    The photo is taken just fine, just no flash on any of the three flashmode setting.
     
  5. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Can you email me with your invoice number. I'll send you the beta build for the next update. It has a fix for flash not firing on some Android devices.
     
  6. DangerosoDavo

    DangerosoDavo

    Joined:
    Jan 18, 2014
    Posts:
    2
    done, thanks
     
    Lanre likes this.
  7. Denchi_

    Denchi_

    Joined:
    Dec 17, 2018
    Posts:
    23
    Hi,

    quick question: I'm looking for something that will let me reliably enumerate webcams and their supported resolutions on Windows and MacOS (including M1 Macs). Is that something NatDevice can do? I've found info on iOS/Android camera support, but not regular webcams.

    Thanks,
    Denchi
     
  8. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Hey there. NatDevice doesn't have native camera support on standalone platforms. It only supports the full featureset on iOS and Android. Also, NatDevice doesn't allow you to enumerate the supported resolutions on any platform; instead you request a preferred value, and the device will find a configuration that is closest to what you requested.
     
    Denchi_ likes this.
  9. Denchi_

    Denchi_

    Joined:
    Dec 17, 2018
    Posts:
    23
    Got it, thanks for the reply.
     
    Lanre likes this.
  10. xeye

    xeye

    Joined:
    Jun 25, 2013
    Posts:
    37
    Our project started with NatCorder awhile ago and we switched to Next Gen because of some resolution issues. Long story short still using Next Gen for the video capture but it's giving major headaches with the microphone, so I grabbed NatDevice to try and see if it would make my life easier in that regard. I am basically a noob setting up the mic stuff in general and having major issues getting started with just this element in NatDevice. Can anyone point me in some "starting/stopping the mic 101" directions for what to do? I'll keep searching for tutorials in the meantime, sadly all the NatMic stuff (previous version of the software) that I've been looking at doesn't seem compatible with what I am trying? Thanks for reading-
     
  11. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    Hi @Lanre. What is the situation regarding the source code? Do you provide it? I am interested specifically in the Java source code (no C++).
     
  12. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Have you checked out the NatDevice docs? I don't know if Next Gen Recorder accepts sample buffers, but if not you can reach out to @pmjo for assistance on it.
     
    xeye likes this.
  13. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Not anymore. I only provide the source code with a paid license that has terms preventing the licensor from reproducing and selling the code. That license is much more expensive than buying the API and using it as-is. For just about all cases, you shouldn't need the source code.
     
  14. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    How would you make this compatible with MLKit PoseDetector? I think you cannot use the camera simultaneously from two plugins so you either need the source code or a feature that would allow you to expose the YUV camera feed on a per-frame basis.

    I'm buying a NatDevice copy if it is directly or indirectly supporting PoseDetector.
     
  15. Resonantmango

    Resonantmango

    Joined:
    May 28, 2017
    Posts:
    7
    Any tips for implementing this? I have tried (naively) to just pass the
    sampleBuffer 
    into this tool, with no luck.
     
  16. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    NatDevice (along with all of the NatSuite Framework) uses RGBA32 for pixel buffer layouts. So the easiest thing to do, as far as I can tell, is to convert the pixel buffer to a Bitmap then use the bitmap with MLKit. Here is some example code.

    On a related note, I've been working on a machine learning library for Unity, NatML. It integrates tightly with NatDevice for high performance computer vision, and unlike Unity Barracuda, it takes advantage of the latest hardware technology (like the neural engine on iOS) and is insanely easy to use. It's very much under development, and I haven't implemented a pose detection model yet but with your use case I can prioritize it (I'm currently working on object detection). If you're interested in becoming a beta tester, fill out this survey.
     
  17. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Can you share your code? I might be able to help smoothen things out.
     
  18. Resonantmango

    Resonantmango

    Joined:
    May 28, 2017
    Posts:
    7
    I was able to get it working by using the FFT implementation in keijiro's LASP (I broke its dependence on libsoundio - which is desktop only - and pass in the sampleBuffer from NatDevice). Works well!

    Another question - Mic latency is very good on an Android build, but is noticeable on iOS.. Any tips for reducing this latency? I have 'Prepare iOS for Recording' checked in Player Settings, the sample rate is the default 44100 Hz... not sure what else I could check. App is running at 60 fps with only a few ms per frame, the rest is taken up by Vsync, so my frame budget is fine.

    Edit: What determines the length of NatDevice's sampleBuffer? Could that explain it?
     
  19. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    That's great. I might create an open source example project to demonstrate this. I'd appreciate any pointers you might have.
    This has been fixed in the upcoming 1.1.0 version. Email me with your invoice number and I'll share the current beta build.
    On Android, NatDevice uses the smallest buffer size configurable for the current audio format. On iOS, macOS, and Windows, NatDevice doesn't explicitly request any buffer size; it's up to the native media API's to provide whatever they choose. On iOS, you might be able to request smaller buffer sizes, but you'd run the risk of messing with Unity's audio engine.
     
  20. CangoDev

    CangoDev

    Joined:
    Aug 27, 2018
    Posts:
    6
    Hi, i've just noticed that app crashes on samsung s10 when i call device.StopRunning();
     
  21. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Hey there I saw your email. I'll need to see the full (unfiltered) logs from logcat.
     
  22. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    Thank you for the response @Lanre. This is make perfect sense but is less efficient than accessing YUV directly. Do you plan do add any way to plug into the YUV source? This will greatly improve the value of NatDevice.
     
  23. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    NatDevice doesn't expose YUV, and only actually encounters YUV data on Android (on other platforms, we don't have YUV data to begin with). This will not be changing in the future.
     
  24. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    @Lanre
    Do you think there is a faster way of sendings sbytes to Java other than calling functions on
    AndroidJavaObject
    s? It is insanely slow for a 1920x1080 image (100 milliseconds).
     
  25. musicdeveloper

    musicdeveloper

    Joined:
    Oct 16, 2019
    Posts:
    67
    Hello Yusuf,
    I'm getting the following error in Terminal from NatDevice.bundle when trying to sign my MacOS app created in Unity 2020.3.4f1:
    Code (CSharp):
    1. Contents/Plugins/NatDevice.bundle: replacing existing signature
    2. /Library/Developer/CommandLineTools/usr/bin/codesign_allocate: fatal error: file not in an order that can be processed (code signature data out of place)
    - The issue seems to be directly related to upgrading from Unity 2019.4.24f1 to Unity 2020.3.4f1.

    - The codesigning commands I'm running (and have run successfully for the last two years are):
    Code (CSharp):
    1. codesign -f --deep -s "3rd Party Mac Developer Application: [Developer Name]" [App].app/Contents/Plugins/NatDevice.bundle
    2.  
    3. codesign -f --deep -s '3rd Party Mac Developer Application: [Developer Name]' --entitlements "[App].entitlements" "[App].app"
    4.  
    It seems that this issue has affected other plugins as well - but I believe they eventually posted an update to their framework to fix the issue on MacOS:
    https://forum.unity.com/threads/file-browser-native-file-browser.510403/page-7

    I tried the PostProcessBuild solution suggested on that page for NatDevice. I am then able to sign the .app successfully, but when I try to upload to App Store Connect, I get the error:
    ERROR ITMS-90135: "The executable could not be re-signed for submission to the App Store. The app may have been built or signed with non-compliant or pre-release tools."


    Strangely, I am also using NatCorder.bundle in the app, and it isn't causing any issues at all.

    Any help would be much appreciated!

    Thank you

    Update - I was able to eventually get a build submitted to Apple by building an Xcode project, and creating the build in Xcode - very time consuming and difficult process, but it does work for now. If you do find some change that can be made to the framework to make it work better with the MacOS builds and codesigning, that would be still be great.
     
    Last edited: Apr 22, 2021
  26. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Sending data to Java shouldn't take that long. Maybe all the time is being spent in your Java code? There isn't a 'faster' way per se.
     
  27. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Thanks for the very detailed explanation. You're the first person who has reported code signing issues, and it looks like Unity has a fix in Unity 2020.3. I'll see if there's anything I can do for the time being.
     
  28. musicdeveloper

    musicdeveloper

    Joined:
    Oct 16, 2019
    Posts:
    67
    Yes, I think the recent Unity update fixed this... I actually had to downgrade back to 2019.4.25f1 today because of issues with 2020.3 and VideoPlayer bugs - so I can only speak to 2019.4.25f1, but I think it is fixed there. I imagine they applied the same fix to 2020.3, and this issue is likely fixed now.
    What a nightmare MacOS codesigning is.
    Thank you for considering the issue!
     
    Lanre likes this.
  29. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    @Lanre Yes indeed it was my mistake in my profiling. It takes ~ 7 milliseconds if profiled correctly. The only thing I need for full happiness is access to the source code or the ability to get the orientation of camera in Android and iOS. Do you think you can add this feature? If yes, how long would it take?

    The use case is getting
    sensorOrientation
    as in this code (for using MLKit, https://developers.google.com/ml-kit/vision/pose-detection/android):

    Code (CSharp):
    1. private int getRotationCompensation(String cameraId, Activity activity, boolean isFrontFacing)
    2.         throws CameraAccessException {
    3.     // Get the device's current rotation relative to its "native" orientation.
    4.     // Then, from the ORIENTATIONS table, look up the angle the image must be
    5.     // rotated to compensate for the device's rotation.
    6.     int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    7.     int rotationCompensation = ORIENTATIONS.get(deviceRotation);
    8.  
    9.     // Get the device's sensor orientation.
    10.     CameraManager cameraManager = (CameraManager) activity.getSystemService(CAMERA_SERVICE);
    11.     int sensorOrientation = cameraManager
    12.             .getCameraCharacteristics(cameraId)
    13.             .get(CameraCharacteristics.SENSOR_ORIENTATION);
    14.  
    15.     if (isFrontFacing) {
    16.         rotationCompensation = (sensorOrientation + rotationCompensation) % 360;
    17.     } else { // back-facing
    18.         rotationCompensation = (sensorOrientation - rotationCompensation + 360) % 360;
    19.     }
    20.     return rotationCompensation;
    21. }
     
    Last edited: Apr 23, 2021
  30. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    I don't provide the NatDevice source code. If I am to provide you access, it will be under a license that is sold separately, and is much (much) more expensive. As for running the pose detection model, you don't need any orientation with NatDevice because it already guarantees that all frames are upright.
     
  31. Virtumonde

    Virtumonde

    Joined:
    Apr 15, 2020
    Posts:
    84
    @Lanre what do you mean by "all frames are upright"? My guess is MLKit uses it internally to aid in pose detection. Is this unreasonable to suppose?
     
  32. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    The reason why the MLKit example includes orientation code is because the native camera stream is not upright. So MLKit expects you to provide the correct orientation, which it will use to first make the frame upright, before performing inference. NatDevice fully removes the need for this, because it guarantees that the camera data you receive is already upright. So you can pass in zero orientation, meaning MLKit can skip the orientation pre-processing step.
     
  33. ProjectileVomitTV

    ProjectileVomitTV

    Joined:
    Aug 12, 2020
    Posts:
    30
    Hi, I just bought NatDevice yesterday. I'm really struggling with the documentation, specifically for AudioDevice. Everything online seems geared towards recording. I just want to bring microphone audio in-game and hear it in-game, preferably through an AudioSource so it can be spatialised. Do you have any sample code for this?
     
  34. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Hey there. The HotMic example illustrates recording to an audio clip and playing it back. There isn't an example for realtime streaming (i.e. playing back the microphone audio in Unity in realtime), primarily because this is not a common use case. One problem you will immediately run into is audio feedback, where audio picked up by the microphone gets played back by the speaker, then picked up by the microphone and on and on.
     
  35. ProjectileVomitTV

    ProjectileVomitTV

    Joined:
    Aug 12, 2020
    Posts:
    30
    I had a look at the HotMic example, but it's specifically only realtime audio I need. I'm aware of feedback issues as I'm a musician and audio engineer, and so know how to get around that with mic placement and gain structure, etc. So is it possible to get realtime playback? And if so how?
     
  36. Kenchan1993

    Kenchan1993

    Joined:
    Apr 9, 2021
    Posts:
    3
    my Latest version 1.0.2

    FlashMode is not work , how can fix it ?
    Code (CSharp):
    1. async void settings()
    2.   {
    3.     Debug.Log("settings");
    4.     // Find a front-facing camera
    5.     var criterion = MediaDeviceQuery.Criteria.GenericCameraDevice;
    6.     query = new MediaDeviceQuery(criterion);
    7.     camera_settings.device = query.currentDevice as ICameraDevice;
    8.     flashIcon.color = camera_settings.device is CameraDevice cameraDevice && cameraDevice.flashSupported ? Color.white : Color.gray;
    9.  
    10.     if (camera_settings.device != null)
    11.     {
    12.       // Start the camera preview
    13.       previewTexture = await camera_settings.device.StartRunning();
    14.       // Create barcode reader
    15.       barcodeReader = new BarcodeReader();
    16.       DecodingOptions decoding = new DecodingOptions();
    17.       decoding.PossibleFormats = new List<BarcodeFormat>()
    18.         {
    19.           BarcodeFormat.QR_CODE,
    20.           BarcodeFormat.CODE_93,
    21.           BarcodeFormat.CODE_39
    22.         };
    23.       barcodeReader.Options = decoding;
    24.       pixels = previewTexture.GetPixels32();
    25.       // Display preview
    26.       cameraTexture.texture = previewTexture;
    27.       //aspectFitter.aspectRatio = (float)previewTexture.width / previewTexture.height;
    28.       StopButton.SetActive(false);
    29.       scanning = true;
    30.     }
    31.     else
    32.     {
    33.       Debug.Log("Error");
    34.     }
    35.   }
    Code (CSharp):
    1. public void ToggleFlashMode()
    2.   {
    3.     Debug.Log("ToggleFlashMode");
    4.     // Only `CameraDevice` supports setting focus point, not `ICameraDevice`
    5.     if (query.currentDevice is CameraDevice device)
    6.     {
    7.       device.flashMode = device.flashMode == FlashMode.On ? FlashMode.Off : FlashMode.On;
    8.       flashIcon.color = device.flashMode == FlashMode.On ? Color.white : Color.gray;
    9.     }else{
    10.       Debug.Log("NO work ToggleFlashMode");
    11.     }
    12.   }
     
    Last edited: Apr 28, 2021
  37. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Okay it's good to know that you're an audio engineer; this should come in handy in implementing realtime streaming. Wha you need to do is to create an AudioClip with the PCMReaderCallback. This callback is used by Unity to fill its sample buffer for playback. You'll need to buffer up the samples coming from NatDevice, then copy them into Unity's sample buffer when the PCMReaderCallback is invoked.

    You'll have to handle stuff like synchronization across threads and preventing buffer overruns, but it sounds like you have that experience anyway. I'm not an audio engineer so I'm not as familiar.
     
  38. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    This is the last outstanding issue in the upcoming update. I'll keep you posted.
     
  39. ProjectileVomitTV

    ProjectileVomitTV

    Joined:
    Aug 12, 2020
    Posts:
    30
    Ok cool, thanks! Can I ask another question? Since I'm making a standalone application, NatDevice cameras are just wrappers for WebCamTexture, right? So is it going to be more efficient just to use a WebCamTexture? Or is there an advantage to using NatDevice in this case?
     
  40. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    The overhead with using NatDevice's `WebCamDevice` as opposed to `WebCamTexture` is minimal, so to keep your code uniform it might not be worth the switching. Also the next update, 1.1.0, will add native camera support on macOS, so it's something else to consider.
     
    ProjectileVomitTV likes this.
  41. psyhama

    psyhama

    Joined:
    Sep 4, 2018
    Posts:
    24
    Hi, we're testing NatDevice (v1.0.2) on Android, and found a bit of a problem.

    We're using it like enabling and disabling camera frequently by calling StartRunning and StopRunning, and we found that this exception has been raised and the app crashed.

    05-06 13:28:15.260: E/Unity(13676): InvalidOperationException: An attempt was made to transition a task to a final state when it had already completed.
    05-06 13:28:15.260: E/Unity(13676): at System.Threading.Tasks.TaskCompletionSource`1[TResult].SetResult (TResult result) [0x00013] in <44afb4564e9347cf99a1865351ea8f4a>:0
    05-06 13:28:15.260: E/Unity(13676): at NatSuite.Devices.Internal.NativeCameraDevice+<>c__DisplayClass58_0.<StartRunning>b__0 (System.IntPtr pixelBuffer, System.Int32 width, System.Int32 height, System.Int64 timestamp) [0x0005f] in <6154a14878724e3eb0a02d63b37a3615>:0
    05-06 13:28:15.260: E/Unity(13676): at NatSuite.Devices.Internal.NativeCameraDevice.OnFrame (System.IntPtr context, System.IntPtr pixelBuffer, System.Int32 width, System.Int32 height, System.Int64 timestamp) [0x0000e] in <6154a14878724e3eb0a02d63b37a3615>:0
    05-06 13:28:15.260: E/Unity(13676): UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
    05-06 13:28:15.260: E/Unity(13676): UnityEngine.DebugLogHandler:LogException(Exception, Object)
    05-06 13:28:15.260: E/Unity(13676): UnityEngine.Logger:LogException(Exception, Object)
    05-06 13:28:15.260: E/Unity(13676): UnityEngine.Debug:LogException(Exception)
    05-06 13:28:15.260: E/Unity(13676): UnityEngine.<>c:<RegisterUECatcher>b__0_0(Object, UnhandledExceptionEventArgs)
    This modification below to the NativeCameraDevice.cs seems to be a solution. The original code checks if it is the first frame or not by previewTexture, but it can already be null by calling StopRunning().


    Code (CSharp):
    1.  
    2.         public override Task<Texture2D> StartRunning () {
    3.             var startTask = new TaskCompletionSource<Texture2D>();
    4.  
    5.             //===== add an explicit flag for first frame check =====
    6.             bool isFirstFrame = true;
    7.             Action<IntPtr, int, int, long> handler = (pixelBuffer, width, height, timestamp) => {
    8.                 // Update preview texture
    9.                 // remove this flag
    10.                 //var firstFrame = !previewTexture;
    11.                 previewTexture = previewTexture ?? new Texture2D(width, height, TextureFormat.RGBA32, false, false);
    12.                 previewTexture.LoadRawTextureData(pixelBuffer, width * height * 4);
    13.                 previewTexture.Apply();
    14.                 // Complete task
    15.  
    16.                 //===== use the first frame flag and clear the flag =====
    17.                 if (isFirstFrame)
    18.                 {
    19.                     isFirstFrame = false;
    20.                     startTask.SetResult(previewTexture);
    21.                 }
    22.             };
    23.             device.StartRunning(OnFrame, (IntPtr)GCHandle.Alloc(handler, GCHandleType.Normal));
    24.             return startTask.Task;
    25.         }
    I hope it helps.

    Regards,
    Makoto
     
  42. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Hey there, your usage is actually the cause of this issue. The call to `StartRunning` is not interruptible, so if you decide to stop the camera, you must only do so once the `StartRunning` task completes.
     
  43. aljubair707

    aljubair707

    Joined:
    May 6, 2021
    Posts:
    3
    Hello! when I capture a photo on my device with natcam, they don't save at all. How do I save photos on gallery app. Thanks
     
  44. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    Hey there, NatDevice does not automatically save photos anywhere. The photos are returned directly to you. You can use NatShare to save images to the camera roll.
     
  45. psyhama

    psyhama

    Joined:
    Sep 4, 2018
    Posts:
    24
    Hi, thank you for your reply.

    I firstly guess so but it's a bit strange because I execute StartRunning and StopRunning sequentially, on one UniTask loop.

    So I took some log. It seems that the 'handler' part of StartRunning is executed periodically after StartRunning() finished.
    It look like that when I call StopRunning() after StartRunning() finished, then the handler of previous run of StartRunning() executed and it thinks that it is the first run (from previewTexture) and executes SetResult() and it crashes.

    Though, what's still strange is that sometimes calling StopRunning() is ok, and this time it is not.

    ```
    Code (CSharp):
    1.  
    2. 05-07 14:02:07.233: I/Unity(10481): NatCameraDevice StartRunning() called.
    3. 05-07 14:02:08.341: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  firstFrame: True
    4. 05-07 14:02:08.359: I/Unity(10481): NatCameraDevice StartRunning() SetResult.
    5. 05-07 14:02:08.372: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    6. 05-07 14:02:08.380: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    7. 05-07 14:02:08.403: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    8. 05-07 14:02:08.436: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    9.  
    10. ...
    11.  
    12. 05-07 14:02:12.255: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    13. 05-07 14:02:12.371: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    14. 05-07 14:02:12.419: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    15. 05-07 14:02:12.451: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    16. 05-07 14:02:12.512: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    17. 05-07 14:02:12.514: I/Unity(10481): NatCameraDevice StopRunning() called.
    18.  
    19. 05-07 14:02:20.651: I/Unity(10481): NatCameraDevice StartRunning() called.
    20. 05-07 14:02:21.474: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  firstFrame: True
    21. 05-07 14:02:21.483: I/Unity(10481): NatCameraDevice StartRunning() SetResult.
    22. 05-07 14:02:21.516: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    23. 05-07 14:02:21.538: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    24. 05-07 14:02:21.574: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    25. 05-07 14:02:21.619: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    26.  
    27. .....
    28. 05-07 14:02:26.051: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    29. 05-07 14:02:26.090: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    30. 05-07 14:02:26.120: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    31. 05-07 14:02:26.153: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    32. 05-07 14:02:26.203: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  (UnityEngine.Texture2D) firstFrame: False
    33. 05-07 14:02:26.217: I/Unity(10481): NatCameraDevice StopRunning() called.
    34. 05-07 14:02:26.425: I/Unity(10481): NatCameraDevice StartRunning() handler called. previewTexture:  firstFrame: True <------ ****** firstFrame become true without calling StartRunning().  ******
    35. 05-07 14:02:26.429: I/Unity(10481): NatCameraDevice StartRunning() SetResult.
    36.  
    37. <the app crashed here>
    38.  
    39.  
     
  46. aljubair707

    aljubair707

    Joined:
    May 6, 2021
    Posts:
    3
    Um, I got the natshare api. But the thing is how'd you connect it with your project. Give me some heads up please. And thanks for the answer.
     
  47. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    I'm not familiar with UniTask. Here is one of the NatDevice test cases:
    Code (CSharp):
    1. async void Start () {
    2.     // Get device
    3.     var query = new MediaDeviceQuery(MediaDeviceCriteria.CameraDevice);
    4.     device = query.current as CameraDevice;
    5.     // Run
    6.     for (var i = 0; i < 10; i++) {
    7.         await device.StartRunning();
    8.         device.StopRunning();
    9.     }
    10. }
    Can you share your own test code?
     
  48. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    See the NatShare readme.
     
  49. aljubair707

    aljubair707

    Joined:
    May 6, 2021
    Posts:
    3

    Argument 3: cannot convert from 'method group' to 'ScreenOrientation'

    The type or namespace name 'DeviceCamera' could not be found (are you missing a using directive or an assembly reference?)



    hello! what does this mean? I was adding tap capture method. If I add deviceCamera, I get the 2nd error Instead of the 1st one.
     
    Last edited: May 8, 2021
  50. Lanre

    Lanre

    Joined:
    Dec 26, 2013
    Posts:
    3,962
    I'll need to see your code in order to be of any help. Also, `DeviceCamera` is from a much older version of NatCam. Upgrade to the latest NatDevice on the Asset Store.