NatDevice - Media Device API

Hi, I add natcam plugin to my project that integrated vuforia SDK,the arscene was crashed when run at ipad2018(6th)

 

don't worry, it's more important to me if you solve the preview issue

 

NatCam can't run at the same time as Vuforia because only one service can gain exclusive access to use the hardware camera.

 

I don't know these until you tell me translation,thank you

 

NatCam and Vuforia can't run at the same time because only one service can use the camera at any point in time. The same restriction goes for WebCamTexture and Vuforia, or WebCamTexture and NatCam.

 

Hi, did you solve it? Because i have a very similar problem and I am struggling to solve it

 

Make sure you are running with the latest version of NatCam (2.2).

 

We are experiencing a problem with zoomRatio when running our application on an iPhone XR. The max zoomRatio for an iPhone XR reads as 189 which is a lot larger than most other devices we have tested on. It seems like the zoomRatio isn't using the entire range of 1-189. Is this intended behavior for the iPhone XR?

Thanks

 

This is weird. Usually, the zoom ratio is anywhere from 4 to 8. I'm not sure what you mean by the zoom ratio not using the entire range; NatCam sets the value passed in verbatim on the AVCaptureDevice.

 

I don't currently have the iPhone XR at the office to test it, but it seemed like it would get to full zoom before reaching the maxRatio of 189. Do you have any ideas on how we could bandaid the issue before the next NatCam update? The last update fixed the other issues we were having. We're using the zoomRatio to do distance calculations in OpenCV and need a workable scale from 4 to 8

 

Unfortunately there isn't anything we can do to resolve this. I guess a safe option is to consider the max zoom ratio as `minimum(DeviceCamera.MaxZoomRatio, 8)` (you can switch out 8 for any number that works for you. This looks like a bug in the XR.

 

Thank you for releasing the latest version of NatCam (2.2.1).
The known Android platform bug seems to be revised definitely as far as I tried it.

However, I have encountered another problem.
The latest version of NatCam (2.2.1) seems to have the same problem as Unity's WebCamTexture API.
This is a problem of the low illumination of a front camera occurring only with some Android devices.
https://forum.unity.com/threads/android-webcamtexture-in-low-light-only-some-models.520656/
https://forum.unity.com/threads/released-opencv-for-unity.277080/page-33#post-3445178
https://answers.unity.com/questions/1425938/android-native-camera-low-lightlow-iso.html
This problem was not present in previous versions.

I am in trouble as I have been using NatCam as an alternative to this problem.

 

I think I know why this is happening. Email me tomorrow for an updated build.

 

the feed is very slow and tearing on brand new phone samsung s10+
while using webcamtexture from unity works very smooth.
I tried to reduce the preview texture alot, but that doesn't make much difference. is there any way I can optimise this so i get a smooth preview?

deviceCamera.PreviewResolution = new Vector2Int(1520, 720);

I have also tried half of that resolution still not very smooth considering very good processor in a brand new phone, so I can rule out performance issues

deviceCamera.PhotoResolution = new Vector2Int(4608, 3456);

thanks

 

Can you email me with the full logs from logcat? Thanks.

 

Hello @Lanre , with the latest Unity support for Android 64-bit architecture and the Google Play Store requirement for such architecture from August 2019, is the NatCam native Android libraries supporting 64-bit? And what about NatCorder?

 

With the latest versions of our API's, everything is written in Java. So no native (C/C++) libs, and no need for 32- or 64-bit compilation.

 

So latest NatCam cannot run in the same project as Vuforia? In 2017 you said you can release the NatCam before initiating Vuforia. This no longer the case?

 

You can have them in the same project, but they can't run at the same time. The equivalent of 'releasing' NatCam now is to simply make sure that all DeviceCamera.Cameras are not running.

 

I'm still struggling a little on iOS regarding the best NatCam setup for augmented reality. Basically the focus and exposure are constantly adjusting such that you lose the AR feel of the app. Do you have a suggestion regarding a good implementation for focus and exposure?

For example, I thought about implementing a custom "tap to expose/focus" method where by the ExposureMode and FocusMode will be set to AutoFocus and AutoExpose for a brief second or two, just enough to allow the focus and exposure to take place, and then back to off/locked. Either that or a coroutine that sleeps for 10 seconds, then sets focus/exposure on then off again. Does this seem like it would work? Any idea how long would I need to wait for the focus/exposer to take place?

Thanks!

Edit: The problem with leaving Exposure locked is that the AR screen is sometimes too dark or overexposed depending on how the environment is when the user starts the app.

Thanks Lanre, any ideas would be appreciated.

 

Hi Lanre, i have the same issue with EnoxSoftware about low light on Oppo F3S, please help me about it

 

If you want to maintain parity with the Human Visual System, then you should technically allow autoexposure. The eyes perform continuous AE. Focus is more of an issue, because NatCam doesn't support setting infinite focus.

This is a more reliable way to do it. NatCam doesn't provide callbacks for when AF or AE routines complete, so you shouldn't assume a specific time. A coroutine sounds more ideal.

 

The next update will fix this.

 

We purchased the asset with great expectation, it works great in Redmi4 where as the camera preview is lagging in Samsung S9+ and Note5. Tried building in unity by changing different settings like OpenGLES2, OpenGLES3, AutoGraphics enabled, Mono & IL2CPP etc. but results are same.

Screen recording : https://drive.google.com/open?id=1BKvPKgjDu1UN7xbKtexKOqPOcEHjkU3r

 

As the user above, we just purchased NatCam but the performance is very poor on both Samsung S6 and Samsung S7, works fine on iPhone 7.
Issues are exactly the same as in the screen recording shown by atul109.

 

Hi Lanre,
thanks for the great work in the latest update of NatCam.
we have also noticed that the framerate has dropped considerably, even by setting low preview resolutions.

Can you provide suggestions on how to improve NatCam performance?

Have a nice day

 

Hi Lanre,
on my device (Nexus 6P) flash doesn't work on latest version 2.2.1. When the camera is requested flash turns on for a half second then turn off and when taking a photo it doesn't turn on.

Also, the front face camera is taking rotated photos, but on the preview is fine. On previous versions, I handled that using callback's orientation parameter and then rotating the texture.

 

Hi Lanre,

Appreciate all the great work you've done with this app. We're developing an application with NatCam that utilizes the front camera on a Samsung GS7. Everything seems to work swimmingly, with the exception of autofocus. Autofocus technically *works* but it's a PDAF sensor, and it doesn't seem like phase focus works (it's a contrast-based sweep mode). However, replacing the camera2 API with Samsung's soon to be deprecated Camera SDK in a native Android app yields extremely fast focus performance by enabling PHASE_AF_MODE when building the capture request.

I'm wondering if there's a way to add Samsung's API support in place of camera2 API within NatCam to get the performance of PDAF in NatCam?

In addition to this, there is a lot of advanced camera customization available to Samsung phones with their SDK, including noise reduction, black level, lens correction, turning off OIS, etc. It would certainly be a dream to have control of these options!

 

How to set ( Android mobile camera focal length ) using natcam. How to fix same pixel value for different mobile and capture same person.

 

Hello!
I've noticed with the new updates that switching device cameras too often will cause a black screen or the app to crash. Even when called using a coroutine and timers in between. Any idea why this is happening? Thanks!

 

Hi, I bought NatCam a few days ago because I needed to get the HFOV of my front web camera.
I tested debugging the HFOV every frame in the Unity editor on my Mac laptop but always got a value of 0.

Here is how I get the HFOV.
Debug.Log("HFOV is: " + deviceCamera.HorizontalFOV);

Could you tell me why it's returning 0 instead of the actual HFOV?

 

Hi, tried the latest version of NatCam (2.2.1) in Unity 2019 (both 1.7 and 1.5). On some devices (tried a P8 lite, a P9 lite a P20 lite and a Samsung Galaxy A5) its super slow (like 4 FPS on average) on others (Galaxy S8, P30 Pro) it runs fine. Reverting back to and older version (2.1f2) it runs fine on all devices tested.

 

I responded to your email. Sorry for the delay.

We have an update, 2.3.0, that should improve performance on these devices. Shoot me an email with your invoice number and I'll share the build.

 

I'm aware of the flash issue. The camera2 API seems to have totally changed flash functionality. I'll look into it sometime soon.

I'll try to reproduce this and get back to you.

 

Not at all unfortunately. We have to build with the official API's and nothing else.

 

NatCam doesn't support setting the focus distance.

 

We've fixed state management issues like this in the next update, 2.3.0. Email me with your invoice number and I'll share the build with you.

 

On all platforms other than Android and iOS, NatCam will fall back to WebCamTexture (which does not provide this functionality). You should have gotten a warning in the console when trying to do this.

 

Hi, I have a problem with the camera being inverted. On Android, the front facing camera (in horizontal mode) is horizontally inverted. On IOS the camera is vertically inverted (basically up side down). Is there an option where I can make sure that it is always correct?

 

Email me with your invoice number and I'll share the 2.3.0 update that should implicitly fix these issues.

 

Hi, we have a problem setting the resolution on iOS. In contrast, on Android, the

CapturePhoto()

function works fine, and returns the photo with the specified resolution. But on iOS, regardless of what we set as PhotoResolution, even when set to the camera's native resolution, we always get an image of 1920x1080 pixels. This CS code in Unity:

Code (CSharp):

1. deviceCamera.PhotoResolution = new Vector2Int(3264, 2448);

works fine when compiled to Android but is simply disregarded on iOS and the resolution is always 1920x1080.

We specifically bought NatCam because it promised "NatCam allows you to set the camera's resolution." It's such a basic thing, I imagine we might be missing something. Is there something else we need to set beforehand or some other trick?

 

On iOS, you can't set the photo resolution independently of the preview resolution. Both are 'tied together' in what Apple calls 'formats'. Typically you can get the highest photo resolution by either setting the preview resolution to 1280x720 (set preview res before photo res), or by not setting a preview resolution at all.

 

Hi Lanre,

First of all thanks for your hard work. I've been using Natcam for a long time without issues, but I've started a new project using the latest Natcam on Asset Store, and the first time the camera is opened on Android, it asks for permissions and then when permissions are granted the screen remains black. Then I close the app, and open it again and it works fine. I attach an extract of the log.

Other thing is I should provide compatibility with Android 5.0 devices in my app, and as I can see now the minimum for NatCam is 6.0. Should I work with an older version? or is there anything else I can do?

Best regards

 

Hi Lanre!

I've got a question when you get a sec.

I'm trying to make a QR scanner using ZXing and NatCam, but can't seem to get the
BarcodeReader to ever return not null when trying to decode with the byte array I'm passing in from the deviceCamera's CaptureFrame function.

The code I'm using is fairly simple I think; here's the camera setup where I am starting the camera:

private byte[] previewData;
private int blockWidth;

_deviceCamera.PreviewResolution = new Vector2Int(720, 720);
_deviceCamera.StartPreview(OnStart);

public void OnStart(Texture preview) {
previewData = new byte[preview.width * preview.height * 4];
blockWidth = preview.width; // 720

rawImage.texture = preview;
rawImage.material.mainTexture = preview;
fitter.aspectRatio = preview.width / (float)preview.height;
}

And then here's where I'm attempting to decode the previewData, in Update:

if (_deviceCamera.IsRunning && !decoding) {
_deviceCamera.CaptureFrame(previewData);

Result data = barReader.Decode(previewData, blockWidth, blockWidth, RGBLuminanceSource.BitmapFormat.RGBA32
);
if (data != null)
{
Debug.Log("Data NOT NULL.");
decoding = true;
dataText = data.Text;
} else {
Debug.Log("Data NULL.");
}

I was able to get this working with the WebCamTexture but for whatever reason this always shows Data NULL when trying to scan our QR code. Not entirely sure what I'm doing wrong here or where to start debugging as there appears to be no errors thrown

If you happen to have any thoughts, I'd greatly appreciate it!

 

Hey, Im using the newest version of natcam and running it on an galaxy s8. It takes photos fine but the preview is black. It seems like some other people have this issue but i cant seem to find an answer to it anywhere on this forum.

 

Hi WorkingBrian,
I think it may be RGBLuminanceSource.BitmapFormat parameter in Decode function. Try with RGBLuminanceSource.BitmapFormat.Unknown, It will try to figure out in which format is the data.
I have something similar working like that.

Code (CSharp):

1. _camera.CaptureFrame(buffer);
2.        
3.  
4.         if (buffer != null)
5.         {
6.             BarcodeReader barcodeReader = new BarcodeReader { AutoRotate = false };
7.             barcodeReader.Options.TryHarder = true;
8.  
9.             Result res = barcodeReader.Decode(buffer, imgCamera.texture.width, imgCamera.texture.height, RGBLuminanceSource.BitmapFormat.Unknown);
10.  
11.             if (res != null )
12.             {
13.                 Debug.Log("QR Found -> " + res.Text);
14.                 OnQrRecognized(res.Text);
15.             }
16.         }

I hope it may help you, best regards.

 

Thanks so much, tonidurans! The problem seemed to be that my width and height were set to be equal, when I guess they need to be the exact width / height from the image. Either way, your example saved the day!!

 

Having some issues with ios. I tried version 2.3.0. My project is horizontal. The one horizontal rotation works, the other one shows a 90 degree wrong rotation. Is there an easy fix for that?
Also on IPad it seems to crash when using the rearcamera.

 

Hey there!

Pretty new at NatCam but I noticed that every few seconds, I get some duplicated frames. How can I fix that?
I am using the GrayCam example and not much else.
Since I using the camera to analyze the pixels, it is quite important for me that no duplicate frames are found. Is it a problem with the settings?

 

Thanks for the awesome package!

It does exactly what I need, with the big exception that I need to support android API level 22 (for Amazon 2017 edition Fire devices). I'm aware that the minimum API level is listed as 23. I'm wondering if there's any possible way to get NatCam working on level 22, either getting sent a legacy version, or disabling bits of functionality.