NatDevice - Media Device API

What device does this occur on? And what is the framerate of your application?

 

Samsung A5 and Samsung Galaxy S8+, and we run at 30 fps, thanks for the quick answer! Also, there seems to be some delay on CapturePhoto. Meaning i still have to hold the camera still for a short while after pressing the button. Any way to fix this?

Thanks!

 

Reducing the preview resolution should help with this. Maybe 1280x720.

Only way to improve this is to reduce the photo resolution. On Android, the camera HAL only provides frames in the YUV format. Also, frames are always in the 'natural orientation' of the camera. So NatCam has to convert to RGBA and reorient the image, which takes longer for larger images.

 

Hi Natcam & everyone,

Great app! We've used it for a long time. We were making updates to an application and noticed we were still using v1.3 and decided to update to the latest version 2.1f2 . So we deleted the previous library folder, and downloaded the newest one into our project. There were many changes made to the API since 1.3 so we reimplemented the use of the library and it works well. However, when we try to build the Xcode project on the simulator, or even export the ipa for Release, it runs into this error every time.

WARNING:

-undefined dynamic_lookup is deprecated on iOS Simulator

ERROR:

error: Invalid bitcode signature

clang: error: linker command failed with exit code 1 (use -v to see invocation)

We've retraced our steps from deleting the old library to putting in the new one, and it seems even with commenting out all the Natcam code from the project and just including the Natcam plugin folder into the project triggers this bug.

TLDR; Only just including the newest library of Natcam in our project is causing the above error.

Any advice or guidance?

Thanks.

 

Hey there! We've had one other report of this issue. Can you email me your invoice number? I'll share a rebuilt library with you.

 

We faced the same and it was resolved by disabling arm7. I guess it has to do with the dropping of OpenGL support on iOS.

I sent a mail to dev if there would be a workaround we missed or it would mean no support for iPhone 5 and iPad 4th gen.

 

I should probably divulge some insider information. Technically, we deprecated support for OpenGL ES on iOS. But in reality, nothing actually changed. The OpenGL ES deprecation on iOS happened in the NatRender support library...but the library isn't used on iOS. NatCam only uses NatRender on Android. I also suspect that the bitcode issue came from NatRender since we didn't explicitly set the bitcode generation marker when compiling. I've resolved that. If you'd like to get your hands on it ASAP, email me your invoice number.

 

Makes a very big difference for android!

When will the official example be updated to 2.1?

 

Hello,

We are using Natcam Core - Webcam API 1.6f2 and we have 2 issues.

-On android there is a significant delay between calling NatCam.CapturePhoto and the execution of the callback. During this delay the live camera input is still displayed. Is there a way to speed this up? Or should we disable live input manually?

-Is there a fast/easy way to set the exposure bias? For now we take a picture and get the value from that picture, but this is not ideal as it will use flash automatically. We tried getting the value from the live input texture and this worked by casting it to a WebcamTexture in editor, but on mobile the type is a Texture2D and casting it to Texture2D results in a texture with no data.

Updating the plugin is not an option. We are at the highest update possible for the Unity version we are using and because of other plugins, we can't update the Unity version. (This is a pretty old project at this point)

 

I'm almost done with upgrading the example to 2.1. Once I complete the Pull Request, it'll be up to EnoxSoftware (dev of OpenCV) to review it and merge it. I'll keep you posted

 

This process is much faster on NatCam 2.1f2. I recommend upgrading.

Oh, it seems like you are trying to estimate the current exposure before setting a new value (is this correct)? Because setting exposure bias is as easy as setting the DeviceCamera.ExposureBias property. If my guess is right, then you'll need to do a ReadPixels on the preview texture to get the texture data in system memory then look through that data. This is another place where you'll benefit greatly from using NatCam 2.1. There is now a CaptureFrame function that gives you access to the preview data in a byte[]. And it takes practically 0ms (since NatCam already keeps the data), unlike ReadPixels which has to synchronize the GPU before performing the readback.

I understand. I still do recommend exploring an update; Unity and NatCam are only getting more performant over time.

 

This updated library he sent us worked! Solved the issue completely. Thanks!

 

Great!

 

Thank you for your answer, i bought your asset...you did a good job with Natcam....iam happy

 

I've been using NatCam on the Note 8 and it works perfectly. I've been requested to support the Lenovo Yoga 3 but the framerate is very choppy and crashes the device often. If I use WebCamTexture on the Lenovo I don't get these issues but I'm left without the ability to control exposure. Is there a workaround for something like this? Thanks.

Preview resolution is 1280x720, 30fps.

 

The device runs Windows RT right? On this platform, NatCam falls back to WebCamTexture so it should work all the same. Can you upload the crash logs in a .txt file?

 

Thanks for the reply. I've attached a crash log. The device is a Lenovo YT3-850F running Android 6.0.1.
I've also uploaded a video testing the camera framerate:

Is there a way to force it to use webcamtexture but still retain the exposure and focus controls?

 

Thanks for the video. From what I can tell, the Lenovo tablet doesn't accelerate conversion of the preview data from YUV to RGBA, which is why the framerate is so low. The workaround is to use a lower preview resolution. There is no way to use WebCamTexture and NatCam at the same time. As for the crash, it should be implicitly fixed in the next update.

 

Any update with this? Thanks.

 

I've submitted a PR. Waiting for EnoxSoftware to approve.

 

Thanks, I've reduced the preview resolution down to 380 x 240 and the photo resolution set as 1920 x 1080.
As expected the preview refresh rate is a lot faster, however, when I try to take a photo using "NatCam.CapturePhoto(OnPhoto);" the texture resolution that comes out of "OnPhoto(Texture2D photo)" is only 380 x 240. Is this how it should function, I had assumed it should output at the specified photo resolution of 1920 x 1080.

 

Hello Lanre, i am quite happy with your asset, the only problem is, when i try to implement a zoom slider, my app crashes on ipad (model: mtxn2fd/a) .

i added to your minicam script in OnStart()

zoomSlide.minValue = 1f; // the type of zoomSlide is Slider
zoomSlide.maxValue = NatCam.Camera.MaxZoomRatio;

and i added to Update()
if (NatCam.IsRunning)
{

NatCam.Camera.ZoomRatio = zoomSlide.value;

}

without these lines everything runs fine, but with this lines it always crashes. thanks in advance.

EDIT: when i set zoomSlide.maxValue in OnStart() to 2.0f without using NatCam.Camera.MaxZoomRatio everything works fine. Looks like NatCam.Camera.MaxZoomRatio returns a value wich causes the app to crash.

 

This depends entirely on the device and the camera being used. So even though you might request a preview or photo resolution, it is up to the device camera to decide what photo resolution actually gets used (cameras have a fixed set of supported preview and photo resolutions). You can see this by checking what the value of camera.PhotoResolution is immediately after setting it.

 

Can you upload the logs from Xcode in a .txt file? That's weird.

 

Is the deprecated `NatCam.Camera.AutofocusEnabled = true;` the same as `NatCam.Camera.FocusLock = false`?

Or what is the recommended migration strategy?

 

Yup. Autofocus is naturally enabled, so is autoexposure and auto white balance. So instead, we inverted the control. So no need to explicitly set FocusLock to false when starting the camera.

 

Okay thanks!

 

I just updated the NatCamWithOpenCVForUnityExample project to 1.0.3 (Update to NatCam 2.1f3).
https://github.com/EnoxSoftware/NatCamWithOpenCVForUnityExample

 

Hello Lanre,

Does NatCam support Vulkan? It seems only works with OpenGL Renderer.

We need Vulkan for compute shaders, is there any options to support it ?

Thanks

[Edit]

Never mind, seems my test device doesn't support Vulkan..

 

NatCam doesn't support Vulkan. Android itself doesn't fully support it.

 

Hello Lanre,
I send you a mail with some Logs.

I'm using the lastest natCam 2.1.3 and my OnePlus5 on the MiniCam exemple.

Portrait mode is working but when doing landscape-right it's crashing the app and when doing landscape left it's doing some crazy pixel art :


Are you aware of this really annoying bug ?

Thanks,
Seb

 

I have a very similar issue when working in Landscape. I believe in my case that there is a mismatch between the photo resolution and the preview resolution. I've tried to set my camera up like this:

Code (CSharp):

1. var camera = useFrontCamera ? DeviceCamera.FrontCamera : DeviceCamera.RearCamera;
2.  
3. camera.PreviewResolution = new Vector2Int(1920, 1080);
4. camera.PhotoResolution = new Vector2Int(1920, 1080);
5. camera.Framerate = 60;
6.  
7. NatCam.StartPreview(camera, OnStart);
8.  

But for some reason it does not stick, as my logging later reveals that it is set to:
Changed camera format for photo resolution: (3264, 1836)
Changed camera 0 framerate to 30.000000

Device:
iPad Air 2 - ios 11 - tested with the latest version of Natcam

Am I doing it wrong?

 

Same problem as @roadley mentioned - can't change resolution.
Furthermore, testing on iPhone7 and using memory profiler in XCode I have found that on Preview setup memory raised to +100mb, and each photo taken raised to another +100mb... After couple of shots I get app crash.

What is wrong?

 

So I managed to change the resolution now, by lowering it to 1280x720. I am now trying a new approach, where I will scale the full resolution photo down to 1920x1080 using a different method.

 

@roadley how did you change the resolution? How to get, what camera resolutions are suitable for our case?

 

So that is what I am unsure about, I simply tried to lower the resolution, and noticed it changing once I put it low enough. As if it is expecting certain resolutions, but I am not sure how to expose those for each camera.

 

Hey Seb, sorry for the late response. I'm aware of this issue (from here). I haven't gotten a chance to investigate since it only seems to happen on some devices, but I have a few ideas. I'll shoot you an email with more questions pretty soon.

 

This is expected behaviour on iOS. On iOS, the camera is configured with a format, of which there are a preset number to choose from (you can't create your own format). Each format has exactly one preview size and a few photo sizes to choose from (as few as one). As a result, when you set the preview resolution on a device camera, NatCam is really just picking the camera format with the closest preview resolution to what you asked for. On the other hand, when you set the photo resolution on a device camera, NatCam has to choose the closest matching photo resolution from those on the current format (if not, changing the photo resolution would inadvertently change the preview resolution). This is why your photo resolution setting doesn't seem to be setting.

 

On iOS 12, apps have been reported to use a lot more memory. What's happening is that the profiler is exposing kernel-space resources (created in some part as a result of the current app) as the app's memory--something that wasn't done previously. As for photos, it sounds like you have a memory leak. You must explicitly destroy photo textures (Texture2D.Destroy); they don't get automatically cleaned up when they are dereferenced in C# code.

 

So, what is the exact way to lower down default resolution? I tried different approaches (setting up smaller resolution before camera start, changing it after initialisation etc) but nothing works

 

This all depends on the device and camera. If you really need the photo to be at some resolution, you have to manually resize the photo texture.

 

I think I am getting a similar issue with regards to photo resolution. I'm setting our photo resoulation to deviceCamera.PhotoResolution = new Vector2Int(9000,9000) to make sure we get the highest resolution possible, which when I Debug.Log(deviceCamera.PhotoResolution) I am getting 3264x2448 which is what we are looking for. When I use NatCam.CapturePhoto() I'm getting a resolution of 1280x720 (which is our preview resolution) from the texture received in the callback function. Is there something I am missing to being able to get a 3264x2448 resolution photo?

Relevant Code:

Code (CSharp):

1.  void Start()
2.     {
3.         var camera = useFrontCamera ? DeviceCamera.FrontCamera : DeviceCamera.RearCamera;
4.         if (!camera)
5.         {
6.             Debug.LogError("Camera is null. Consider using " + (useFrontCamera ? "rear" : "front") + " camera");
7.             return;
8.         }
9.         //camera.PreviewResolution = new Vector2Int(4000, 4000);
10.    
11.         camera.PhotoResolution = new Vector2Int(4000, 4000);
12.    
13.         NatCam.StartPreview(camera, OnStart);
14.         Debug.Log("Photo res: " + camera.PhotoResolution.x + " " + camera.PhotoResolution.y);
15.  
16.     }

Code (CSharp):

1.  
2. void getPhoto()
3.     {
4.         waitingForCallback = true;
5.         NatCam.CapturePhoto(returnTexture);
6.     }
7.     bool waitingForCallback;
8.     void returnTexture(Texture2D photo)
9.     {
10.         waitingForCallback = false;
11.         Texture2D myTexture = photo;
12.         Debug.Log("Recieved photo res: " + photo.height + " x " + photo.width);
13.     }

 

Apparently, this is a bug on Android. I'm looking into it.

 

Hello Lanre, is this bug close to be fixed ?

 

Yes I'm working on an update that should implicitly fix this. I'm trying to have a build by tomorrow or Thursday.

 

Just picked this up today and ran into a couple of issues. I'm running a galaxy s7 edge and experiencing the orientation bug. Other then that the flash doesn't seem to work when taking pictures. Adding

Code (CSharp):

1. if (camera.IsTorchSupported) {
2.         // Put on the torch
3. camera.TorchEnabled = true;
4. }
5.  

also doesn't cause the flash to activate. It does seem to cause my preview to become super blurry for some reason.
When running the app, the toolbar I can pull down from the top of the screen does show that the flash is being controlled by the app so there is some level of communication going on. I can't post more info then that atm, but if you could redirect me to some instructions on submitting diagnostic data I could do that sometime tomorrow.

 

This is a known issue with camera2. Once I get the orientation stuff figured out, I'll get back to this.

 

Hey I know you are very busy but just one little question
With Camera2 do you think it is possible to record at the same time the front and the back camera ?

Have a nice day

 

Even though the camera2 API is advertised as such, most devices fall back to a legacy implementation (think the original camera API), so this is not supported.

 

Hello, I am a user who is using NatCam.

NatCam is used to create applications on IOS-based platforms.

I am using Play (), pause () to play NatCamios as needed.

Each time you use Play and Pause alternately, about 14 to 24 memories are accumulated.

Can you give me the answer to this?

thank you.