NatDevice - Media Device API

I see in comments to 'blameJane' that Natcam says its 'cannot run alongside Vuforia or ARToolkit' is this the case even is the Vuforia camera is not running? I am looking to use natCam instead of ios/AndroidNative do enable the user to pop 'out' of the app, take a photo and come back into the app again- though obviously I'd only be 'imitating' this if I used Natcam. But I can stop the Vuforia camera and restart it again, so that Natcam and Vuforia are not running concurrently. Would this work or no?

 

Hey Lanre,

Will 1.5f3 support portrait orientation? We're using 1.3 and we've never had good results in portrait. Thanks!

Cheers,

Paul

 

Yup, it'll fix this.

With the new unified Orientation enumeration that we'll be introducing in 1.5f3 and a new component that will help orient photos, this should be fixed.

No. But on 1.5b3, I expect to complete it between the end of next week and the upper week.

 

NatCam can run alongside Vuforia in the same app, just not simultaneously. The thing here is that when one API is done, it must release all its native resources (especially access to the camera) before the next API is used. With NatCam, we planned for this and provided the NatCam.Release() API.

On the Vuforia side of things, Vuforia does not offer an explicit Release() function. So you must either 'spoof' Vuforia by calling OnApplicationQuit/OnDisable/OnDestroy on the Vuforia singleton or use reflection to call the private methods inside the Vuforia assembly. MonoDevelop is very useful for the latter technique as it allows you to browse the functions included in the assembly.

 

Can you email me about this issue? I need to know more about it as I have not had reports of it.

 

Hello Lanre! I have an issue about focusing. I tried to off focus by my own script

Code (CSharp):

1. public class Test : MonoBehaviour
2. {
3.     public RawImage image;
4.     public ResolutionPreset resolution;
5.     public FocusMode focusmode;
6.  
7.     void Start()
8.     {
9.         NatCam.Play(DeviceCamera.RearCamera);
10.         NatCam.Camera.SetPreviewResolution(resolution);
11.         NatCam.Camera.FocusMode = focusmode;
12.         NatCam.OnPreviewStart += PreviewStart;
13.     }
14.  
15.     void PreviewStart()
16.     {
17.         image.texture = NatCam.Preview;
18.     }
19. }

i also tried tap to focus but autofocus work, can you help me?

 

Hi. This issue has been fixed in 1.5b3. The beta is not yet complete, but will soon be. Once it is, I will announce it and distribute it to NatCam devs.

 

Thank you! and one more question, why made by your plugin camera work with some little delay or lagging compared native camera app, is this normal? P.S. i used android

 

One quick request for NatCamScaler - please add a

ScaleMode.Letterbox (or however you wish to name it)

with exactly the opposite logic to FillView

if (aspect < viewAspect)
goto case ScaleMode.FixedHeightVariableWidth;
else goto case ScaleMode.FixedWidthVariableHeight;

Obviously I can make this change myself, but having to repeatedly change it if I updated the plugin would be a pain

the idea is to choose the option that will always include the whole of the image on the screen, with letterboxing

 

Hello!!!! how often do you respond?! and one more question, why made by your plugin camera work with some little delay or lagging compared native camera app, is this normal? P.S. i used android

and more an issue... once i change scene from camera scene to another scene without camera and come back to camera scene, camera doesn't work! could you help me?

P.S. solved but without event... i used

Code (CSharp):

1. public class myCamScript : MonoBehaviour
2. {
3.     public RawImage image;
4.     public ResolutionPreset resolution;
5.     public FocusMode focusmode;
6.  
7.     void Start()
8.     {
9.         NatCam.Play(DeviceCamera.FrontCamera);
10.         NatCam.Camera.SetPreviewResolution(resolution);
11.         NatCam.Camera.FocusMode = focusmode;
12.         NatCam.OnPreviewStart += PreviewStart;
13.     }
14.  
15.     void PreviewStart()
16.     {
17.         image.texture = NatCam.Preview;
18.     }
19.  
20.     void OnDisable()
21.     {
22.         NatCam.OnPreviewStart -= PreviewStart;
23.         print("Disabled");
24.     }
25. }
26.  

but it worked on PC but didn't work on Android, i used NatCam.Pause(); and NatCam.Release(); but on Android this didn't work! after i wrote on void Start()
image.texture = NatCam.Preview; and deleted
NatCam.OnPreviewStart += PreviewStart;
and
void OnDisable()
{
NatCam.OnPreviewStart -= PreviewStart;
print("Disabled");
}

and the camera started to work
I think some think wrong with EVENT NatCam.OnPreviewStart

 

Hi Lanre,

I'm having the same issue as EnoxSoftware, is there any chance you could email me a copy of the new library too, please?

Or provide a link to download it?

Thanks,
Luke

 

Hello Again Lanre,

We bought NatCam a few days ago and we've been making some tests with it, we're using the ReplayCam example and it's working well by itself. We are building a karaoke so we are using the input from the device's microphone to detect pitch levels. Both NatCam and our pitch detection script seem to work well together until we stop recording... then the app crashes. We checked the documentation to see if you expose any variables related to the microphone but there doesn't seem to be anything about that.

We are going to keep looking for a solution but it would be really helpful if you could give us a hint of what NatCam does when it stops recording that's interfering with the microphone detection script. We can send you our scene if you like. Thanks!

 

What's the recommended resolution for front facing camera and rear camera?

 

I'm using the preview event as its intended and it works. But If I try it your way it doesn't (as expected). On galaxy s6 edge.

But then I found a new problem. On android the preview texture includes ANY background visuals also. Let me explain...

If I have a canvas, in that canvas is a bunch of UI stuff like panels, buttons .ect and then I have another panel as the last sibling (so it shows on top of all the other stuff) and I assign the preview texture to the rawimage in this panel.

Whenever I use natcam.Play() the preview image will also include both what the camera is seeing AND all the other UI stuff in the background. Even if you remove all the other UI stuff and put some 3d objects in the background like plains, cubes or sphere, the preview texture will include what the camera sees and those 3d objects. It's like they bleed through. This problem does not happen in the editor. Only when deployed to android device.

Here is a picture of the preview image when other things are behind the rawimage in the scene. You can see some UI elements and my laptop monitor behind it.


My solution is to use setActive(false) on all those other things in the scene so they don't end up in the preview texture. This works, but I feel there may be more problems here. I tried scanning a barcode and my app crashed on the phone. Investigating now.

Another issue is, using natcam.Pause() and then using natcam.IsPlaying afterwards will return true on android device. But not in editor (even though the little laptop webcam light remains on in editor).

I am using unity 5.5.0f3. Android 6

Is anyone else having these issues? I really like natcam and want to see this through.

Did some more investigating, I think there may be a problem with the

Code (CSharp):

1. NatCam.RequestBarcode(new BarcodeRequest(OnDetectedBarcode, BarcodeFormat.CODE_39));

If i use my phone to scan a barcode, it will call the OnDetectedBarcode() method as expected. Now If I use natcam.pause() to stop scanning. Do whatever I want to do with the barcode data such as display it on the UI.

But now I may want to scan a new barcode? So I activate scanning again with natcam.Play() and without even doing anything else, the OnDetectedBarcode() method is called automatically with the same barcode. So it is still holding the previous data.

I tried using natcam.Release() instead of pause but it appears to crash the android device (but works in editor).

Did some more investigating, I may have been using the barcodeRequest wrong. I assumed because of the new operator I could use it whenever I wanted and it would create a new clean data structure but it seems you can only call it once, and in start(). I would prefer to be able to call it each time I use natcam.Play() so I can specifically target certain barcode formats. I'm making a tool for work so you can scan barcode labels to get data. The labels come with multiple different barcodes on them. So while I can easily parse the data programmatically using the barcode.format, I would prefer to have two different UI buttons that call two different barcodeRequests targeting different formats each. Because I can't control the end user, and having multiple barcodes close to each other on the product means I can't prevent them from accidently scanning the wrong barcode and thinking "somethings wrong with this app".

Just noticed that the detectOnce parameter doesn't seem to do anything? No matter how I set it, I can only scan the same barcode once, it won't detect it a second time.

 

Hello! my problem appears only when i change scene to another without camera but i use one more asset for maps online maps and if i change scene back to camera scene my camera shows nothnig BUT NatCam.IsPlaying is TRUE!!!
P.S. galaxy s7 unity 5.5.0f3

 

What device does this happen on?

 

We'll be adding this!

 

We have fixed this issue in 1.5b3. The beta will be ready soon.

 

Please Email me.

 

Are you running on iOS? Email me the logs and I will look into them. Also feel free to send me the scene.

 

On both cameras I recommend 1280x720 (ResolutionPreset.HD). Every camera I have come across supports this resolution. On newer devices, you can use FullHD (1920x1080) on the rear camera.

 

This is probably related to the issue where the canvas flickers. This happens on Android and is related to multithreaded rendering and MainDispatch. Check if turning on multithreaded rendering has any effect.

This is most likely a separate issue. Please email me the logs.

I will look into this.

The light remains on because the camera is still open, it is just paused. This is how WebCamTexture implements Pause(). This is correct behaviour.

Thank you! . I can assure that these issues will be resolved in 1.5f3 as we intend to spend at least 3 weeks in beta (I intend to complete and distribute 1.5b3 in a few days).

Ideally, you should use a boolean flag to decide whether you wish to process detected barcodes. NatCam.RequestBarcode is a helper that just helps filter detected barcodes before giving it to you. The problem is that the delegate cannot be unsubscribed (since it uses a lambda). The better way to do this is to manually subscribe to NatCam.OnBarcodeDetect. We are looking into creating a much better API for passing around detected metadata (barcodes, faces, and in 1.5f3, texts).

I need the logs from this. This issue was reported just a few days ago but I need to confirm that it is the same one.

Correct!

We will be considering this as we design a new metadata detection API front end.

That's a bug. I'll look into this.

 

The issue is that when Play() is called after the first time, it creates the OpenGL resources on the wrong thread (without and OpenGL context). This is why it doesn't work subsequently. The fix was as easy as moving an initialization call elsewhere. NatCam.IsPlaying is correct (because the camera is active, it just isn't being drawn by GL).

 

Hi, we've been testing NatCam as a potential replacement plugin for a AR app we are developing for iOS and Android. We have NatCam Professional and have been experiencing an unusual bug with Android devices running Android 6.0 and higher. We build the default StartingOff example as a test app and rather than getting any image from the camera, we get a blank white screen instead. We've tested this on a Nexus 5X and a Nexus 6P. This error does not occur on our other test device the Kindle Fire (2015).

Here is the raw logcat output from running the app: http://pastebin.com/1vXUdYPU

Any ideas what might be going on here? Thanks in advance!

 

Can you create the logs with NatCam's verbose mode set to on. I see a log that shows why the screen remains white, but I need to know why that log is generated in the first place.

 

Hi, yesterday I bought NatCam Extended. I followed steps from "StartingOff" video on Youtube, but I have some problems.

1. When I have "Transform Mode" set to "Vertex Transform" then Zooming doesn't work (yes I have checked "Track Zoom Gestures" and "Tracking Enabled"), but when I have "Transform Mode" set to "UV Transform" camera preview on start seems to be "ultra-zoomed", and when I put two fingers on screen everything resets to normal. Zoom is set to x1 and then pinch is working.

2. Even if I have "Track Focus Gestures" checked it doesn't work, I can touch all over the screen but not focusing on that point.

I have Samsung Galaxy S7 Edge.

Thanks!

 

Here's the same example output log with verbose mode on. This was done on the Nexus 6P running Android N.

http://pastebin.com/SDa671t8

Thanks for your help with figuring this out!

Edit: It turns out I was using the non-latest version of NatCam Professional. I updated and the white screen error no longer happens. However, another error is now occuring with the default StartingOff example. The webcam is rendered twice with a smaller thumbnail image that flashes on/off every other frame. Here is an example.

http://imgur.com/a/p90ZP

Here is the verbose output log with this program. Thanks again!

http://pastebin.com/GkmRmJBt

Edit 2: Another example from a Nexus 5 using Android 6.0

http://imgur.com/a/d72Rz

 

I've just bought NatCam 1.5, builded the startingOff example on my Galaxy S7 edge....and the camera preview was extremly slow and jerky! Also the screen layout was not correct (I had a smaller window in the bottom left corner, and the fullscreen window was flickering). I've builded the other examples as well, and these kinda work, but the preview resolution is only workable when I set it on lowest... which I don't want offcourse.

I heared great references about NatCam, so I bought it...but now I'm a bit stuck.

Is this Galaxy S7 related? of am I doing something wrong...? Can anyone help me with this issue?

Thx!

 

Hello, we have been thinking of buying your asset, but we are hesitating a bit.

We are working on an mobile app, a game with 3D characters and objects.
We want to display the mobile camera as the background of the 3D scene.
In Unity, we have set a camera and a plane with the "webcamtexture" class to do so,
but you know it performs poorly, and we want to run it perfectly.

We would like to know which of your three packages we should buy to accomplish this.
We think it is very simple as we don´t have to scan or take pictures or video,
just to display what the mobile camera sees as the background.
We suppose that with the Core would be enough, but maybe we are wrong!

Thank you. Kind regards, Jose.

 

Hey, is there a way to record some UI stuff in android? I know its using the device cam directly. But is there anyway to add some UI on top of it when its encoding the mp4?

 

What's the recommended resolution for mobile? FullHD is very slow even on a Galaxy S6...

 

Is there a way to check if SaveVideoToGallery() has run out of memory to record - i.e., to force a stop before crashing?

 

Hi @Lanre

I am interesting about you plugin,
I need to capture every video frame and apply post effect & composition , then recording to video without the UI.

As this moment I use webcamtexture to capture video frame, it take me 7-15ms every frame @ 1920x1080(webcam.update/webcam.uploadtexture) , How fast I can expect to get using NatCam?
(is took 40ms - 50ms on GPU to uploadtexture)

I already have plugin to handle the recording video & UI, I don't need to recording video.
unless your plugin can handle it better!

so is the NatCam Core can handle the capture every video frame for me? or another version is suit for me?

 

Does the barcode functionality works on iOS? Can't get it working in my app. Android is fine.

 

We are introducing the NatCamZoomer component that fixes this issue. Email me and I can send you the script.

Does your UI component have the raycastTarget flag on? Make sure it is on. Also, make sure that the focus mode allows for tap to focus:

Code (CSharp):

1. NatCam.Camera.FocusMode = FocusMode.TapToFocus | FocusMode.AutoFocus;

 

We are working to fix the flickering and duplicate issue. For now, a workaround is enabling multithreaded rendering in Player Settings.

 

We have found that this issue is because of NatCam's preview data pipeline. A workaround is simply disabling the NatCam Professional Spec (if you don't need its features) by opening NatCam>Core>Editor>NatCamLinker.cs and disabling this flag:

Code (CSharp):

1. EnableProfessionalSpec = true;

The preview should be liquid smooth once you do this. We will make a fix for this issue in NatCam Pro 1.5f3.

As far as we know, this issue only occurs on the S7 Edge.

 

The Core Specification is all you need. Extended adds barcode and face detection (and very soon, text detection) whereas the Pro Spec adds access to preview data, video recording, and full sources.

 

Not with NatCam because NatCam records directly from the camera. The Everyplay SDK is useful for this.

 

Shoot me an email. Full HD should be liquid smooth. This may be related to the lagging on the S7 Edge. Try the workaround I mentioned above.

 

Currently there isn't. We will look into adding this functionality in a future release.

 

It depends on what you want to do. If you need access to the preview data in system memory, you'll need the Pro Spec. If you only need to do GPU processing (using Graphics.Blit for instance), then the Core Spec will do. You can test this APK. Note that there is an issue where the Pro Spec (only this spec) lags on the S7 Edge and S6.

 

Yes it does. Send me an email and we'll look into it.

 

Yes, raycastTarget flag is on and FocusMode is set as you mentioned, still doesn't work (any special setting for Canvas maybe?)

One more thing, why saved photo from rear camera is mirrored and photo from front camera is normal? Any fix for rear camera saved photo?

Thanks

 

Thanks, enabling multithreaded rendering in Player Settings seems to fix the issue on our test devices.

Another issue we are encountering is trying to use NatCam.Play() after using NatCam.Release(). This was a bug mentioned in the previous page and the notes indicate that this was fixed in the latest version. However, we're still experiencing this issue with the following error that EnoxSoftware was getting here: https://forum.unity3d.com/threads/natcam-webcam-api.374690/page-15#post-2897587

01-17 13:00:46.848 20786 20808 D Unity : NatCam Logging: Generated preview texture with error 0
01-17 13:00:46.849 20786 20808 D Unity : NatCam Logging: Generated render texture with error 0
01-17 13:00:46.849 20786 20808 E Unity : NatCam Error: Creating GL program failed
01-17 13:00:46.865 559 21137 E mm-camera-isp2: bhist_stats44_stats_config_validate:363 Invalid BG from 3A h_num = 0, v_num = 0
01-17 13:00:46.865 559 21137 E mm-camera-isp2:
01-17 13:00:46.884 20786 20808 E Unity : NatCam Error: Creating GL program failed
01-17 13:00:46.919 20786 20808 E Unity : NatCam Error: Creating GL program failed
01-17 13:00:46.934 20786 20808 E Unity : NatCam Error: Creating GL program failed
01-17 13:00:46.981 20786 20808 E Unity : NatCam Error: Creating GL program failed
....
....

Here is the associated output log: http://pastebin.com/SGTnQni9

Thanks again for your help in solving these issues Lanre

 

Do you have any apk's to test the face tracking before I buy?

 

I am trying to install Scanner example on Samsung Galaxy Tab A, SM-T280 and only thing I got i s white screen.
Is this device supported?