NatDevice - Media Device API

I've just purchased Natcam, and the example scene isn't working correctly in the editor. The video screen just creates a coloured flicker. If i press the picture button it does show a complete image. Is there something I can try to get video working correctly?

Thanks

 

Hi @galent. Can you post a screenshot? I am unfamiliar with this issue. I should note that in the editor, NatCam uses WebCamTexture.

 

Do you have face detection sample code for OpenCV?

 

Here's a screen shot

it seems to be related to a UV issue (described here)

That said, when I build and deploy to my Galaxy S5 (Android OS 6.0.1), the image shows but auto-focus doesn't seem to be working, the snapshot button seems to stretch the image a little and barcode detection... not sure if that's working either... hard to tell since it won't autofocus.

Any thoughts?

 

I had issues with the focus too, I initialized NatCam with Verbose mode and sent Lanre my logcat logs, might be useful to send your logs too?

 

I emailed the logcat logs. Still not able to use the autofocus, which I'm hoping is the reason the natcam also can't parse the barcodes I'm putting in front of it.

 

I'm also seeing nothing in the editor when running the exmple. I'm using it just for testing but having it work in editor would be immensely useful.

This error seems relevant:
NatCam Error: Camera is already active
UnityEngine.Debug:LogWarning(Object)
NatCamU.Internals.NatCamExtensions:Warn(String) (at Assets/NatCam/Plugins/Managed/NatCamExtensions.cs:170)
NatCamU.NatCam:set_ActiveCamera(DeviceCamera) (at Assets/NatCam/Plugins/Managed/NatCam.cs:89)
NatCamU.UnitygramBase:Start() (at Assets/NatCam/Scripts/UnitygramBase.cs:31)
Unitygram:Start() (at Assets/NatCam/Examples/Unitygram/Unitygram.cs:25)

And can't seem to press the button on the device (but I do see a preview). Switching from front to back causes a crash 50% of the time. I'll send logcats later today.

 

Hi, I set FOV from NatCam.ActiveCamera.HorizontalFOV and is 56.7 on iPhone 5, but close to real is 32. Why?

Also autofocus is not working on Samsung Galaxy Tab-S.

SONY Experia Z3 Compact - auftofocus is not working!

 

on samsung xcover 3 i get this bug, please help!

SM-G388F
android 5.1.1

 

Hi guys! Sorry I have been silent for the past few days. I have just gotten back from a trip. NatCam 1.3b1 is on track now featuring universal barcode detection, which means you can now scan barcodes in the editor.

 

Not yet. I'll work on it once NatCam 1.3 is released.

 

I would have to agree with you that it a UV issue. NatCam uses WebCamTexture when running in the editor, so it must be related to that. I have a suspicion though: try disabling NatCamPreviewScaler if you have it enabled and let me know if it shows properly. I replied to your email concerning the S5 focusing issue.

 

It could be a focusing issue. It could also be that Google Play Services is not available. We've added a check in 1.3 to make sure that it is available for use. I would email you the latest 1.3 beta once it is completed.

 

Hi. I don't quite understand. The horizontal FOV from the camera depends on what format it is set to. This is specific to the AVFoundation framework.

As for autofocus, can you send me the logs? Email me at olokobayusuf@gmail.com. NatCam 1.2 has a bug where it sometimes ignores setting autofocus when you set it before the preview starts. You can workaround it by calling this:

Code (CSharp):

1. NatCam.ExecuteOnPreviewStart(() => NatCam.ActiveCamera.FocusMode = FocusMode.AutoFocus);

This has been fixed in NatCam 1.3.

 

Hi @fruins. Email me at olokobayusuf@gmail.com. I will send you a new APK to test. This sort of issue should be inherently fixed in NatCam 1.3.

 

Fixed!

 

NatCam 1.3 is very close to submission. It will probably be sent in tomorrow.

 

We are starting a YouTube Tutorial Series for NatCam. Here is the current list of tutorials we plan to release:

1. Starting Off. UnitygramBase, custom initialization, and switching cameras.
2. Scaling the Preview. Using NatCamPreviewScaler.
3. Focusing and Focus Modes. Using NatCamPreviewGestures for easy focusing.
4. Zooming. Using NatCamPreviewZoomer with NatCamPreviewGestures.
5. Capturing Photos. Supplying PhotoCallbacks and using PhotoSaveMode.
6. Detecting Barcodes. Using BarcodeRequests and detecting multiple formats at once.
7. Detecting Faces. Creating a simple rectangle that follows a face.
8. Using Shaders. Creating a black-and-white preview.
9. Advanced: Realtime Instagram-Style Color Grading using Look Up Textures.
10. Unitygram. Creating a simple camera app that combines these lessons.

Please let me know of any requests or suggestions you have for these tutorials. Thank you!

 

Hi Lanre, I just downloaded and tried the new 1.3. the screen is working fine in the editor (unity 5.3.5), but the PDF417 specific scanner isn't working (editor or on my samsung), I can scan the code if I set it to all detection formats, but only if I designate the fallback interface (native won't scan QR or PDF417 in editor or on mobile.

Any idea what's happening there?

 

Hi @galent can you send me logs your samsung? I know that PDF417 takes a little longer to scan than regular barcodes, but QR should get picked up immediately. The native interface doesn't run in the editor--only the fallback does. Try scanning QR when you generate the logs (and remember to put verbose mode on).

 

This might only be an iOS 10 beta error, but just a heads up.

Using Metal as the graphics API results in quite severe stuttering on Metal supported devices. Works great on OpenGLES 3, so this could very well be the case of a beta issue. Hitting a constant 60fps while accessing the camera and rendering items on top feels super nice, all while the camera feed being of great quality.

This is without a doubt one of the best purchases i have made on the app store, the efficiency, ease of use and support is all top notch.

 

Thanks for the notice. I will keep it in mind. NatCam 1.3 should not be affected since it doesn't perform any drawing; it just uploads preview data to the GPU. Thanks for the great review!

 

Hi guys.

I just noticed a bug in NatCam 1.3.

When running a readable preview on iOS, switching cameras might cause a hard crash (EXC_BAD_ACCESS). The fix is trivial: In NatCam.mm, add this RGBA32 nullifying line at the end of -(void) SwitchCamera:

Code (CSharp):

1. ...
2. [session commitConfiguration];
3. if (RGBA32 != NULL) delete [] RGBA32; RGBA32 = NULL; //Add this line

Sorry, thank you, and happy coding!

 

The stuttering only seems to affect the preview texture coming from NatCam when using Metal, but it's an iOS beta, then again i might have broken something somewhere. ¯\_(ツ)_/¯.

 

Dear,

I just bought NatCam & Vuforia for 550$,
I put them together in my project but it looks like conflicts dex files.
Is there any solutions to combine them?

 

Hi, just bought your asset, looks to be great! I'm hoping I can prevent updating my Unity Version to use it though as I have several reasons not to update my project to 5.2 and up. Any chance this will work on 5.1.4? I get this error upon import:
Assets/NatCam/Scripts/NatCamPreviewScaler.cs(19,64): error CS0246: The type or namespace name `IMeshModifier' could not be found. Are you missing a using directive or an assembly reference?

Obviously that is part of the updated UnityEngine.UI but not sure if anything can be done to add that functionality to 5.1.4... any help would be appreciated. Perhaps even an older version of NatCam would be fine as my main interest is to have the camera feed as a background for my AR project.

Edit: this error shows as well if I simply comment out the IMeshModifier bit

Assets/NatCam/Scripts/NatCamPreviewScaler.cs(79,33): error CS0246: The type or namespace name `VertexHelper' could not be found. Are you missing a using directive or an assembly reference?

 

You do not need the scaler to make NatCam work. I'm fairly sure you could just delete that script and make your own scaling method like what i'm doing, you do not need to draw it to a RawImage component.

And just me being curious, why aren't you able to go above version 5.1.4?

 

Hi @admob . NatCam and Vuforia are incompatible. I emailed Vuforia a long time ago so that we could make them work together, but I never got a reply. What are you trying to achieve? Depending on what you want to do, I can advise you on which package to use and how.

EDIT: I just emailed them again. I'll let you know if I hear back.

 

Good point! @christougher, the scaler modifies individual vertices, but the logic stays the same:

Code (CSharp):

1. Find the aspect ratio of the texture applied (the dim vector). Calculated as: float ar = dim.x / dim.y.
2. Depending on the scale mode, multiply or divide the height or width (respectively) of the rect transform by the aspect ratio calculated in (1).
3. Set your new width or height.

This can easily be translated to using rectTransform.localScale instead of the vertices.

 

Does NatCam work on Android in 5.1.4? I would think not because Android depends on the new Unity rendering LLAPI introduced in 5.2. If you aren't using NatCam on Android, then you're going to be fine.

Another error you should encounter is the Texture2D.LoadRawTextureData(IntPtr, int) overload that also wasn't introduced till 5.3 (because double-buffering the preview data in native and managed memory meant 2x memory cost). This is why the minimum recommended Unity version is 5.3. To get around this, replace every instance of:

Code (CSharp):

1. Texture2D.LoadRawTextureData(IntPtr, int);

with

Code (CSharp):

1. Texture2D.LoadRawTextureData(Ext.MarshalNativeBuffer(IntPtr, int)); //Where Ext is an alias for NatCamExtensions

 

Using it for both IOS and android. Had quite a few incompatability issues with 5.2 which led me to rollback to 5.1.4 a while back. Trying wating to recreate anything here. I'm not very good at scripting.

Also I get this when NatCam is first imported:

System.IO.FileNotFoundException: Could not find file "C:\Users\Chris\Documents\Turret\Assets\NatCam\Plugins\Dependencies\ZXing.NET\zxing.unity.dll.mdb".

Deleting the scaler script just leads to more errors with other scripts.... NatCamNativeInterface throws multiple, and I imagine it all snowballs from there... :/

Maybe it's time to just update to 5.3 after all. Gonna break something for sure lol.

 

The errors don't surprise me. I personally advise you to upgrade; the issues you faced in 5.2 might have been solved.

 

Hello,
I have a question about NatCam.PreviewFrame
It always returns the texture in a horizontal orientation and stretches if necessary.
Am i right?

 

Yes, the texture returned is always in landscape orientation. But no stretching is applied. This is because the natural orientation of the camera is always landscape--regardless of the device orientation. This plugs in directly to the rendering pipeline, so the rendering pipeline creates a texture that is always landscape. But the preview is always oriented correctly, only that its height and width are not swapped in portrait mode. This means that if you save it with EncodeToPNG and read it from the device gallery, it will be stretched. If you load it back into a texture in Unity however, it will be correctly displayed. Note that this only occurs when the app is in portrait mode.

One workaround I can think of is creating a RenderTexture with the width and height swapped then blitting from NatCam.PreviewFrame and performing a readback from the GPU (Texture2D.ReadPixels).

Code (CSharp):

1. if (appIsPortrait) {
2. Texture2D frame = NatCam.PreviewFrame;
3. RenderTexture correctRenderTexture = new RenderTexture(frame.height, frame.width, 0); //Swap width and height
4. correctRenderTexture.Create();
5. Graphics.Blit(frame, correctRenderTexture);
6. Texture2D correctFrame = new Texture2D(correctRenderTexture.width, correctRenderTexture.height, frame.format, false);
7. RenderTexture.active = correctRenderTexture;
8. correctFrame.ReadPixels(new Rect(0, 0, correctRenderTexture.width, correctRenderTexture.height), 0, 0);
9. RenderTexture.active = null;
10. correctRenderTexture.Discard();
11. correctFrame.Apply();
12. }

 

I did manage to reach Vuforia and have started preliminary talks with their developers. NatCam-Vuforia interoperability might become a reality. Let's keep our fingers crossed.

 

K, I updated to 5.3.4 (which I already had installed for another project) and now I get this error on import... would this be something fixed with 5.3.5?

Assets/NatCam/Plugins/Managed/NatCamNativeInterface.cs(465,45): error CS1674: `AndroidJavaClass': type used in a using statement must be implicitly convertible to `System.IDisposable'

Edit... Android SDK needed updating... Doing that right now...

Edit: Didn't help.... same error, but it works fine in another 5.3.4 project... any guidance what the problem could be?

Edit (again) anything I can find online references adding an Entity Framework dll (EF5 or EF6) to the project? But this seems to have more to do with Visual Studio?????

 

I would love for that to happen. We use Vuforia for some marker based tracking solutions, but the speed and quality of the camera feed seems much better with NatCam. I know it's early, but would the integration be that NatCam would pass the captured frames to the Vuforia tracking algorith and bypass their camera capture layer or would NatCam just extend the Vuforia camera capture layer?

 

This might be a lot to ask, but try reinstalling Unity. AndroidJavaClass does implement IDisposable. Some NatCam developers use 5.3.4 and have not run into this issue, so I think there might be something wrong with Unity's assemblies. See the Dispose function here.

 

Yeah, I'm installing 5.3.5 right now. Natcam works fine on 5.3.4 on another computer, I'm wondering if it has to do with my Visual Studio 2013...

 

The way I would want it to work--and I'm sure other developers would want this too--is that NatCam will pass the camera preview to Vuforia for processing. The goal of combining NatCam and Vuforia is so that developers have access to NatCam's broad, customizable control API and Vuforia's specialized AR functionality. I am yet to hear back from their developers, but I will try to get as close to such integration as possible.

 

I doubt that it's because of Visual Studio; it sounds more like incomplete/corrupted assemblies. Let me know how it goes.

 

Still get the same error... fresh download of 5.3.5. How can I fix something like this? It's obviously a problem with something other then Natcam so extra thanks for the guidance.

 

So, looking at some similar problems online I got rid of the error with these changes:

Code (CSharp):

1.                         //using (AndroidJavaClass NCNAClass = new AndroidJavaClass("com.yusufolokoba.natcam.NatCam")) {
2.                         AndroidJavaClass NCNAClass = new AndroidJavaClass("com.yusufolokoba.natcam.NatCam");
3.                        
4.                             if (NCNAClass != null) {
5.                                 NCNA = NCNAClass.CallStatic<AndroidJavaObject>("instance");
6.                             }
7.                        

Hope it doesn't break anything...

 

Now when I try to build I get a "cannot merge android manifest files..."

 

You have to manually add this at the end:

Code (CSharp):

1. NCNAClass.Dispose();

This is what IDisposable and the 'using' block automatically do for you.

 

If it is complaining about the minSdkVersion, you must change the minimum API level in Player Settings to at least API level 16.

 

K android files not merging was fixed by changing the minimum API level for Android to 16, but now I get two errors when building that it cannot convert java class into dex format.... I've even updated the Java Development Kit (this error shows the older one) and Android SDK but that doesn't work. Everything builds fine when Natcam is removed...

Edit: Would this be caused by conflicting android manifests? I have several from google play services, Unity Ads and a mystery one. I'll weed them out and see if that works...

Edit: Working!

 

Oh thanks, got that figured out, lol

 

It seems that it is because of google ads. Nice that you have it working!

 

I have the same problem, but this time, changing of API level to 16 or any others levels doesn't work for me:



CommandInvokationFailure: Unable to merge android manifests. See the Console for more details.
/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/bin/java -Xmx2048M -Dcom.android.sdkmanager.toolsdir="/Projects/Android/sdk/tools" -Dfile.encoding=UTF8 -jar "/Applications/Unity/PlaybackEngines/AndroidPlayer/Tools/sdktools.jar" -

stderr[]
stdout[
Warning: [Temp/StagingArea/AndroidManifest-main.xml:14, /Projects/GhostAR/Temp/StagingArea/android-libraries/NatCam/AndroidManifest.xml:3] Main manifest has <uses-sdk android:targetSdkVersion='22'> but library uses targetSdkVersion='23'