[RELEASED] OpenCV for Unity

I've found on iOS devices I cant seem to get portrait mode working. My face will be found if I hold the phone sideways but not if I hold it vertically. I tried using core.flip but it only changes which landscape (left or right) picks up the face. It seems like this should be a code one liner to fix but I cant find it. Any chance I could grab some help on this?

 

It is necessary to rotate the Mat.
http://forum.unity3d.com/threads/released-opencv-for-unity.277080/#post-1880419

 

Is there a way to modify the tracking so that it will track balls instead of faces?

 

hi,

Could you explain what is happening behind Utils.matToTexture2D and Utils.texture2DToMat ? is calling that function will copy all pixels from the mat to texture and vice versa ? meaning that if i have a large image, and i do some operations on the mat for just a couple of block of pixels and call that function every frame, would i just be copying all pixels (including the unmodified pixels) to the texture ?

 

Perhaps, I think that it is possible to track the color ball.
Please refer to the following code.
http://cell0907.blogspot.jp/2013/07/tracking-ball-with-javaopencv.html

 

Those functions copy all pixels(including the unmodified pixels) using ”memcpy”.

 

Hi,

I'm porting a game to Windows Universal and trying to use OpenCV for Unity 3.0.0-rc1_beta and failing WACK for Windows Phone with whole bunch of errors like these (Supported API test):

This API is not supported for this application type - Api=OpenCVForUnity_ByteArrayToMatData. Module=opencvforunity.dll. File=Assembly-CSharp.dll.​

WACK for Windows Store apps is passing without problems but for Windows Phone appears to have these issues.
I am using Unity 5.0.2f1 and Visual Studio 2013, with Windows App Certification Kit 10.0.

Cheers,
Ivan

 

Thank you for bug report.

When I tested using the Windows App Certification Kit 3.4, OpenCVForUnitySample on Windows Phone 8,1 was passing without problems.
Unity5.0.0f4 and Visual Studio 2013, with Windows App Certification Kit 3.4

but, I have not yet tested it using Windows App Certification Kit 10.0.
Let me confirm it.

 

At picture 1
point at measure
At picture 2
How to measure distance in red line?

 

I got error.
I guess unity doen't intergrate opencv.

how to integrate unity connection with opencv?
I watched many video but I couldn't fix it.

this plugin opencv and unity video.

- where is import file?

 

Plugin does not seem to be loaded correctly. Please check the setup procedure in ReadMe.pdf.

This video is not related to ”OpenCV for Unity”.

 

Thanks, it appears that VS2015 preview might have some issues with WACK.

 

Hi

I have trouble getting Imgproc.distanceTransform to work. I get a gray image. Any help is greatly appreciated.

~ce

Code (CSharp):

1. using UnityEngine;
2. using OpenCVForUnity;
3.  
4. public class SignedDistTest : MonoBehaviour
5. {
6.     public Texture2D inputImage;
7.     public Texture2D outputImage;
8.  
9.     void Start()
10.     {
11.         // convert texture to OpenCV Mat
12.         Mat contourImage = new Mat( inputImage.height, inputImage.width, CvType.CV_8UC1 );
13.         Utils.texture2DToMat( inputImage, contourImage );
14.    
15.         // threshold and invert operation
16.         Imgproc.threshold( contourImage, contourImage, 0, 255, Imgproc.THRESH_BINARY_INV | Imgproc.THRESH_OTSU );
17.    
18.         // compute signed distance field
19.         Mat signedDistImage = new Mat( inputImage.height, inputImage.width, CvType.CV_32FC1 );
20.         Imgproc.distanceTransform( contourImage, signedDistImage, Imgproc.CV_DIST_L2, Imgproc.CV_DIST_MASK_3 );
21.  
22.         // display
23.         outputImage = new Texture2D( inputImage.width, inputImage.height, TextureFormat.ARGB32, false );
24.         OpenCVForUnity.Utils.matToTexture2D( signedDistImage, outputImage, null );
25.     }
26. }

 

In Utils.matToTexture2D(), The input Mat object has to be of the types 'CV_8UC4' (RGBA) , 'CV_8UC3' (RGB) or 'CV_8UC1' (GRAY).

You can visualize ”signedDistImage” in the following code.

Code (CSharp):

1.         // compute signed distance field
2.         Mat signedDistImage = new Mat( inputImage.height, inputImage.width, CvType.CV_32FC1 );
3.         Imgproc.distanceTransform( contourImage, signedDistImage, Imgproc.CV_DIST_L2, Imgproc.CV_DIST_MASK_3 );
4.  
5. 　　　  Mat outputMat = new Mat(contourImage.cols(), contourImage.rows(), CvType.CV_8UC1);
6.         Core.convertScaleAbs(signedDistImage, outputMat);
7.         Core.normalize(outputMat, outputMat, 0.0, 255.0, Core.NORM_MINMAX);
8.  
9.         // display
10.         outputImage = new Texture2D( inputImage.width, inputImage.height, TextureFormat.ARGB32, false );
11.         OpenCVForUnity.Utils.matToTexture2D( outputMat, outputImage, null );

 

Awesome plugin, seriously adding a lot of potential to app development.

I saw in the recent update that you added support for OpenCV 3.0.0. Does this mean the OCR tesseract methods could be working?

Thanks

 

"text" module is not currently implemented.
However, I aim at implementing "text" module in the future.
and I do not know whether it can support all platforms.（Win and Mac is ok）

 

Help ????? Metaio + OpencvForUnity + IOS
metaio webcamera is a gray texture
if i do not use opencvforunity in my project , the metaio webcamera works
Any confict with opencvforunity and metaio??
how can i fix it?

 

Hi,

I wanted to check if I can use this plugin to record the the webcamtexture output to a video(.avi) file, I am aware that opencv provide the VideoCapture and VideoWriter classes, just wanted to check if the plugin supports them.

Thanks,
Andy.

 

Thank you for bug report.
As a result of investigation, I found a solution.However, in order to fix, it is necessary to update "opencvforunity.a".
This bug will be fixed in "OpenCV for Unity version 1.1.8".

 

OpenCV for Unity API
http://enoxsoftware.github.io/OpenCVForUnity/doc/html/annotated.html
VideoWriter class is not implemented.

 

Hi,
If i want to do image recognition for multiple images with same pattern, is it possible to do that?

 

Thanks for the reply, is there any chance it might get added in any of the later builds?

 

Is it a question about ”MarkerBased AR Sample”?
It is feasible that you want to do.

 

There are no plans to implement VideoWriter class currently.

In "OpenCV Java", VideoWriter is not implemented.
http://answers.opencv.org/question/25922/opencv-java-binding-does-not-generate-videowriter-class/

 

I know that sample. Yes it is similar that i want but i want to have image not marker or combined both of them like the Vuforia's Frame Marker (https://developer.vuforia.com/library/articles/Training/Frame-Markers-Guide)

 

I think it's possible to create a MarkerLessAR App using "OpenCV for Unity".

The following page will be helpful.
https://github.com/MasteringOpenCV/code/tree/master/Chapter3_MarkerlessAR
http://www.morethantechnical.com/2015/03/16/bootstrapping-planar-ar-tracking-without-markers-wcode/

 

how can i use the class about objectness or saliency?

 

I have not yet completed the implementation of all modules of opencv_contrib.
https://github.com/Itseez/opencv_contrib/tree/3.0.0/modules
Unfortunately, saliency module has not yet been implemented.

 

Help. I got this error while calling the findHomography function. How can i fix this?

Part of the code.

Code (CSharp):

1. MatOfPoint2f mat_obj = new MatOfPoint2f(obj.ToArray());
2. MatOfPoint2f mat_scene = new MatOfPoint2f(scene.ToArray());
3.  
4. Mat H = Calib3d.findHomography(mat_obj, mat_scene);

 

obj and scene does not seem to be properly initialized.

It works well in the following code.

Code (CSharp):

1.             List<Point> obj = new List<Point>();
2.             List<Point> scene = new List<Point>();
3.  
4.             for (int i = 0; i < 10; i++) {
5.                 obj.Add(new Point());
6.                 scene.Add(new Point());
7.             }
8.  
9.             MatOfPoint2f mat_obj = new MatOfPoint2f(obj.ToArray());
10.             MatOfPoint2f mat_scene = new MatOfPoint2f(scene.ToArray());
11.          
12.             Mat H = Calib3d.findHomography(mat_obj, mat_scene);

 

OpenCV for Unity
Released Version 1.1.7

Version changes
1.1.7
[Common] Update to OpenCV2.4.11.
[Common] Add Beta Version of “OpenCV for Untiy” based on “OpenCV3.0.0”(support Unity5).

 

is there a way to read and write a single value in a mat? (i think in C opencv it is mat.at(x,y)).

if there is no such way apart from converting from a byte array. how could i create a simple 3x2 mat with the values [0,0,x][0,0,y]?

thank you!

 

In the same way as when using the OpenCV Java, it is possible to read and write a single value in a mat.

Code (CSharp):

1. 　　　　　　Mat m = new Mat(2,3,CvType.CV_8UC1);
2.             m.put (0,0, 0,0,1,0,0,2);
3. //or
4. //            m.put (0,2, 1);
5. //            m.put (1,2, 2);
6.             Debug.Log ("m.ToString " + m.ToString ());
7.             Debug.Log ("m.dump " + m.dump());
8.  
9.             Debug.Log ("m.get(0,2) " + m.get (0,2)[0]);
10.             Debug.Log ("m.get(1,2) " + m.get (1,2)[0]);
11.  

 

Hello,

I installed this asset including the markerbased AR addon, bud i get lots of double plugin plugin errors when building a scene. Even with the demo scenes i get around 8 double plugin errors.

When i remove these double plugins i get a working build but when i start my app on a device i get a white webcam screen, seems it is not recognizing the tablet webcam. It all works fine in the editor.

the demo in the app store works fine, no idea what the problem is

I work with unity 5.1.2 on a Windows 10 OS

Hope this can be resolved.

 

Import Settings of the plugin file does not seem to be right.
Please refer to the ReadMe.pdf.

 

I have changed these settings, but still getting the 'Plugins colliding with each other.' error.

No idea what the ' Put the file that you want to use for Utils.getFilePath() in the “Aseets/StreamingAssets/”folder.' means it's a bit confusing.

 

File in the "armeabi-v7a" folder, please set the CPU to ARMv7.
File in the "x86" folder, please set the CPU to x86.

Files that are placed in “Aseets/StreamingAssets/”folder will be able to retrieve the file path using the ”Utils.getFilePath()” method.

 

These settings are set, but the plugin warning keeps reporting double plugins.

I am new with this asset, with 'files' you mean what sort of files?

 

Just to clarify: With OpenCVforUnity you provide full C# source, but your OpenCV lib itself and any C/C++ code handling marshalling is closed-source (that is, you don't provide a full build we can re-generate with CMake)?

We already have OpenCV bridging in our projects (mostly for cases like fiducial 2D markers and some other AR tracking handling), and would like a more general implementation with more general Mat handling (we basically just copy into 1D arrays and pass.) However, we have also optimized many internal OpenCV methods (like cv::adaptiveThreshold, which is embarrassingly bad), so we can't use a vanilla OpenCV lib. So to clarify, do you provide full source, or only source to the C# side?

 

Does an error occur sequentially even if you delete the "x86"folder once?

 

C++ side of "OpenCV for Unity" is closed-source.
OpenCV library sources that are built into the "OpenCV for Unity" are not customized.

 

OpenCV for Unity
Released Version 1.1.8

Version changes
1.1.8
[iOS] Fix problem when working with Metaio(UnityAppController problem).
[iOS] Change file name from “OpenCVForUnity/Plugins/iOS/MyAppController.mm” to “OpenCVForUnity/Plugins/iOS/OpenCVForUnityAppController.mm”.
[Common] Add [System.Serializable] to basic class.
[iOS]Move “OpenCVForUnity/ iOSforXcode/iOS_BuildPostprocessor.cs” to “OpenCVForUnity/Editor”folder.
[Common] Add Beta2 Version of “OpenCV for Untiy” based on “OpenCV3.0.0”(support Unity5).

 

If you are looking for video recording on iOS check out Ividcappro

 

Work on android in real time camera ?

 

Hello everybody,
At this time Im using Unity - Vuforia or AR Toolkit . Before I can buy now a version of Open CV for Unity I have some questions here, for developing a Marker Based AR- App, because I really dont get it.
1.How can I make/generate/train my own markers?
2.Furthermore, is it possible to use my existing marker from a Vuforia/Unity project)?Are there any restrictions to how the marker should look like?
3.Are there any tutorials for a OpenCV-noob like me, for example, how to create a marker, which will be recognized later?
Many Thanks in advance

 

You can convert the WebCamTexture to Mat.
Please try to download DemoApplication.
https://play.google.com/store/apps/details?id=com.enoxsoftware.opencvforunitysample

 

I am trying to figure out how to set up getPerspectiveTransform using Mats for src and dst. The docs are not clear on how the Mats are to be structured.

UPDATE: Solved myself, but will leave here in case anyone else runs into the same issue.

float[] srcPoint = new float[]{x[0] , y[0] , x[1] , y[1] , x[2] , y[2] , x[3] , y[3]};
Mat srcMat = new Mat(4,2,CvType.CV_32F);
srcMat.put(0, 0,srcPoint );

Set up the dst the same way and pass to getPerspectiveTransform.

 

i'm trying to return just user face into texture
can anyone help me

 

Have you tried the ”DetectFaceSample” and ”WebCamTextureDetectFaceSample”?

 

yes, it's work perfectly, i want to create a texture for the user face while tracjing his face