[RELEASED] OpenCV for Unity

Hello!

I can't have the VideoCapture or VideoWriter workingon windows. Even the VideoCaptureSample scene isn't working. What does i have to do to make it works?

Unity 5.3.4f1 on Windows

Thank you

 

I am also encountering this error in the editor trying to run some of the samples:

ExecutionEngineException: SIGILL
OpenCVForUnitySample.Texture2DToMatSample.Start () (at Assets/OpenCVForUnity/Samples/Texture2DToMatSample/Texture2DToMatSample.cs:20)

This is in the Texture2DToMatSample but I get the same or similar errors in other sample scenes. Once I have this error even trying to quit Unity results in a beachball and I have to force quit. Similarly if I try to rerun the main scene to try a different sample sub-scene, it just beachballs and never starts running. So the error screws up Unity enough that even the editor itself doesn't really recover.

OS version: 10.11.1
Unity version: 5.3.1f1, 5.3.2f1, 5.3.3f1 and 5.3.4f1
OpenCV for Unity version: Latest 2.0.0 from asset store (purchased today, April 5, 2016)

I have not yet tried on device. I don't suppose there's a way to get an older version of OpenCV for Unity through some means? I don't have the luxury of having one "lying around."

Thanks.

 

Do you have moved the “OpenCVForUnity/StreamingAssets/768x576_mjpeg.mjpeg” to “Assets/StreamingAssets/768x576_mjpeg.mjpeg” ?
If you want to load video format files other than the mjpg format files, you need to set the PATH variable.

1)Download "OpenCV for Windows Version 3.1.0"(http://opencv.org/downloads.html).
2)Set PATH to "opencv_ffmpeg310.dll" or "opencv_ffmpeg310_64.dll".
if 32bit, "¥path¥to¥opencv¥build¥x86¥vc12¥bin¥".
if 64bit, "¥path¥to¥opencv¥build¥x64¥vc12¥bin¥".

 

I copy the ffmpegDDL in my project folder instead of modifying the path.The project is working fine now.
Thank you

 

Hi. I copied the opencv_ffmpeg310.dll and opencv_ffmpeg310_64.dll in Assets folder but I can't get the VideoCapture working with .MP4 video types. What am I doing wrong ?

 

why i can't download unitypackage from assetstore, even if i use VPN.
Always show that download 36% and then stop.i have try many times, it doesn't work !
and i suggest you , would you reduce you package store size?

 

Please copy to your project folder.

 

thanks! it works like a charm!

 

Hi I have another question. I am working on my MSc Thesis where I implemented an Augmented Reality project in C++ using OpenCV and OpenGL. Now I am currently converting the C++ code to C# (Unity3D - that's why I bought your asset) but I am also using SURF which you did not implemented. How can I extend your asset and implement SURF for educational purposes? Can you implement it for me? I don't get good results using ORB or (anything else).

In addition to this, I want to read the xml file where I stored the camera's calibration matrix. How I read xml with your asset? I can't find FileStorage class.

Thanks!

 

Hi!

I need my 3d model follow to face points in real time, but points on mouth does not work correctly, points of mouth always disappear real mouth. On eyes and eyebrows it works good. I tried to add some filters like Canny, adaptiveThreshold, Sobel e.t. but became worse. I know about haarcascade_mcs_mouth classifier. Is there a method to better detect mouth?

 

Package size in the next update(v 2.0.1) will be reduced to 431.7 MB
from 688.9 MB.
version 2.0.1 is currently pending review.

 

Unfortunately, I do not plan to implement the SURF and SIFT.
Also, Since this package is a clone of OpenCV Java, you are able to use the same API as OpenCV Java 3.1.0(http://docs.opencv.org/java/3.1.0/). FileStorage class has not been implemented.

 

It seems that the tracking points have been affected by the brightness and direction of the lighting.
It might improve if you adjust the lighting.

 

Hi! I am trying to run one of the examples (that works on my Unity 5.3.3) in my Android 6.0 and when i am trying to install the app, an error happend. "Application not installed". I dont know what happen, the apk is generated and the example works on unity.
I import everything but i dont know what happen. Please help me.

 

Hi @EnoxSoftware
I try to make a game base on players face, but get some troubles when test on phone:
- First, I use your Face Tracker scripts. It run well at first, but after some seconds, the position of points of the face are not accurate. Especially when I get the face out of the screen, it's totally wrong.
- Then I try to catch the face each frame --> work well, but the performance go down terrible. (average FPS is about 4-5)

So could you give me some tips please? To have accuracy and fair perfomance...
Thanks a lot

 

I think this error is not caused by the asset.

https://developer.vuforia.com/forum/unity-3-extension-technical-discussion/unable-install-apk

 

Hi! i did it. I mean... if i add all the scenes and build it, it works. However if i try to build just one of the examples ii doesn't work on my smartphone. So i dont know what happen.

 

Hi there, I just wanted to know if Extended Tracking is available in OpenCV for Unity. Is there some documentation about it? Thanks.

 

Hello, I have a problem... when I appear in front of the camera the effects draw, ok!, but when I disappear the effects continue on the screen, so How could I delete when nobody it's in front?

 

Also, I would like to know where someone can read about how to integrate an image as a trigger with the plugin. Is there a tutorial about it somewhere? I noticed there's a sample project in the Asset Store but I can't really try it out without buying the plugin first. Thanks

 

Hi!!!
i am trying to create a Roi from a Mat. But i dont know how to do it. Another option is create a mask, but i dont know how to create it.
Besides i want to use dilate function but i dont know how to create the element.

Please could anyone help me¿? Thank you so much.

 

Unfortunately ,In particular, Android device, it seems to be easy to fail in the tracking of the face.
It seems due to the performance of the autofocus.
In this sample, Once the face is moved to the outside of the camera, It is necessary to reset the detection of the face.
Also,
It seems that the tracking points have been affected by the brightness and direction of the lighting.

 

OpenCV for Unity
Released Version 2.0.1

Version changes
2.0.1
[OSX]Fixed SIGILL Exception.
[Common]Added Utils.setDebugMode() method.
[Common]Added MatchTemplateSample, StereoBMSample, SeamlessCloneSample and WebCamTextureDetectCirclesSample.
[Common]Added flipVertical flag, flapHorizontal flag and GetWebCamDevice() method to WebCamTextureToMatHelper.cs.
[Common]Excluded Old Version based on “OpenCV 2.4.11”.(In order to reduce the package size)

 

In the following such a setting, app can be installed without any problems in my environment.

Once, you might need to uninstall the old app.

 

I just bought OpenCVForUnity and so far it rocks. Just a heads up; the examples puts out a LOT of CS0618 warnings: `UnityEngine.Application.LoadLevel(string)' is obsolete: `Use SceneManager.LoadScene'. Unity 5.3.4.

 

Code (CSharp):

1.  
2.             Texture2D imgTexture = Resources.Load ("lena") as Texture2D;
3.             Mat imgMat = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC4);
4.             Utils.texture2DToMat (imgTexture, imgMat);
5.                        
6.  
7.             Texture2D imgTexture2 = Resources.Load ("chessboard") as Texture2D;
8.             Mat imgMat2 = new Mat (imgTexture2.height, imgTexture2.width, CvType.CV_8UC4);
9.             Utils.texture2DToMat (imgTexture2, imgMat2);
10.  
11.  
12.             Mat roi1 = new Mat (imgMat, new OpenCVForUnity.Rect (0, 0, imgMat.width (), 200));
13.             Mat roi2 = new Mat (imgMat2, new OpenCVForUnity.Rect (0, 200, imgMat.width (), 200));
14.  
15.  
16.             roi1.copyTo (roi2);
17.  
18.  
19.             Texture2D texture = new Texture2D (imgMat2.cols (), imgMat2.rows (), TextureFormat.RGBA32, false);
20.             Utils.matToTexture2D (imgMat2, texture);
21.  
22.  
23.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
24.  

 

Hi,

I'm looking for an example of Imgproc.watershed function in OpenCV for Unity. I've gone through several examples in C++ and Python but I have not been able to get the watershed function to work in Unity.

Do you have an example for watershed in Unity?

 

I figured out my issue. I was using a 4 channel image as the source. When I converted to a 3 channel image it is working.

 

Hi! I am trying to use the code about ROI but there is an error in Unity;
Here is the error
CvException: CvType.CV_32SC2 != m.type() || m.cols()!=1
Mat [ 0*0*CV_8UC1, isCont=False, isSubmat=False, nativeObj=0x4905075360, dataAddr=0x0 ]
OpenCVForUnity.Converters.Mat_to_vector_Mat (OpenCVForUnity.Mat m, System.Collections.Generic.List`1 mats) (at Assets/OpenCVForUnity/org/opencv/utils/Converters.cs:293)
OpenCVForUnity.Converters.Mat_to_vector_vector_Point (OpenCVForUnity.Mat m, System.Collections.Generic.List`1 pts) (at Assets/OpenCVForUnity/org/opencv/utils/Converters.cs:572)
OpenCVForUnity.Imgproc.findContours (OpenCVForUnity.Mat image, System.Collections.Generic.List`1 contours, OpenCVForUnity.Mat hierarchy, Int32 mode, Int32 method) (at Assets/OpenCVForUnity/org/opencv/imgproc/Imgproc.cs:3479)
OpenCVForUnitySample.prueba.datos () (at Assets/Scripts/prueba.cs:178)
OpenCVForUnitySample.prueba.OnWebCamTextureToMatHelperInited () (at Assets/Scripts/prueba.cs:67)
UnityEngine.Events.InvokableCall.Invoke (System.Object[] args)
UnityEngine.Events.InvokableCallList.Invoke (System.Object[] parameters)
UnityEngine.Events.UnityEventBase.Invoke (System.Object[] parameters)
UnityEngine.Events.UnityEvent.Invoke ()
OpenCVForUnitySample.WebCamTextureToMatHelper+<init>c__Iterator0.MoveNext () (at Assets/Scripts/WebCamTextureToMatHelper.cs:211)

ImMat is a 8UC4 image.
Here is my code:
//procesamientodela roi
OpenCVForUnity.Rect region;
if (Ymin > 20) {
region = new OpenCVForUnity.Rect (0, Ymin - 20, imMat.height (), Ymax - Ymin);
} else
{
region = new OpenCVForUnity.Rect (0, Ymin, imMat.height (), Ymax - Ymin);
}
Mat roi1 = new Mat (imMat, region);

 

Thank you very much for reporting.

 

I fix the problem, this is the code i used;

//procesamientodela roi
Mat roi1;
Mat imRoi= imBw.clone ();

if (Xmin > dv) {
if(Xmax<imMat.width()-dv){
roi1 = new Mat (imRoi, new OpenCVForUnity.Rect (Xmin-dv,0, Xmax+dv-Xmin, imMat.height()));
}
else{
roi1 = new Mat (imRoi, new OpenCVForUnity.Rect (Xmin-dv,0, Xmax-Xmin, imMat.height()));
}

} else {
if(Xmax<imMat.width()-dv){
roi1 = new Mat (imRoi, new OpenCVForUnity.Rect (Xmin, 0, Xmax+dv-Xmin, imMat.height()));
}else{
roi1 = new Mat (imRoi, new OpenCVForUnity.Rect (Xmin, 0, Xmax-Xmin, imMat.height()));
}
}

roi1.copyTo (imRoi);

 

I want to use moments function, in each of the contours i found in my image with findcontours. How can i do this?

 

Hello, I created a C++ dll using OpenCV. I have a method that returns a Mat. Is it able by importing the dll to Unity3d and then by calling the function that returns the Mat object to store it in a Mat using your asset ? For example Mat test = callc++DllFunctionToGetMat();
Thanks

 

Hi!
i am trying to do the matchtemplate function, but the result of the minmaxLoc is the same number in the position 0,0, both of them. Please help me!!!

This is my code.

Mat marcador,resultado,imagen;
resultado = null;

imagen = new Mat (texturaCam.height, texturaCam.width, CvType.CV_8UC4);
Utils.webCamTextureToMat (texturaCam, imagen);

Texture2D imTextura = Resources.Load ("marcador") as Texture2D;
marcador = new Mat(imTextura.height, imTextura.width, CvType.CV_8UC4);

resultado = new Mat (imTextura.height - texturaCam.height + 1, imTextura.width - texturaCam.width + 1, CvType.CV_8UC1);
Utils.texture2DToMat (imTextura, marcador);

Imgproc.matchTemplate (imagen, marcador, resultado, 1);
Core.normalize (resultado, resultado);
Core.MinMaxLocResult puntos = Core.minMaxLoc (mat_Puntos);

 

Please refer to MultiObjectTrackingBasedOnColorSample and HandPoseEstimationSample.
https://github.com/EnoxSoftware/OpenCVForUnity/search?utf8=✓&q=findContours

 

Probably, it is not possible to assign Mat class of different dll directly.
you can receive the data of Mat Via a data address of Mat or a byte array.
http://enoxsoftware.github.io/OpenC..._utils.html#a721b794c719915cb450c794fce7a8568
http://enoxsoftware.github.io/OpenC..._utils.html#add24973a01ceebdaf2b26de3df3b993a

 

http://docs.opencv.org/3.1.0/df/dfb...#ga586ebfb0a7fb604b35a23d85391329be&gsc.tab=0
Please change CvType of resultado to CvType.CV_32FC1.
https://github.com/EnoxSoftware/Ope...atchTemplateSample/MatchTemplateSample.cs#L29

 

Thank you very much. Is it possible for vice versa? For example I have a frame (Mat) using VideoCapture from your asset (C#) and then pass it to the C++ dll function to calculate the camera pose. Can you post me a sample code please?

 

Code (CSharp):

1. //C# side
2. VideoCapture capture;
3. Mat srcMat = new Mat();
4. capture.retrieve ( srcMat, 0);
5. Native_SetMat( (IntPtr)srcMat.dataAddr (), srcMat.width (), srcMat.height (), (int)srcMat.elemSize ());
6.  
7.  
8. //Your C++ dll side
9. extern "C" UNITY_INTERFACE_EXPORT void UNITY_INTERFACE_API Native_SetMat( const unsigned char* byteArray, int width, int height, int bytesPerPixel) {
10.  
11.    cv::Mat dstMat( height, width, CV_8UC3);
12.    memcpy( dstMat.data, byteArray, width*height*bytesPerPixel);
13.  
14. }
15.  

 

Thank you so much enoxSoftware for all your support.
I am developing a software with image processing and i need to use some classifier. I just need to know how to use the knn and svm function, because i dont know exactly how to do it!!!
And i need to know how the input Mat has to be.

Thank you so much for all your support.

 

I am trying to use this ->

float OpenCVForUnity.KNearest.findNearest ( Mat samples,
int k,
Mat results,
Mat neighborResponses,
Mat dist
)

but i dont know what are each of these mats... Besides i think the classifier must be trained. or not?¿ please help me

 

http://answers.opencv.org/question/53022/how-to-use-knearest-and-ann-in-opencv-30/

Rewrite in C#

Code (CSharp):

1.                         Utils.setDebugMode (true);
2.  
3.                         KNearest knn = KNearest.create ();
4.  
5.                         Mat trainFeatures = new Mat (6, 4, CvType.CV_32FC1);
6.                         trainFeatures.put (0, 0,
7.                                2, 2, 2, 2,
8.                                3, 3, 3, 3,
9.                                4, 4, 4, 4,
10.                                5, 5, 5, 5,
11.                                6, 6, 6, 6,
12.                                7, 7, 7, 7);
13.  
14.                         Mat trainLabels = new Mat (1, 6, CvType.CV_32FC1);
15.                         trainLabels.put (0, 0,
16.                              2, 3, 4, 5, 6, 7);
17.  
18.                         knn.train (trainFeatures, Ml.ROW_SAMPLE, trainLabels);
19.  
20.  
21.                         Mat testFeature = new Mat (1, 4, CvType.CV_32FC1);
22.                         testFeature.put (0, 0,
23.                              3, 3, 3, 3);
24.  
25.                         int K = 1;
26.                         Mat response = new Mat ();
27.                         Mat dist = new Mat ();
28.                         Mat results = new Mat ();
29.                         knn.findNearest (testFeature, K, results, response, dist);
30.  
31.  
32.                         Debug.Log ("response.dump() " + response.dump ());
33.                         Debug.Log ("dist.dump() " + dist.dump ());
34.  
35.                         Utils.setDebugMode (false);

 

Hello,

I have a old project working on Android with the plugin v1.2.4 !
Today I have to use it on iOS ! Because I don't have a Mac I tried the Unity CloudBuild to create a .ipa ! (Unity 5.3.4p4, last version XCode)
But I have an error :

I can't manually copy opencv2 on XCode because I use the CloudBuild. Is there another solution ? Or maybe something that was fixed in a recent version ?

 

Thank you so much!! Are you thinking about doing and API where explain how to use the diferent functions or an explanation about the functions?¿
Because now i am trying to use another classifier (like SVM) and i dont know how to use them.
So please could you tell me how can i implement the SVM classifier?¿

Thank you

 

This issue is already fixed in up-to-date versions(v2.0.1).
The following code has been added.

Code (CSharp):

1. #if UNITY_5 && UNITY_IOS
2. using UnityEngine;
3. using UnityEditor;
4. using UnityEditor.Callbacks;
5. using UnityEditor.iOS.Xcode;
6. using System.Collections;
7. using System.IO;
8.  
9. namespace OpenCVForUnity
10. {
11.         public class iOS_BuildPostprocessor : MonoBehaviour
12.         {
13.  
14.                 [PostProcessBuild]
15.                 public static void OnPostprocessBuild (BuildTarget buildTarget, string path)
16.                 {
17.                         if (buildTarget == BuildTarget.iOS) {
18.  
19.                                 #if UNITY_5_0 || UNITY_5_1 || UNITY5_2
20.                                 string projPath = path + "/Unity-iPhone.xcodeproj/project.pbxproj";
21.                                 #else
22.                                 string projPath = PBXProject.GetPBXProjectPath (path);
23.                                 #endif
24.            
25.                                 PBXProject proj = new PBXProject ();
26.                                 proj.ReadFromString (System.IO.File.ReadAllText (projPath));
27.                    
28.                                 string target = proj.TargetGuidByName ("Unity-iPhone");
29.  
30.                                 // Add our framework directory to the framework include path
31.                                 //proj.SetBuildProperty (target, "FRAMEWORK_SEARCH_PATHS", "$(inherited)");
32.                                 proj.AddBuildProperty (target, "FRAMEWORK_SEARCH_PATHS", "$(PROJECT_DIR)/Frameworks/OpenCVForUnity/Plugins/iOS");
33.  
34.                                 File.WriteAllText (projPath, proj.WriteToString ());
35.                         }
36.                 }
37.         }
38. }

 

Hello, I am doing a head tracking with green ball using MultiObjectTrackingBasedOnColorSample can you tell me if this is possible?

 

In this plugin can I use random image (for example, downloaded from a file or from the Internet at runtime) as a marker (target) for the augmented reality?

 

Hi!

I need to know how to put a ROI pixels inside a Mat to ones.

This is my code right now, after doing lot of changes:

OpenCVForUnity.Rect rect = new OpenCVForUnity.Rect (Mathf.RoundToInt(xMin*w), Mathf.RoundToInt(yMin*h), Mathf.RoundToInt((xMax-xMin)*w), Mathf.RoundToInt((yMax-yMin)*h));
Mat maskROI = new Mat(mask,rect);
maskROI.copyTo (mask);
maskROI.setTo(new Scalar(255), maskROI);
maskROI.copyTo (mask, mask (rect));

I get an error!!!

 

I think that it is probably possible.
but, I do not know whether precision is high.

 

MarkerBased AR Sample can detect and recognize NxN block markers.