[RELEASED] OpenCV for Unity

I've been trying to get the dlib face tracking working on the Hololens. When I build the project on the hololens I get an error that the opencvforunity.dll does not have the right architecture. I thought that there was (beta) support for Windows 10 UWP. Is there any way I can get this to work on a Hololens?

 

@EnoxSoftware

I've been experimenting with SVMs today, and haven't been able to save the trained SVM to disk. Whenever I call the SVM's "save" method, Unity crashes with an "unhandled NULL exception". I've tried a few different ways, using a path to an existing empty file, a path to an existing empty file with a ".xml" extension, and to a nonexistent file (with and without the extension) with the same result. I've tried this with the SVM in the provided SVMExample scene as well as my own with no difference.

For what it's worth, I'm running this in the Unity editor (5.5.0p1) on a 13" late 2013 MacBook Pro, with macOS Sierra 10.12.2.

Any ideas what might be going wrong? It doesn't seem to be related to the path I'm passing into the method, and I can't catch the exception to get more details on it. Mostly, I'm curious if this same issue is repeatable on other platforms or Unity versions.

 

Hi, Enox

I find circle on Camera Preview from NatCam Core, with your OpenCVforUnity.

The code below work fine on iOS, But does not work on Android.

...
cameraMat = new Mat(renderTexture.height, renderTexture.width, CvType.CV_8UC4);
...
void OnRenderImage(RenderTexture source, RenderTexture destination)
{
Utils.textureToMat(source, cameraMat);

Mat grayMat = new Mat(cameraMat, new Range(0, 128), new Range(0, 128));
Imgproc.cvtColor(grayMat, grayMat, Imgproc.COLOR_RGBA2GRAY);

using (Mat circles = new Mat())
{
Imgproc.HoughCircles(grayMat, circles, Imgproc.CV_HOUGH_GRADIENT, 2, 10, 160, 50, 10, 40);

...


Maybe Utils.textureToMat(RenderTexture, Mat) does not work, I don't know because of NatCam Preview.

Of course, The code below work fine on android.

tex.ReadPixels(new UnityEngine.Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
Utils.texture2DToMat(tex, cameraMat);

I need help. thank you.

 

Please set ImportSettings as follows.
Platform : WebPlayer
SDK : UWP
CPU : x86 or x64 or ARM

 

"OpenCV for Unity"v2.0.3 is a clone of OpenCV Java bindings.
For now Algorithm.save () method seems to have a bug in the Java bindings of OpenCV.
https://github.com/Itseez/opencv/issues/5894

 

Hello,

I'm new on this forum and new at using openCV for unity so sorry in advance if my question was already answer.

I'm trying to cheak every element of my Mat in unity then compare the value, i try this code in openCV for consoleApplication in VisualStudio2015 :

Code (CSharp):

1.  
2.         float threshold = 40.0f;
3.         float dist;
4.  
5.         for (int j = 0; j < motion.rows; ++j){
6.             for (int i = 0; i < motion.cols; ++i)
7.             {
8.                 cv::Vec3b pix = motion.at<cv::Vec3b>(j, i);
9.  
10.                 dist = (pix[0] * pix[0] + pix[1] * pix[1] + pix[2] * pix[2]);
11.                 dist = sqrt(dist);
12.  
13.                 if (dist > threshold)
14.                 {
15.                     foregroundMask.at<unsigned char>(j, i) = 255;
16.                 }
17.             }
18.         }    

This code work on my console so i decide to translate this code in c# for unity but the best i can do is this :

Code (CSharp):

1.  double threshold = 40.0f;
2.         double dist = 20;
3.      
4.         for (int j = 0; j < motion.rows(); ++j)
5.         {
6.             for (int i = 0; i < motion.cols(); ++i)
7.             {
8.                 Vector3 pix = new Vector3(i, j, motion.type());
9.                 dist = (pix[0] * pix[0] + pix[1] * pix[1] + pix[2] * pix[2]);
10.                 dist = Math.Sqrt(dist);
11.              
12.                 if (dist > threshold)
13.                 {
14.                     foregroundMask = new Mat(motion.rows(), motion.cols(), 255);
15.                 }
16.                
17.             }
18.         }

And this code make unity crash every time. So the big problem is they are nothing to replace Vec3b and replace it with a vector3 didn't work well.
How can i use Mat.at<vec3b> and Vec3b for openCV for unity ?


Any help will be good, i'm really stuck right now :/. Thanks

 

I have a requirement of estimating the geometric transformation between 2 sets of 2D points (one set having more than 10 points and both the sets having the same number of points) and then using that transformation matrix to transform another set of 2D points. Can you please let me know if this can be done using OpenCVForUnity? if so how?

 

Hi Enox
I have an installation and am trying to detect faces via webcam at some distance (up to 12ft). How can I adjust the current webcam texture async example to detect faces that small?

 

Does it work well when using only OpenCV for Unity?
Also, Is "OpenGL 2" or "OpenGL 3" set for the Graphics API?

 

Hey, thanks for getting back to me so quickly. Kind of a shame that the issue's over a year old and hasn't been addressed by anyone. Hopefully I can use all the get/set methods and properties to create my own serialization.

 

Please change the parameter of CascadeClassifier.detectMultiScale().
http://docs.opencv.org/3.2.0/d1/de5...sifier.html#aaf8181cb63968136476ec4204ffca498
scaleFactor Parameter specifying how much the image size is reduced at each image scale.
minSize Minimum possible object size. Objects smaller than that are ignored.

 

Code (CSharp):

1.             float threshold = 40.0f;
2.             float dist;
3.  
4.             byte[] pix = new byte[3];
5.             byte[] byte255 = new byte[]{255};
6.          
7.             for (int j = 0; j < motion.rows(); ++j){
8.                 for (int i = 0; i < motion.cols(); ++i)
9.                 {
10.                     motion.get (j, i, pix);
11.                  
12.                     dist = (pix[0] * pix[0] + pix[1] * pix[1] + pix[2] * pix[2]);
13.                     dist = Mathf.Sqrt(dist);
14.                  
15.                     if (dist > threshold)
16.                     {
17.                         foregroundMask.put (j, i, byte255);
18.                     }
19.                 }
20.             }    

 

I think that it is probably feasible using Calib3d.findHomography().
http://stackoverflow.com/questions/...ene-using-homography-and-perspective-transfor

 

I can't seem to access the NatCam demonstration linked here:

https://enoxsoftware.com/opencvforunity/documentation/integration-examples/

 

Yes, Work fine that find circle on Natcam's Preview on iOS with any graphic api(opengles2, opengles3, metal) but
does not work on Android both with opengles2, opengles3.

Of course, work fine on the Unity webcam under all conditions.

 

Thank you!! NatCam.PreviewMatrix (http://docs.natcam.io/)

 

Thanks Enox work perfectly !

 

Does the problem only occur when combining NatCam and OpenCVforUnity?
Does Utils.textureToMat() work properly if OpenCVforUnity is used alone?

 

I wrote via the support form on your website but have not received a reply.

I have a USB camera that I wish to use on Android. I have used other software from the app-store which detects the presence of this camera, but when I ran your Android demo the camera was not detected. Can your library use USB webcams?

By the way, the DNA logo you have on your website is left-handed DNA, it should be right-handed

​

 

If that USB webcam is available in the WebCamTexture class, that webcam is also available from OpenCVforUnity.

Thanks!!

 

Hello,
i'm facing a problem with the handpose estimation sample , here is the error :

Code (CSharp):

1. NullReferenceException: Object reference not set to an instance of an object
2. OpenCVForUnity.Point.op_Inequality (OpenCVForUnity.Point lhs, OpenCVForUnity.Point rhs) (at Assets/OpenCVForUnity/org/opencv/core/Point.cs:239)
3. OpenCVForUnitySample.HandPoseEstimationSample.Update () (at Assets/OpenCVForUnity/Samples/HandPoseEstimationSample/HandPoseEstimationSample.cs:184)
4.  

 

Thanks for replying to my earlier question. I am trying to optimize the facedetection webcam demo since I am also displaying a canned HD video in the same application and playing back on a less powerful PC(Intel NUC)
Is it possible to somehow hide or not actually render the webcam feed yet still have the face detection work?
Are there any other optimizations you can suggest?

How is CUDA support enabled so that OpenCV runs entirely on the GPU?
http://enoxsoftware.github.io/OpenCVForUnity/doc/html/class_open_c_v_for_unity_1_1_gpu.html

Thanks in advance!

 

Thank you very much for reporting.
To fix it, please delete line 232 to line 240 of Assets/OpenCVForUnity/org/opencv/core/Point.cs.
This bug will be fixed in the next version upgrade.

Code (CSharp):

1.         //
2.         #region Operators
3.        
4.         #if UNITY_PRO_LICENSE || ((UNITY_ANDROID || UNITY_IOS || UNITY_WEBGL) && !UNITY_EDITOR) || UNITY_5
5.        
6.         public static Point operator + (Point pt)
7.         {
8.             return pt;
9.         }
10.        
11.         public static Point operator - (Point pt)
12.         {
13.             return new Point (-pt.x, -pt.y);
14.         }
15.        
16.         public static Point operator + (Point p1, Point p2)
17.         {
18.             return new Point (p1.x + p2.x, p1.y + p2.y);
19.         }
20.        
21.         public static Point operator - (Point p1, Point p2)
22.         {
23.             return new Point (p1.x - p2.x, p1.y - p2.y);
24.         }
25.        
26.         public static Point operator * (Point pt, double scale)
27.         {
28.             return new Point (pt.x * scale, pt.y * scale);
29.         }
30.        
31.         public static Point operator * (double scale, Point pt)
32.         {
33.             return new Point (pt.x * scale, pt.y * scale);
34.         }
35. ///////////////////////////////////////////////////////////////////////////////////////////////        
36. //        public static bool operator == (Point lhs, Point rhs)
37. //        {
38. //            return lhs.Equals (rhs);
39. //        }
40. //        
41. //        public static bool operator != (Point lhs, Point rhs)
42. //        {
43. //            return !lhs.Equals (rhs);
44. //        }
45. ///////////////////////////////////////////////////////////////////////////////////////////////        
46.         #endif
47.        
48.         #endregion
49.         //

 

Tank you it works now!!!

 

how can i get more accuracy result in the hand pos estimation or hand tracking??

 

Yes, Even if you delete this line, it is still possible to detect the face.

Code (CSharp):

1. Utils.matToTexture2D (rgbaMat, texture, webCamTextureToMatHelper.GetBufferColors ());

OpenCVforUnity currently does not support the Cuda modules.

 

I am not familiar with hand tracking algorithms.
Another example using OpenCV may be helpful.

 

Ok thanks.

 

Thanks! One other question. How would I modify the HOG descriptor example to accept webcam input instead of a video?

Trying to achieve something close to

 

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. using UnityEngine.UI;
4.  
5. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
6. using UnityEngine.SceneManagement;
7. #endif
8. using OpenCVForUnity;
9.  
10. namespace OpenCVForUnitySample
11. {
12.     /// <summary>
13.     /// WebCamTexture to mat sample.
14.     /// </summary>
15.     [RequireComponent (typeof(WebCamTextureToMatHelper))]
16.     public class WebCamTextureHOGDescriptorSample : MonoBehaviour
17.     {
18.         /// <summary>
19.         /// The texture.
20.         /// </summary>
21.         Texture2D texture;
22.  
23.         /// <summary>
24.         /// The web cam texture to mat helper.
25.         /// </summary>
26.         WebCamTextureToMatHelper webCamTextureToMatHelper;
27.  
28.         /// <summary>
29.         /// The is flip vertical toggle.
30.         /// </summary>
31.         public Toggle isFlipVerticalToggle;
32.      
33.         /// <summary>
34.         /// The is flip horizontal toggle.
35.         /// </summary>
36.         public Toggle isFlipHorizontalToggle;
37.  
38.         /// <summary>
39.         /// The rgb mat.
40.         /// </summary>
41.         Mat rgbMat;
42.  
43.         /// <summary>
44.         /// HOGDescriptor
45.         /// </summary>
46.         HOGDescriptor des;
47.  
48.         // Use this for initialization
49.         void Start ()
50.         {
51.             Utils.setDebugMode (true);
52.  
53.             webCamTextureToMatHelper = gameObject.GetComponent<WebCamTextureToMatHelper> ();
54.             webCamTextureToMatHelper.Init ();
55.  
56.             isFlipHorizontalToggle.isOn = webCamTextureToMatHelper.flipVertical;
57.             isFlipHorizontalToggle.isOn = webCamTextureToMatHelper.flipHorizontal;
58.  
59.             des = new HOGDescriptor ();
60.         }
61.  
62.         /// <summary>
63.         /// Raises the web cam texture to mat helper inited event.
64.         /// </summary>
65.         public void OnWebCamTextureToMatHelperInited ()
66.         {
67.             Debug.Log ("OnWebCamTextureToMatHelperInited");
68.  
69.             Mat webCamTextureMat = webCamTextureToMatHelper.GetMat ();
70.  
71.             texture = new Texture2D (webCamTextureMat.cols (), webCamTextureMat.rows (), TextureFormat.RGBA32, false);
72.  
73.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
74.  
75.             gameObject.transform.localScale = new Vector3 (webCamTextureMat.cols (), webCamTextureMat.rows (), 1);
76.             Debug.Log ("Screen.width " + Screen.width + " Screen.height " + Screen.height + " Screen.orientation " + Screen.orientation);
77.  
78.                                  
79.             float width = webCamTextureMat.width ();
80.             float height = webCamTextureMat.height ();
81.                                  
82.             float widthScale = (float)Screen.width / width;
83.             float heightScale = (float)Screen.height / height;
84.             if (widthScale < heightScale) {
85.                 Camera.main.orthographicSize = (width * (float)Screen.height / (float)Screen.width) / 2;
86.             } else {
87.                 Camera.main.orthographicSize = height / 2;
88.             }
89.  
90.             rgbMat = new Mat (webCamTextureMat.rows (), webCamTextureMat.cols (), CvType.CV_8UC3);
91.         }
92.  
93.         /// <summary>
94.         /// Raises the web cam texture to mat helper disposed event.
95.         /// </summary>
96.         public void OnWebCamTextureToMatHelperDisposed ()
97.         {
98.             Debug.Log ("OnWebCamTextureToMatHelperDisposed");
99.  
100.         }
101.  
102.         /// <summary>
103.         /// Raises the web cam texture to mat helper error occurred event.
104.         /// </summary>
105.         /// <param name="errorCode">Error code.</param>
106.         public void OnWebCamTextureToMatHelperErrorOccurred (WebCamTextureToMatHelper.ErrorCode errorCode)
107.         {
108.             Debug.Log ("OnWebCamTextureToMatHelperErrorOccurred " + errorCode);
109.         }
110.  
111.         // Update is called once per frame
112.         void Update ()
113.         {
114.             if (webCamTextureToMatHelper.IsPlaying () && webCamTextureToMatHelper.DidUpdateThisFrame ()) {
115.  
116.                 Mat rgbaMat = webCamTextureToMatHelper.GetMat ();
117.  
118.                 Imgproc.cvtColor (rgbaMat, rgbMat, Imgproc.COLOR_RGBA2RGB);
119.  
120.                 using (MatOfRect locations = new MatOfRect ())
121.                 using (MatOfDouble weights = new MatOfDouble ()) {
122.                     des.setSVMDetector (HOGDescriptor.getDefaultPeopleDetector ());
123.                     des.detectMultiScale (rgbMat, locations, weights);
124.  
125.                     OpenCVForUnity.Rect[] rects = locations.toArray ();
126.                     for (int i = 0; i < rects.Length; i++) {
127.                         //                                              Debug.Log ("detected person " + rects [i]);
128.                         Imgproc.rectangle (rgbaMat, new Point (rects [i].x, rects [i].y), new Point (rects [i].x + rects [i].width, rects [i].y + rects [i].height), new Scalar (255, 0, 0, 255), 2);
129.                     }
130. //                    Debug.Log (locations.ToString ());
131. //                    Debug.Log (weights.ToString ());
132.                 }
133.  
134.                 Imgproc.putText (rgbaMat, "W:" + rgbaMat.width () + " H:" + rgbaMat.height () + " SO:" + Screen.orientation, new Point (5, rgbaMat.rows () - 10), Core.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar (255, 255, 255, 255), 2, Imgproc.LINE_AA, false);
135.  
136.                 Utils.matToTexture2D (rgbaMat, texture, webCamTextureToMatHelper.GetBufferColors ());
137.             }
138.         }
139.  
140.         /// <summary>
141.         /// Raises the disable event.
142.         /// </summary>
143.         void OnDisable ()
144.         {
145.             webCamTextureToMatHelper.Dispose ();
146.  
147.             if (rgbMat != null) {
148.                 rgbMat.Dispose ();
149.                 rgbMat = null;
150.             }
151.  
152.             if (des != null) {
153.                 des.Dispose ();
154.                 des = null;
155.             }
156.  
157.             Utils.setDebugMode (false);
158.         }
159.  
160.         /// <summary>
161.         /// Raises the back button event.
162.         /// </summary>
163.         public void OnBackButton ()
164.         {
165.             #if UNITY_5_3 || UNITY_5_3_OR_NEWER
166.             SceneManager.LoadScene ("OpenCVForUnitySample");
167.             #else
168.             Application.LoadLevel ("OpenCVForUnitySample");
169.             #endif
170.         }
171.  
172.         /// <summary>
173.         /// Raises the play button event.
174.         /// </summary>
175.         public void OnPlayButton ()
176.         {
177.             webCamTextureToMatHelper.Play ();
178.         }
179.  
180.         /// <summary>
181.         /// Raises the pause button event.
182.         /// </summary>
183.         public void OnPauseButton ()
184.         {
185.             webCamTextureToMatHelper.Pause ();
186.         }
187.  
188.         /// <summary>
189.         /// Raises the stop button event.
190.         /// </summary>
191.         public void OnStopButton ()
192.         {
193.             webCamTextureToMatHelper.Stop ();
194.         }
195.  
196.         /// <summary>
197.         /// Raises the change camera button event.
198.         /// </summary>
199.         public void OnChangeCameraButton ()
200.         {
201.             webCamTextureToMatHelper.Init (null, webCamTextureToMatHelper.requestWidth, webCamTextureToMatHelper.requestHeight, !webCamTextureToMatHelper.requestIsFrontFacing);
202.         }
203.  
204.         /// <summary>
205.         /// Raises the is showing face points toggle event.
206.         /// </summary>
207.         public void OnIsFlipVerticalToggle ()
208.         {
209.             if (isFlipVerticalToggle.isOn) {
210.                 webCamTextureToMatHelper.flipVertical = true;
211.             } else {
212.                 webCamTextureToMatHelper.flipVertical = false;
213.             }
214.         }
215.  
216.         /// <summary>
217.         /// Raises the is showing axes toggle event.
218.         /// </summary>
219.         public void OnIsFlipHorizontalToggle ()
220.         {
221.             if (isFlipHorizontalToggle.isOn) {
222.                 webCamTextureToMatHelper.flipHorizontal = true;
223.             } else {
224.                 webCamTextureToMatHelper.flipHorizontal = false;
225.             }
226.         }
227.     }
228. }

 

How can we use OpenCVForUnity as an asset package, without requiring camera permission in the final app ?
My app doesn't need to use the camera nor the microphone.

Thanks

 

Hey Enox,

Thanks for such a great asset! I was just wondering if calculations on Gpu are already supported? ( 'Cause I can see this page in your github
http://enoxsoftware.github.io/OpenCVForUnity/doc/html/class_open_c_v_for_unity_1_1_gpu.html but I can't seem to find the classes yet in the package).

Also if you can suggest or recommend a way for us to further speed up ORB - feature detection/extraction/matching in 'Markerless AR Sample', it'd be super awesome!

 

If you do not want to automatically add "CAMERA" permissions to the Android manifest, please comment out "Utils.webCamTextureToMat()" methods in "Assets/OpenCVForUnity/org/opencv/unity/Utils.cs".

 

OpenCVforUnity currently does not support the Cuda modules(GPU).

 

OpenCV for Unity
Released Version 2.1.2

Version changes
2.1.2
[Common]Fixed WebCamTextureToMatHelper.cs.(flipVertical and flipHorizontal flag)

2.1.1
[Common]Fixed OpenCVForUnityMenuItem.cs.(No valid name for platform: 11 Error)
[Common]Added Utils.textureToTexture2D() method.
[Common]Added Mat class operators.
[Common]Added PolygonFilterSample.

 

hi,
thanks for that great asset.
i am trying to write a program (already working in python). but now want to build it in Unity to have some more design freedom

basically trying to detect 4-5 textobjects simultanously (dark grey on white) on live webcam.
but at the moment it wont detect anything + it is super laggy (i5 8gb)
thats what i got so far.

im trying it with matchtemplate at the moment

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
5. using UnityEngine.SceneManagement;
6. #endif
7. using OpenCVForUnity;
8.  
9. namespace OpenCVForUnitySample
10. {
11.         /// <summary>
12.         /// Web cam texture detect circles sample.
13.         /// </summary>
14.         [RequireComponent(typeof(WebCamTextureToMatHelper))]
15.         public class WebCamTextureDetectCirclesSample : MonoBehaviour
16.         {
17.    
18.                 /// <summary>
19.                 /// The colors.
20.                 /// </summary>
21.                 Color32[] colors;
22.  
23.                 /// <summary>
24.                 /// The texture.
25.                 /// </summary>
26.                 Texture2D texture;
27.  
28.                 /// <summary>
29.                 /// The web cam texture to mat helper.
30.                 /// </summary>
31.                 WebCamTextureToMatHelper webCamTextureToMatHelper;
32.  
33.                 /// <summary>
34.                 /// The gray mat.
35.                 /// </summary>
36.                 Mat grayMat;
37.                 Texture2D tempTexture;
38.                 Mat tempMat;
39.  
40.                 // Use this for initialization
41.                 void Start ()
42.                 {
43.                         webCamTextureToMatHelper = gameObject.GetComponent<WebCamTextureToMatHelper> ();
44.                         webCamTextureToMatHelper.Init ();
45.                         tempTexture = Resources.Load ("template_") as Texture2D;
46.                         tempMat = new Mat (tempTexture.height, tempTexture.width, CvType.CV_8UC4);
47.                 }
48.  
49.                 /// <summary>
50.                 /// Raises the web cam texture to mat helper inited event.
51.                 /// </summary>
52.                 public void OnWebCamTextureToMatHelperInited ()
53.                 {
54.                         Debug.Log ("OnWebCamTextureToMatHelperInited");
55.  
56.                         Mat webCamTextureMat = webCamTextureToMatHelper.GetMat ();
57.  
58.                         colors = new Color32[webCamTextureMat.cols () * webCamTextureMat.rows ()];
59.                         texture = new Texture2D (webCamTextureMat.cols (), webCamTextureMat.rows (), TextureFormat.RGBA32, false);
60.  
61.                         grayMat = new Mat (webCamTextureMat.rows (), webCamTextureMat.cols (), CvType.CV_8UC1);
62.  
63.                         gameObject.transform.localScale = new Vector3 (webCamTextureMat.cols (), webCamTextureMat.rows (), 1);
64.  
65.                         Debug.Log ("Screen.width " + Screen.width + " Screen.height " + Screen.height + " Screen.orientation " + Screen.orientation);
66.  
67.                         float width = 0;
68.                         float height = 0;
69.                                    
70.                         width = gameObject.transform.localScale.x;
71.                         height = gameObject.transform.localScale.y;
72.                                    
73.                         float widthScale = (float)Screen.width / width;
74.                         float heightScale = (float)Screen.height / height;
75.                         if (widthScale < heightScale) {
76.                                 Camera.main.orthographicSize = (width * (float)Screen.height / (float)Screen.width) / 2;
77.                         } else {
78.                                 Camera.main.orthographicSize = height / 2;
79.                         }
80.  
81.                         gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
82.  
83.                 }
84.  
85.                 /// <summary>
86.                 /// Raises the web cam texture to mat helper disposed event.
87.                 /// </summary>
88.                 public void OnWebCamTextureToMatHelperDisposed ()
89.                 {
90.                         Debug.Log ("OnWebCamTextureToMatHelperDisposed");
91.                         if (grayMat != null)
92.                                 grayMat.Dispose ();
93.                        
94.                 }
95.  
96.                 // Update is called once per frame
97.                 void Update ()
98.                 {
99.  
100.                         if (webCamTextureToMatHelper.isPlaying () && webCamTextureToMatHelper.didUpdateThisFrame ()) {
101.  
102.                                 Mat rgbaMat = webCamTextureToMatHelper.GetMat ();
103.  
104.                                 Imgproc.cvtColor (rgbaMat, grayMat, Imgproc.COLOR_RGBA2GRAY);
105.  
106.                                 //Create the result mat
107.                                 int result_cols = rgbaMat.cols () - tempMat.cols () + 1;
108.                                 int result_rows = rgbaMat.rows () - tempMat.rows () + 1;
109.                                 Mat result = new Mat (result_rows, result_cols, CvType.CV_32FC1);
110.  
111.                                 int match_method = Imgproc.TM_CCOEFF_NORMED;
112.  
113.  
114.                                 Imgproc.matchTemplate (rgbaMat, tempMat, result, match_method);
115.  
116.                                 Imgproc.threshold (result, result, 0.8, 1.0, Imgproc.THRESH_TOZERO);//threshold = 0.8
117.  
118.                                 for (int i=0; i<result.rows(); i++) {
119.                                     for (int j=0; j<result.cols(); j++) {
120.                                         if (result.get (i, j) [0] > 0) {
121.  
122.                                             Imgproc.rectangle (rgbaMat, new Point (j, i), new Point (j + tempMat.cols (), i + tempMat.rows ()), new Scalar (255, 0, 0, 255), 2);
123.                                             Debug.Log ("value" + result.get (i, j) [0]);
124.                                         }
125.                                     }
126.                                 }
127.                                 Imgproc.putText (rgbaMat, "W:" + rgbaMat.width () + " H:" + rgbaMat.height () + " SO:" + Screen.orientation, new Point (5, rgbaMat.rows () - 10), Core.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar (255, 255, 255, 255), 2, Imgproc.LINE_AA, false);
128.  
129.                                 Utils.matToTexture2D (rgbaMat, texture, colors);
130.                         }
131.  
132.                 }
133.    
134.                 /// <summary>
135.                 /// Raises the disable event.
136.                 /// </summary>
137.                 void OnDisable ()
138.                 {
139.                         webCamTextureToMatHelper.Dispose ();
140.                 }
141.  
142.                 /// <summary>
143.                 /// Raises the back button event.
144.                 /// </summary>
145.                 public void OnBackButton ()
146.                 {
147.                         #if UNITY_5_3 || UNITY_5_3_OR_NEWER
148.             SceneManager.LoadScene ("OpenCVForUnitySample");
149.                         #else
150.                         Application.LoadLevel ("OpenCVForUnitySample");
151.                         #endif
152.                 }
153.  
154.                 /// <summary>
155.                 /// Raises the play button event.
156.                 /// </summary>
157.                 public void OnPlayButton ()
158.                 {
159.                         webCamTextureToMatHelper.Play ();
160.                 }
161.  
162.                 /// <summary>
163.                 /// Raises the pause button event.
164.                 /// </summary>
165.                 public void OnPauseButton ()
166.                 {
167.                         webCamTextureToMatHelper.Pause ();
168.                 }
169.  
170.                 /// <summary>
171.                 /// Raises the stop button event.
172.                 /// </summary>
173.                 public void OnStopButton ()
174.                 {
175.                         webCamTextureToMatHelper.Stop ();
176.                 }
177.  
178.                 /// <summary>
179.                 /// Raises the change camera button event.
180.                 /// </summary>
181.                 public void OnChangeCameraButton ()
182.                 {
183.                         webCamTextureToMatHelper.Init (null, webCamTextureToMatHelper.requestWidth, webCamTextureToMatHelper.requestHeight, !webCamTextureToMatHelper.requestIsFrontFacing);
184.                 }
185.         }
186. }

can you give me hint ?

 

"result.get (i, j) [0]" is slow, so it's better to get the results at once.

Code (CSharp):

1. //                for (int i=0; i<result.rows(); i++) {
2. //                    for (int j=0; j<result.cols(); j++) {
3. //                        if (result.get (i, j) [0] > 0) {
4. //                          
5. //                            Imgproc.rectangle (rgbaMat, new Point (j, i), new Point (j + tempMat.cols (), i + tempMat.rows ()), new Scalar (255, 0, 0, 255), 2);
6. //                            Debug.Log ("value" + result.get (i, j) [0]);
7. //                        }
8. //                    }
9. //                }
10.                 float[] tmpResult = new float[result.total() * result.elemSize()];
11.                 result.get (0, 0, tmpResult);
12.                 int width = result.width();
13.                 int height = result.height();
14.                 for (int i=0; i<height; i++) {
15.                     for (int j=0; j<width; j++) {
16.                         if (tmpResult[i*width + j] > 0) {
17.                          
18.                             Imgproc.rectangle (rgbaMat, new Point (j, i), new Point (j + tempMat.cols (), i + tempMat.rows ()), new Scalar (255, 0, 0, 255), 2);
19. //                            Debug.Log ("value" + result.get (i, j) [0]);
20.                         }
21.                     }
22.                 }

 

hi,
thanks for that great asset.

I need to record an android video using FaceDetection and the VideoWriter class, and I would like to know if OpenCV for Unity is already supported. If so, do you have any examples? Is it possible to use ffmpeg externally to do encode, or use some other library that writes a Mat to a video in android?

 

Perhaps it's just a crazy idea.. do you think a Raspberry Pi3 could handle this? Perhaps an android build.
What do you think about it?

 

VideoWriter class has been implemented from "OpenCV for Unity".
However, this class is not supported in Android.
https://enoxsoftware.com/opencvforunity/documentation/support-modules/
http://forum.unity3d.com/threads/released-opencv-for-unity.277080/page-16#post-2709724

 

I tested on Raspberry Pi 2 (Windows10 loT Core,UWP).
Perhaps, I think that Android Build is also executable on Raspberry Pi (Android OS).

 

We're using the simple blob detection with nice success, but we can't seem to get the number of blobs down. The MinDistanceBetweenBlobs value seems to have no effect, from 0 to 1000 I see no difference in the number of blobs detected. Is this a known issue or am I misunderstanding this value?

 

I changed the code and parameters and tested it as follows. The minDistBetweenBlobs parameter seems to function well.

Code (CSharp):

1. Texture2D imgTexture = Resources.Load ("lena") as Texture2D;

Code (CSharp):

1. %YAML:1.0
2. thresholdStep: 20.
3. minThreshold: 10.
4. maxThreshold: 200.
5. minDistBetweenBlobs: 10.

minDistBetweenBlobs: 10.


minDistBetweenBlobs: 100.

 

Hello,

I am using this method.
static int OpenCVForUnity.Imgproc.connectedComponentsWithStats(Mat image, Mat labels, Mat stats, Mat centroids)
https://enoxsoftware.github.io/Open...mgproc.html#ad87c662fe4cf039dfe5cf1e362c4fb6e

Would you tell me how to get points from Mat of centroids.

 

Hi! Is there any example how I can detect a leg or a tree (vertically oriented object) on picture with OpenCV for Unity? Thanks

 

Hello, I'm trying to reduce the size of the binaries that this library generates for inclusion in the iOS store. Initially my app was about 111M for a single iOS device type but I was able to get it down to 108M by deleting some OpenCV dlls that I know I wasn't using (e.g. ximgproc stitching video bgsegm bioinspired face photo flann optflow plot text videostab xphoto). Unfortunately, this is still past the 100M limit from Apple. While I would enjoy using this library when releasing if I can't shrink the app it will not be possible. Do you have any tips for how to reduce file size? Particularly those that helped *you* get to the smaller file sizes on the app store?

 

Please refer to this code.

Code (CSharp):

1. using UnityEngine;
2. using System.Collections.Generic;
3.  
4. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
5. using UnityEngine.SceneManagement;
6. #endif
7. using OpenCVForUnity;
8.  
9. namespace OpenCVForUnitySample
10. {
11.     /// <summary>
12.     /// ConnectedComponents Sample
13.     /// </summary>
14.     public class ConnectedComponentsSample : MonoBehaviour
15.     {
16.  
17.         // Use this for initialization
18.         void Start ()
19.         {
20.             Texture2D imgTexture = Resources.Load ("matchshapes") as Texture2D;
21.            
22.             Mat srcMat = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC1);
23.            
24.             Utils.texture2DToMat (imgTexture, srcMat);
25.             Debug.Log ("srcMat.ToString() " + srcMat.ToString ());
26.  
27.             Mat dstMat = new Mat (srcMat.size (), CvType.CV_8UC3);
28.  
29.             //labeling
30.             Mat labels = new Mat ();
31.             Mat stats = new Mat ();
32.             Mat centroids = new Mat ();
33.             int total = Imgproc.connectedComponentsWithStats (srcMat, labels, stats, centroids);
34.  
35.             Debug.Log ("labels.ToString() " + labels.ToString ());
36.             Debug.Log ("stats.ToString() " + stats.ToString ());
37.             Debug.Log ("centroids.ToString() " + centroids.ToString ());
38.             Debug.Log ("total " + total);
39.            
40.             // determine drawing color
41.             List<Scalar> colors = new List<Scalar> (total);
42.             colors.Add (new Scalar (0, 0, 0));
43.             for (int i = 1; i < total; ++i) {
44.                 colors.Add (new Scalar (Random.Range (0, 255), Random.Range (0, 255), Random.Range (0, 255)));
45.             }
46.            
47.             // draw labels
48.             for (int i = 0; i < dstMat.rows(); ++i) {
49.                 for (int j = 0; j < dstMat.cols(); ++j) {
50.                     Scalar color = colors [(int)labels.get (i, j) [0]];
51.                     dstMat.put (i, j, color.val [0], color.val [1], color.val [2]);
52.                 }
53.             }
54.            
55.             // draw rectangle
56.             for (int i = 1; i < total; ++i) {
57.  
58.                 int x = (int)stats.get (i, Imgproc.CC_STAT_LEFT) [0];
59.                 int y = (int)stats.get (i, Imgproc.CC_STAT_TOP) [0];
60.                 int height = (int)stats.get (i, Imgproc.CC_STAT_HEIGHT) [0];
61.                 int width = (int)stats.get (i, Imgproc.CC_STAT_WIDTH) [0];
62.  
63.                 OpenCVForUnity.Rect rect = new OpenCVForUnity.Rect (x, y, width, height);
64.  
65.                 Imgproc.rectangle (dstMat, rect.tl (), rect.br (), new Scalar (0, 255, 0), 2);
66.             }
67.            
68.             // draw centroids
69.             for (int i = 1; i < total; ++i) {
70.  
71.                 int x = (int)centroids.get (i, 0) [0];
72.                 int y = (int)centroids.get (i, 1) [0];
73.  
74.                 Imgproc.circle (dstMat, new Point (x, y), 3, new Scalar (255, 0, 0), -1);
75.             }
76.            
77.             // draw index of label
78.             for (int i = 1; i < total; ++i) {
79.  
80.                 int x = (int)stats.get (i, Imgproc.CC_STAT_LEFT) [0];
81.                 int y = (int)stats.get (i, Imgproc.CC_STAT_TOP) [0];
82.  
83.                 Imgproc.putText (dstMat, "" + i, new Point (x + 5, y + 15), Core.FONT_HERSHEY_COMPLEX, 0.5, new Scalar (255, 255, 0), 2);
84.             }
85.            
86.            
87.             Texture2D texture = new Texture2D (dstMat.cols (), dstMat.rows (), TextureFormat.RGBA32, false);
88.            
89.             Utils.matToTexture2D (dstMat, texture);
90.            
91.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
92.         }
93.  
94.  
95.         // Update is called once per frame
96.         void Update ()
97.         {
98.  
99.         }
100.        
101.         public void OnBackButton ()
102.         {
103.             #if UNITY_5_3 || UNITY_5_3_OR_NEWER
104.             SceneManager.LoadScene ("OpenCVForUnitySample");
105.             #else
106.             Application.LoadLevel ("OpenCVForUnitySample");
107.             #endif
108.         }
109.     }
110. }

 

Thank you Enox !

 

Unfortunately, there is no such sample at the moment.

 

Unfortunately, I don't know other ways to reduce the size of iOS app.