[RELEASED] OpenCV for Unity

@Enox Software
Can I share C# array memory with Mat?
For example:
byte [] buffer = new byte[size];
Mat bufferMat = new Mat(bufferPtr, cols, rows, type);
I need to do something like buffer(i) = 0,and something like blur(bufferMat...)

 

I do not know whether it is possible to share the C # array memory and the mat data.
but, You can get the data pointer of Mat using Mat.dataAddr ().
http://enoxsoftware.github.io/OpenC..._1_mat.html#ac576026f06bc7b3d5e9ef5e6e442061c

 

Hi Enox

I try to do an easy example. It will draw keypoints on video from webcam.
I see video. Detector return a lot of key points, but drawKeypoints don't draw them.

Please help me...
This is my code

Code (CSharp):

1.  
2. Mat rgbaMat = webCamTextureToMatHelper.GetMat ();
3. FeatureDetector detector = FeatureDetector.create(FeatureDetector.AKAZE);
4. DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.AKAZE);
5. MatOfKeyPoint keypoints1 = new MatOfKeyPoint();
6. Mat descriptors1 = new Mat();
7.  
8.                              
9. detector.detect(rgbaMat, keypoints1);
10. extractor.compute(rgbaMat, keypoints1, descriptors1);
11. Mat resultImg = rgbaMat.clone();
12.              
13.  
14.              
15. Features2d.drawKeypoints(rgbaMat, keypoints1, resultImg, new Scalar(255, 0, 0, 20), Features2d.DRAW_RICH_KEYPOINTS);
16.            
17. Imgproc.putText (resultImg, "Keypoints:" + keypoints1.rows (), new Point (5, rgbaMat.rows () - 10), Core.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar (255, 255, 255, 255), 2, Imgproc.LINE_AA, false);
18. Utils.matToTexture2D (resultImg, texture, colors);

 

This error was displayed using Utils.setDebugMode(true) method.

It seems the CvType of Mat must be CV_8SC3 .

Code (CSharp):

1.             Mat rgbaMat = webCamTextureToMatHelper.GetMat ();
2.             Mat rgbMat = new Mat(rgbaMat.rows(), rgbaMat.cols(), CvType.CV_8SC3);
3.             Imgproc.cvtColor(rgbaMat, rgbMat, Imgproc.COLOR_RGBA2RGB);
4.  
5.             FeatureDetector detector = FeatureDetector.create(FeatureDetector.AKAZE);
6.             DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.AKAZE);
7.             MatOfKeyPoint keypoints1 = new MatOfKeyPoint();
8.             Mat descriptors1 = new Mat();
9.            
10.            
11.             detector.detect(rgbMat, keypoints1);
12.             extractor.compute(rgbMat, keypoints1, descriptors1);
13.             Mat resultImg = rgbMat.clone();
14.            
15.            
16.            
17.             Features2d.drawKeypoints(rgbMat, keypoints1, resultImg, new Scalar(255, 0, 0), Features2d.DRAW_RICH_KEYPOINTS);
18.            
19.             Imgproc.putText (resultImg, "Keypoints:" + keypoints1.rows (), new Point (5, rgbMat.rows () - 10), Core.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar (255, 255, 255), 2, Imgproc.LINE_AA, false);
20.             Utils.matToTexture2D (resultImg, texture, colors);

 

Just bought this plugin. Runs fine in the editor, but when launching an app containing this plugin as windows standalone it immediately crashes in

app.exe!RegisterPlugin(void *,struct UnityPluginFunctions &) Unknown
app.exe!RegisterPlugin(void *,struct UnityPluginFunctions &) Unknown
> app.exe!FindAndLoadUnityPlugin(char const *,void * *) Unknown
...

 

What is the environment you tested?
OS version
Unity version
OpenCV for Unity version

 

Greetings. I recently bought this asset and I am wondering if there is anything I am missing...
I made a test PNG image and I am splitting into different channels, but when I make a Mat from a Texture, which I eventually put back in a new texture to display, it has a small offset, and it is flipped from my original image.
I have used Core.flip, but it is very inconsistent.

I am learning OpenCV on the go, that is why I am doing something so simple, yet, I am dazzled at not being able to even split channels correctly.

 

This code works fine on my envionment.
Windows8.1 Unity Editor
Unity5.0.0
OpenCV for Unity 2.0.8

Code (CSharp):

1.         void Start ()
2.         {
3.  
4.             Texture2D imgTexture = Resources.Load ("lena") as Texture2D;
5.  
6.             Mat imgMat = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC4);
7.  
8.             Utils.texture2DToMat (imgTexture, imgMat);
9.             Debug.Log ("imgMat.ToString() " + imgMat.ToString ());
10.  
11.             List<Mat> mv = new List<Mat>();
12.             Core.split(imgMat, mv);
13.  
14.  
15.             Texture2D texture = new Texture2D (imgMat.cols (), imgMat.rows (), TextureFormat.RGBA32, false);
16.  
17.             Utils.matToTexture2D (mv[0], texture);
18.  
19.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
20.  
21.         }

 

Hi!!
I have a question.
In Texture2DFaceTrackerSample, what is the order of red point?
jar : 0 ~ 14
but... what next?

 

You can visualize by this code.
https://github.com/EnoxSoftware/Fac...ster/Scripts/FaceTrackerARSample.cs#L464-L470

 

Mat resultImg = new Mat();
Debug.Log(webcamMat.rows()); // 480
Debug.Log(webcamKeypoints.rows()); // 500
Features2d.drawKeypoints(webcamMat, webcamKeypoints, resultImg);
Debug.Log(resultImg.rows()); // 0

Why result image returns 0 rows mat? Shouldn't it return mat with keypoints draw on it?

 

Hi, i'm having trouble to use Core.perspectiveTransform.
The output is always empty without cols or rows.

my code is:

Mat H = Calib3d.findHomography(obj, scene);

MatOfPoint2f obj_corners = new MatOfPoint2f ();
obj_corners.put (0, 0, new double[] { 0, 0 });
obj_corners.put (1, 0, new double[] { srcMat.cols(), 0 });
obj_corners.put (2, 0, new double[] { srcMat.cols(), srcMat.rows() });
obj_corners.put (3, 0, new double[] { 0, srcMat.rows() });

MatOfPoint2f scene_corners = new MatOfPoint2f ();
Core.perspectiveTransform(obj_corners, scene_corners, H);


any ideas?

 

You can use:
Mat obj_corners = new Mat(4, 1, CvType.CV_32FC2);
Mat scene_corners = new Mat(4, 1, CvType.CV_32FC2);

Instead of:
MatOfPoint2f obj_corners = new MatOfPoint2f ();
MatOfPoint2f scene_corners = new MatOfPoint2f ();

I making same algoritm, for me it draws corners, but drawing it in really wrong places.

 

I think perhaps it is the same problem with this post.
https://forum.unity3d.com/threads/released-opencv-for-unity.277080/page-19#post-2836393

 

Hello, we wanted to track two small colored objects in Unity 3D for an Android application. As we are novices at programming, our team bought the open cv for Unity plugin.
The sample scenes of Multiple Object Tracking based on colour sample works fine. However I still do not understand how can I apply those functions in my application.
Is there any tutorial for this?
Thank you.
Best,
Yesh.

 

Hi Enox,

Im having probloms loading xml detector on android. I followed the instruction and put my xml detector under StreamingAssets and my code:
string path = Utils.getFilePath("detector.xml");// Application.streamingAssetsPath + "/aGest.xml";

CascadeClassifier cascade = new CascadeClassifier(path);

UnityEngine.Debug.Log(path);
if(cascade.empty())
UnityEngine.Debug.Log("no detector empty");

It indicates that the 'cascade' is empty and nothing happens.

 

https://github.com/opencv/opencv/issues/7223
I think it is the same issue as this.

Do the following codes work fine?

Code (CSharp):

1. CascadeClassifier cascade = new CascadeClassifier();
2. cascade.load(path);

 

I made Multiple Object Tracking based on colour Sample based on this code.

 

I got shocked just now.
Because the face ratio is exactly same with others.
I got point in my face. and my firends too.
and then, I compared my ratio with firend's ratio.
But the ratio is exactly same.
All points are exactly on the same position at each ratio.
for example, gorilla's colNoseLength/rowNoseLength == human's colNoseLength/rowNoseLength.
and my friend's fatty colNoseLength/rowNoseLength == baby's 's colNoseLength/rowNoseLength.
there is a no diffrent. super exactly same ratio!. unbelievable!!
this asset just finds face color, and then calcaulates the face length and then, puts the decided face point. finally say that
"Hey! I got it! I got your face point! look at the number of each point!"
but the point is not your face point but the decided point which is adjusted by your face length.
Maybe it is possible that I'm wrong. but the truth is that the point is not your face point, but the fake same point.
what is the meaning of point number....?
It just same!!!
the point number is meaningless.
Did I miss?
How can I get the real position on the face?

 

Are you posting about FaceTracker Sample?

 

yes. yes.
I'm Posting about FaceTracker Sample.
Yeah.

I modified addPoints method in FaceTracker.cs bellow.

public void addPoints (MatOfRect rects)
{
points.AddRange (detector.convertMatOfRectToPoints (rects));

Debug.Log (detector.convertMatOfRectToPoints (rects).Count);

//If only one face is detected
if (detector.convertMatOfRectToPoints (rects).Count == 1) {

//Nose Ratio
double distance_RowNose = 0;
double distance_ColNose = 0;
distance_RowNose = detector.convertMatOfRectToPoints (rects) [0] [42].x - detector.convertMatOfRectToPoints (rects) [0] [40].x;
distance_ColNose = (detector.convertMatOfRectToPoints (rects) [0] [37].y - detector.convertMatOfRectToPoints (rects) [0] [41].y) * -1;
Debug.Log ("Nose row : "+distance_RowNose);
Debug.Log ("Nose column : "+distance_ColNose);
Debug.Log("Nose Ratio Is"+(distance_ColNose/distance_RowNose));

if(distance_ColNose/distance_RowNose == 0.738612122695148)
{
Debug.Log("Ratio is same!");
}
//Nose Ratio

//
}
}

 

Hi Enox, no, still no detector is loaded.
the following code indicat that file doesnt exists is that a right way to test it?

string path = Utils.getFilePath("detector.xml");
if(File.Exists(path))
UnityEngine.Debug.Log("file exists");
else
UnityEngine.Debug.Log("file doesnt exists");

 

At the time when addPoints() method is called, the process of fitting the points to the face has not yet been invoked.
When you call the track() method, points are fitted to the face.

 

Thank you!
God bless you!
forgive me my ignorance and rudeness.
Thank you very very much.
I hope joy and peace abide in you all.
thanks for reply!
Thanks!
Very very helpful!

 

Hello, I was bought your asset on end of August 2016.Unfortunately I lost my notebook on October. I just bought a new notebook this month and I try to re-download your assets in the asset store, but always failed. I see that you have updated your asset on October. Maybe you can help give me a clue to the problem I have. Or problems happen at the unity? Just for information, I re-download another assets and no problems. Only when I download your assets the error occurred.

 

Hello,

I'm trying to work with the ThinPlateSplineShapeTransformer class but Unity craches when I execute my code:

MatOfPoint2f image_points = new MatOfPoint2f();
MatOfPoint2f image_points_warped = new MatOfPoint2f();
MatOfDMatch matches = new MatOfDMatch();
image_points.fromArray (
new Point (0, 0),
new Point (0, 1),
new Point (1, 0)
);
image_points_warped.fromArray (
new Point (0, 0),
new Point (0, 1),
new Point (2, 0)
);
matches.fromArray (
new DMatch (0, 0, 0),
new DMatch (1, 1, 0),
new DMatch (2, 2, 0)
);
OpenCVForUnity.ThinPlateSplineShapeTransformer tps = new OpenCVForUnity.ThinPlateSplineShapeTransformer(IntPtr.Zero);
tps.estimateTransformation(image_points, image_points_warped, matches);//Crashes here
//tps.applyTransformation(mat);//Applies the transformation to mat, commented until crash is resolved

The crash is due to tps.estimateTransformation(). The beginning of the error.log file is:

Unity Editor [version: Unity 5.4.1f1_649f48bbbf0f]

opencvforunity.dll caused an Access Violation (0xc0000005)
in module opencvforunity.dll at 0033:b4363788.

Error occurred at 2016-11-22_163640.
C:\Program Files\Unity_5\Editor\Unity.exe, run by Lydorn.
29% memory in use.
16338 MB physical memory [11554 MB free].
18770 MB paging file [13127 MB free].
134217728 MB user address space [134215894 MB free].
Read from location 00000008 caused an access violation.

I can copy the whole file if necessary.
So am I using the class correctly? Do you know how I can resolve the crash?

Thank you very much for your help!

 

Since I can not deal with this AssetStore issue, please contact Unity's AssetStore team.

 

Code (CSharp):

1. OpenCVForUnity.ThinPlateSplineShapeTransformer tps = Shape.createThinPlateSplineShapeTransformer(0);

https://github.com/opencv/opencv/issues/7084

 

Thank you for the quick reply! I'll try that.

 

Hi there,

I would like to know does this version support GPU module (OpenCL) in Android or not?

Thanks.

 

Hello again Enox,

So I tried to use ThinPlateSplineShapeTransformer with your correction and it doesn't crash anymore. When I applied the transform with warpImage the result is an image with every pixel set to one color (the mean color it seems). So I tried applying the transform to a few points with applyTransform and it always returns an array with one point: [0, 0]. Here is my code:

MatOfPoint image_points = new MatOfPoint(
new Point (0, 0),
new Point (640, 0),
new Point (0, 720),
new Point (1280, 720),
new Point (1280/2, 720/2)
);
MatOfPoint image_points_warped = new MatOfPoint (
new Point (0, 0),
new Point (640, 0),
new Point (0, 720),
new Point (1280, 720),
new Point (1280/4, 720/4)
);
MatOfDMatch matches = new MatOfDMatch (
new DMatch (0, 0, 0),
new DMatch (1, 1, 0),
new DMatch (2, 2, 0),
new DMatch (3, 3, 0),
new DMatch (4, 4, 0)
);
Debug.Log(image_points.dump());
Debug.Log(image_points_warped.dump());
Debug.Log(matches.dump());

OpenCVForUnity.ThinPlateSplineShapeTransformer tps = Shape.createThinPlateSplineShapeTransformer();
tps.estimateTransformation(image_points, image_points_warped, matches);

MatOfPoint test_points = new MatOfPoint(
new Point (100, 100),
new Point (100, 50)
);
MatOfPoint2f test_points_warped = new MatOfPoint2f();
tps.applyTransformation(test_points, test_points_warped);
Debug.Log(test_points_warped.dump());

Do you know where the problem may lie this time?

Thank you,

Nicolas Girard

 

Since this package is a clone of OpenCV Java, you are able to use the same API as OpenCV Java 3.1.0.
"OpenCV for Unity" does not support OpenCL module.

 

It seems necessary to transpose Mat.
http://stackoverflow.com/questions/32207085/shape-transformers-and-interfaces-opencv3-0

Code (CSharp):

1.             Utils.setDebugMode(true);
2.  
3.  
4.             Texture2D imgTexture = Resources.Load ("input") as Texture2D;
5.  
6.             Mat img = new Mat (imgTexture.height, imgTexture.width, CvType.CV_8UC4);
7.  
8.             Utils.texture2DToMat (imgTexture, img);
9.             Debug.Log ("imgMat.ToString() " + img.ToString ());
10.  
11.  
12.             OpenCVForUnity.ThinPlateSplineShapeTransformer tps = Shape.createThinPlateSplineShapeTransformer (0);
13.             MatOfPoint2f sourcePoints = new MatOfPoint2f (
14.                 new Point (0, 0),
15.                 new Point (799, 0),
16.                 new Point (0, 599),
17.                 new Point (799, 599)
18.             );
19.             MatOfPoint2f targetPoints = new MatOfPoint2f (
20.                 new Point (100, 0),
21.                 new Point (799, 0),
22.                 new Point (0, 599),
23.                 new Point (799, 599)
24.             );
25.             MatOfDMatch matches = new MatOfDMatch (
26.                 new DMatch (0, 0, 0),
27.                 new DMatch (1, 1, 0),
28.                 new DMatch (2, 2, 0),
29.                 new DMatch (3, 3, 0)
30.             );
31.  
32.  
33.             //http://stackoverflow.com/questions/32207085/shape-transformers-and-interfaces-opencv3-0
34.             Core.transpose(sourcePoints, sourcePoints);
35.             Core.transpose(targetPoints, targetPoints);
36.  
37.             Debug.Log ("sourcePoints " + sourcePoints.ToString());
38.             Debug.Log ("targetPoints " + targetPoints.ToString());
39.  
40. //            tps.estimateTransformation (sourcePoints, targetPoints, matches);
41.             tps.estimateTransformation (targetPoints, sourcePoints, matches);
42.  
43.             MatOfPoint2f transPoints = new MatOfPoint2f ();
44.             tps.applyTransformation (sourcePoints, transPoints);
45.  
46.             Debug.Log ("sourcePoints " + sourcePoints.dump());
47.             Debug.Log ("targetPoints " + targetPoints.dump());
48.             Debug.Log ("transPoints " + transPoints.dump());
49.  
50.  
51.             Mat res = new Mat();
52.  
53.             tps.warpImage( img, res);
54.  
55.  
56.             Texture2D texture = new Texture2D (res.cols (), res.rows (), TextureFormat.RGBA32, false);
57.  
58.             Utils.matToTexture2D (res, texture);
59.  
60.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
61.  
62.  
63.             Utils.setDebugMode(false);
64.  

 

OpenCV for Unity
Released Version 2.0.9

Version changes
2.0.9
[WebGL]Added WebGL(beta) support.(Unity5.3 or later)

WebGL Demo
https://enoxsoftware.github.io/OpenCVForUnity/webgl_sample/index.html

 

How to merge two images into one by using the OpenCV in Unity?
The two images are texture2ds. One is the background,jpg image; the other is the foreground,png image. Now I want to merge them into one. If I use “SetPixel” to make it, the frame rate will be very low. So, is there any methods by using the OpenCV to make it?

 

estimateAffine3D
I have a question concerning the use of estimateAffine3D. If I input 2 matrices srcMat and dstMat, set the outMat as an empty Matrix full of 0.0f and run it, the outMat doesn't change and is still full of 0s.

Code (CSharp):

1. Calib3d.estimateAffine3D(srcMat, dstMat, outMat, inliers);

The problem seems similar to http://stackoverflow.com/questions/31703059/python-cv2-estimateaffine3d-why-do-my-output-arguments-remain-unchanged.
How do I adapt that to c# with openCV unity?

Code (CSharp):

1.  Mat outMat;
2.         outMat = new Mat(4, 4, CvType.CV_64FC1);
3.         outMat.put(0, 0, 0.0f);
4.         outMat.put(0, 1, 1.0f);
5.         outMat.put(0, 2, 0.0f);
6.         outMat.put(1, 0, 0.0f);
7.         outMat.put(1, 1, 0.0f);
8.         outMat.put(1, 2, 1.0f);
9.         outMat.put(2, 0, 0.0f);
10.         outMat.put(2, 1, 0.0f);
11.         outMat.put(2, 2, 0.0f);
12.         outMat.put(3, 0, 0.0f);
13.         outMat.put(3, 1, 0.0f);
14.         outMat.put(3, 2, 0.0f);
15.         outMat.put(0, 3, 0.0f);
16.         outMat.put(1, 3, 0.0f);
17.         outMat.put(2, 3, 0.0f);
18.         outMat.put(3, 3, 0.0f);
19.  
20.         Mat inliers = new Mat(3, 1, CvType.CV_32FC1);
21.  
22.         Mat srcMat;
23.         srcMat = new Mat(3, 3, CvType.CV_64FC1);
24.         srcMat.put(0, 0, 0.0f);
25.         srcMat.put(0, 1, 1.0f);
26.         srcMat.put(0, 2, 0.0f);
27.         srcMat.put(1, 0, 0.0f);
28.         srcMat.put(1, 1, 0.0f);
29.         srcMat.put(1, 2, 1.0f);
30.         srcMat.put(2, 0, 0.0f);
31.         srcMat.put(2, 1, 0.0f);
32.         srcMat.put(2, 2, 0.0f);
33.         Debug.Log("srcMat" + srcMat.dump());
34.  
35.         Mat dstMat;
36.         dstMat = new Mat(3, 3, CvType.CV_64FC1);
37.         dstMat.put(0, 0, 0.0f);
38.         dstMat.put(0, 1, 0.0f);
39.         dstMat.put(0, 2, 0.0f);
40.         dstMat.put(1, 0, 0.0f);
41.         dstMat.put(1, 1, 1.0f);
42.         dstMat.put(1, 2, 1.0f);
43.         dstMat.put(2, 0, 1.0f);
44.         dstMat.put(2, 1, 1.0f);
45.         dstMat.put(2, 2, 0.0f);
46.         Debug.Log("dstMat" + dstMat.dump());
47.  
48.         Debug.Log("outMat" + outMat.dump());
49.         Calib3d.estimateAffine3D(srcMat, dstMat, outMat, inliers);
50.         Debug.Log("outMat" + outMat.dump());

 

It can be accomplished with the following code.

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. #if UNITY_5_3 || UNITY_5_3_OR_NEWER
5. using UnityEngine.SceneManagement;
6. #endif
7. using OpenCVForUnity;
8.  
9. namespace OpenCVForUnitySample
10. {
11.     /// <summary>
12.     /// Texture2D to mat sample.
13.     /// </summary>
14.     public class CopyToMatSample : MonoBehaviour
15.     {
16.  
17.         // Use this for initialization
18.         void Start ()
19.         {
20.             Utils.setDebugMode(true);
21.  
22.             //load image and convert Texture2D to Mat.
23.             Texture2D bgTexture = Resources.Load ("background") as Texture2D;
24.             Mat bgMat = new Mat (bgTexture.height, bgTexture.width, CvType.CV_8UC3);
25.             Utils.texture2DToMat (bgTexture, bgMat);
26.             Debug.Log ("bgMat.ToString() " + bgMat.ToString ());
27.  
28.             Texture2D monalisaTexture = Resources.Load ("monalisa") as Texture2D;
29.             Mat monalisaMat = new Mat (monalisaTexture.height, monalisaTexture.width, CvType.CV_8UC3);
30.             Utils.texture2DToMat (monalisaTexture, monalisaMat);
31.             Debug.Log ("monalisaMat.ToString() " + monalisaMat.ToString ());
32.  
33.             Texture2D maskTexture = Resources.Load ("mask") as Texture2D;
34.             Mat maskMat = new Mat (maskTexture.height, maskTexture.width, CvType.CV_8UC1);
35.             Utils.texture2DToMat (maskTexture, maskMat);
36.             Debug.Log ("maskMat.ToString() " + maskMat.ToString ());
37.  
38.             //copy
39.             monalisaMat.copyTo( bgMat, maskMat);
40.  
41.             //convert Mat to Texture2D.
42.             Texture2D texture = new Texture2D (bgMat.cols (), bgMat.rows (), TextureFormat.RGBA32, false);
43.             Utils.matToTexture2D (bgMat, texture);
44.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
45.  
46.             Utils.setDebugMode(false);
47.         }
48.    
49.         // Update is called once per frame
50.         void Update ()
51.         {
52.    
53.         }
54.  
55.         public void OnBackButton ()
56.         {
57.             #if UNITY_5_3 || UNITY_5_3_OR_NEWER
58.             SceneManager.LoadScene ("OpenCVForUnitySample");
59.             #else
60.             Application.LoadLevel ("OpenCVForUnitySample");
61.             #endif
62.         }
63.     }
64. }

 

Ok, got it, prescribed the wrong matrix size. Has to be

Code (CSharp):

1. srcMat = new Mat(7, 3, CvType.CV_64FC1);

then it works.

 

Hello!
Are there any special requirements for the mask image? Because when I change the mask image, the result doesn’t meet the purpose.

 

Hi! Is it possible to remove background from device camera feed when looking at a person or several persons with open cv and your package? Thanks

Please see the attached examples

 

Mask image must be two colors(black and white).
This image that I converted to two colors worked well.

 

I think the following sample code will be helpful.
BackgroundSubtractorMOG2Sample
GrabCutSample

 

Hi,
I'm trying to get face tracking on a video working. We are using AVPro to retrieve each frame of the video and it returns a Texture2D, however the pixel format is BGRA32. When trying to convert the texture to mat using Utils.texture2DToMat and Utils.matToTexture2D (like in Texture2DToMatSample.cs) it only returns a black texture. I would appreciate any help or code snippet that demonstrates how to convert a Texture2D with BGRA32 to a mat and back.

Thank you,
André

 

https://github.com/EnoxSoftware/AVProWithOpenCVForUnitySample

Code (CSharp):

1. targetTexture = new Texture2D (info.GetVideoWidth (), info.GetVideoHeight (), TextureFormat.ARGB32, false);
2. texture = new Texture2D (info.GetVideoWidth (), info.GetVideoHeight (), TextureFormat.RGBA32, false);
3. colors = new Color32[texture.width * texture.height];
4. rgbaMat = new Mat (texture.height, texture.width, CvType.CV_8UC4);
5.  
6. //Convert AVPro's Texture to Texture2D
7. mediaPlayer.ExtractFrame(targetTexture, timeSeconds, _accurateSeek, _timeoutMs);
8.  
9. //Convert Texture2D to Mat
10. Utils.texture2DToMat (targetTexture, rgbaMat);
11.  
12. Imgproc.putText (rgbaMat, "AVPro With OpenCV for Unity Sample", new Point (50, rgbaMat.rows () / 2), Core.FONT_HERSHEY_SIMPLEX, 2.0, new Scalar (255, 0, 0, 255), 5, Imgproc.LINE_AA, false);
13.  
14. //Convert Mat to Texture2D
15. Utils.matToTexture2D (rgbaMat, texture, colors);
16.  

 

Hi Enox,

I am using opencvforunity in a project. As I am only using few opencv functions, is there anyway to include the only used modules in the final package?

For iOS, I can change the head files and remove the not needed packages in Xcode project, but still didn't find a way for Android yet - seems we have to add the 10M .so file?

Thank you so much!
Sheng

 

Unfortunately, there is no way to include only the used modules.

 

OpenCV for Unity
Released Version 2.1.0

Version changes
2.1.0
[Common]Fixed WebCamTextureToMatHelper class.
[Common]Added Utils.getVersion().
[Common]Fixed Utils.getFilePathAsync().

 

Hi i m working on hand detection using YCrCb skin color detection using unity3d and opencv.
May I know how can i convert these lines of codes?

Code (CSharp):

1. for(int i = 0; i < input_image.cols; i++)  
2.             for(int j = 0; j < input_image.rows; j++){
3.             Vec3b ycrcb = ycrcb_image.at<Vec3b>(j, i);
4.             if(skinCrCbHist.at<uchar>(ycrcb[1], ycrcb[2]) > 0)
5.                 output_mask.at<uchar>(j, i) = 255;
6.         }

Many Thanks!!!!

 

Since this asset is a clone of OpenCV Java, you are able to use the same API as OpenCV Java.

Code (CSharp):

1.             byte[] ycrcb = new byte[3];
2.             for (int i = 0; i < input_image.cols(); i++)
3.                 for (int j = 0; j < input_image.rows(); j++) {
4.                     ycrcb_image.get (j, i, ycrcb);
5.                     if (skinCrCbHist.get (ycrcb [1], ycrcb [2]) [0] > 0)
6.                         output_mask.get (j, i) [0] = 255;
7.                 }

 

When setting MenuItem[Tools/OpenCV for Unity/Set Plugin Import Settings], it seems that the following error occurs on Unity5.5 or later.

No valid name for platform: 11
UnityEditor.PluginImporter:SetCompatibleWithPlatform(BuildTarget, Boolean)
OpenCVForUnity.OpenCVForUnityMenuItem:SetPlugins(String[], Dictionary`2, Dictionary`2) (at Assets/OpenCVForUnity/Editor/OpenCVForUnityMenuItem.cs:193)
OpenCVForUnity.OpenCVForUnityMenuItem:SetPluginImportSettings() (at Assets/OpenCVForUnity/Editor/OpenCVForUnityMenuItem.cs:124)


To fix, Replace line 192 - line 194 with the following code.

from

Code (CSharp):

1.                                                 foreach (BuildTarget target in Enum.GetValues(typeof(BuildTarget))) {
2.                                                     if ((int)target != -1)pluginImporter.SetCompatibleWithPlatform (target, false);
3.                                                 }

to

Code (CSharp):

1.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.Android, false);
2.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.iOS, false);
3.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneWindows, false);
4.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneWindows64, false);
5.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneOSXIntel, false);
6.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneOSXIntel64, false);
7.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneOSXUniversal, false);
8.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneLinux, false);
9.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneLinux64, false);
10.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.StandaloneLinuxUniversal, false);
11.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.WSAPlayer, false);
12.                         pluginImporter.SetCompatibleWithPlatform (BuildTarget.WebGL, false);

This bug will be fixed in the next version.