[RELEASED] OpenCV for Unity

hi @EnoxSoftware , how do I read an ipCamera feed in OpenCVForUnity?

I've tried the following, but no luck.

Code (CSharp):

1.   VideoCapture camera0 ;
2. void Start()
3.     {
4.      camera0 = new VideoCapture("http://192.168.0.21:8081/m.mjpg");
5.     }
6. void Update() {
7.  
8.         if (camera0.isOpened())
9.         {
10.             Debug.Log("success");
11.         }
12.     }
13.  

Could you please point me in the right direction,
Thanks.

 

Thanks for the last answer.

I am currently trying to locate the four corners of a quadrilateral shape. I have a pretty good contour previously using hsv and erode/dilate to filter out the noise.



I applied the convexhull function first to make it convex and then used approxPolyDP. I iterated the approxPolyDPfunction until I got an eps that can produce a quadrilateral.

Code (CSharp):

1. while (approxCurve.rows()>4)
2.                 {
3.                     eps = eps + 0.001;
4.                     Imgproc.approxPolyDP(new MatOfPoint2f(contours[i].toArray()), approxCurve, eps, true);
5.                 }

I then connected the diagonals to find the center.

However, I was unable to locate the corners very accurately in most situations. Is there a way to improve this?

(previous contours are for the color yellow)


Thanks a lot.

 

Unfortunately, streaming video playback is supported only on Windows. To play streaming file, ffmpeg.dll is required.
1)Download "OpenCV for Windows Version 3.4.2"(http://opencv.org/downloads.html).
2)Copy "opencv_ffmpeg342_64.dll" to Project Folder.


I succeeded in playing this file.
capture.open ("http://archive.org/download/SampleMpeg4_201307/sample_mpeg4.mp4");

 

Have you already tested the Moment () method and boxPoints () method?
https://opencv-python-tutroals.read...res/py_contour_features.html#contour-features

 

Hey,
Thanks for the response, the two methods mentioned might not be perfect in this scenario. In this case, I would hope the program to recognize a tilted rectangualr, which can be any form of quadrilaterals on the image.

For the previous problem, the corner detection got a lot better after I switch the dilate/erode mode to cross instead of square.

Thanks again for the fast reply.

 

Hi
I'm trying to convert screen captured texture to mat.
Screen captured texture format is BGRA32.
When trying to convert the texture to mat using Utils.texture2DToMat, it only returns a white texture.
Is there a way to convert the texture of BGRA32 to mat?

 

Hey try converting textureToTexture2D before texture2DToMat.
I had a similar issue which got solved by this.

 

It worked! Thank you!

 

Hi
I want to save the descriptor (Mat) in a file.
As below.

Code (CSharp):

1. FileStorage fs("Keypoints.yml", FileStorage::WRITE);
2. write(fs, "keypoints_1", keypoints_1);
3. write(fs, "descriptors_1", tempDescriptors_1);
4. fs.release();

Is there a way to call this with Open CV for Unity?

 

By the looks of it you're trying to save/load a yaml file?

https://assetstore.unity.com/packages/tools/integration/yamldotnet-for-unity-36292
Try the link above Or,
You could find one of many parsers for this. If the file format it different then look out for the appropriate parser.

 

it is possible to make recognition of some specific objects with opencv asset of unity

 

Solved!
I used MatOfByte to save it with byte[].

 

By changing cascade file or model file, you can detect specific objects.
https://github.com/EnoxSoftware/Ope...nExample/FaceDetectionWebCamTextureExample.cs
https://github.com/EnoxSoftware/Ope...dnn/DnnObjectDetectionWebCamTextureExample.cs

 

Hello everyone and @EnoxSoftware ,
does anyone know the opencv method used in this video ?

Spoiler: Video

how to cut pictures from what we only draw .

Thank you

 

Hi

I am getting a problem to make a build of the latest version

"If the version of Unity is less than 2017.2, you have to set opencv2.framework to Embedded Binaries manually."

I am using unity 5.6.3 I need to this version because I am running a sample that requires a old version of vuforia

Can some on point me in the right direction a script would be awesome.

Thanks in advance

 

Hi, I just downloaded this and I'm trying to get it to run within a Magic Leap project and running into this runtime error when attempting to create a Mat from a Texture2D:

Code (csharp):

1. EntryPointNotFoundException: Unable to find an entry point named 'core_Mat_n_1Mat__III' in 'opencvforunity'.
2. at UnityEngine.JsonUtility.FromJson (System.String json, System.Type type) [0x00000] in <filename unknown>:0
3. at UnityEngine.JsonUtility.FromJson (System.String json, System.Type type) [0x00000] in <filename unknown>:0
4. at OpenCVForUnity.Mat.core_Mat_n_1Mat__III (Int32 rows, Int32 cols, Int32 type) [0x00000] in <filename unknown>:0
5. at OpenCVForUnity.Mat..ctor (Int32 rows, Int32 cols, Int32 type) [0x00000] in <filename unknown>:0

Any ideas? Thanks in advance.

 

Hello @EnoxSoftware,
I just bought your OpenCV and Dlib Landmark Face detector for the CVVTuber Example.

I follow through your tutorial on how to set up OpenCV and CVVTuber
And on the console, it said the opencvforunity.dll is successfully set

SetPluginImportSettings Success :Assets/OpenCVForUnity/Plugins/Windows/x86_64/opencvforunity.dll

UnityEngine.Debug:Log(Object)

OpenCVForUnity.OpenCVForUnityMenuItem:SetPlugins(String[], Dictionary`2, Dictionary`2) (at Assets/OpenCVForUnity/Editor/OpenCVForUnityMenuItem.cs:378)

OpenCVForUnity.OpenCVForUnityMenuItem:SetPluginImportSettings() (at Assets/OpenCVForUnity/Editor/OpenCVForUnityMenuItem.cs:168)

But when I hit play, it displayed these warnings and errors:

The warning:

Plugins: Failed to load 'Assets/OpenCVForUnity/Plugins/Windows/x86_64/opencvforunity.dll' with error 'The specified module could not be found.

'.

OpenCVForUnity.Mat:.ctor(Int32, Int32, Int32) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:526)

OpenCVForUnity.Mat:.ctor(Int32, Int32, Int32) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:520)

CVVTuber.<_Initialize>c__Iterator0:MoveNext() (at Assets/CVVTuberExample/CVVTuber/Scripts/Utils/WebCamTextureToMatHelper.cs:513)

UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)

CVVTuber.WebCamTextureToMatHelper:Initialize() (at Assets/CVVTuberExample/CVVTuber/Scripts/Utils/WebCamTextureToMatHelper.cs:307)

CVVTuber.WebCamTextureMatSourceGetter:Setup() (at Assets/CVVTuberExample/CVVTuber/Scripts/WebCamTextureMatSourceGetter.cs:52)

CVVTuber.<Start>c__Iterator0:MoveNext() (at Assets/CVVTuberExample/CVVTuber/Scripts/Core/CVVTuberControllManager.cs:27)

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

The first error:

DllNotFoundException: opencvforunity

OpenCVForUnity.Mat..ctor (Int32 rows, Int32 cols, Int32 type) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:520)

CVVTuber.WebCamTextureToMatHelper+<_Initialize>c__Iterator0.MoveNext () (at Assets/CVVTuberExample/CVVTuber/Scripts/Utils/WebCamTextureToMatHelper.cs:513)

UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)

UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)

CVVTuber.WebCamTextureToMatHelper:Initialize() (at Assets/CVVTuberExample/CVVTuber/Scripts/Utils/WebCamTextureToMatHelper.cs:307)

CVVTuber.WebCamTextureMatSourceGetter:Setup() (at Assets/CVVTuberExample/CVVTuber/Scripts/WebCamTextureMatSourceGetter.cs:52)

CVVTuber.<Start>c__Iterator0:MoveNext() (at Assets/CVVTuberExample/CVVTuber/Scripts/Core/CVVTuberControllManager.cs:27)

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

And the second one:

DllNotFoundException: opencvforunity

OpenCVForUnity.Mat..ctor () (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:486)

OpenCVForUnity.MatOfPoint3f..ctor (OpenCVForUnity.Point3[] a) (at Assets/OpenCVForUnity/org/opencv/core/MatOfPoint3f.cs:45)

CVVTuber.DlibHeadRotationGetter.Setup () (at Assets/CVVTuberExample/CVVTuber/Scripts/DlibHeadRotationGetter.cs:93)

CVVTuber.CVVTuberControllManager+<Start>c__Iterator0.MoveNext () (at Assets/CVVTuberExample/CVVTuber/Scripts/Core/CVVTuberControllManager.cs:27)

UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)

What should I do now? This is the third time I tried.

 

You need to add Embedded Binaries to the output Xcode project.
Please refer to ReadMe.pdf for details.

 

Since I do not have MagicLeap, I have not tested whether OpenCVforUnity works with MegicLeap LuminOS yet.

 

Is ImportSettings of opencvforunity.dll set like this image?


Also,
Could you tell me the environment you tested?
OpenCV for Unity version :
DlibFaceLandmarkDetector version :
CVVTuberExample
Unity version :
WindowsOS version :

 

Mine looks like this. I've tried to reload Unity several times. But the warning stayed the same.

Dlib FaceLandmark Detector version: 1.2.4
OpenCV for Unity version: 2.3.2
Unity version: 2018.2.6f1
CVVTuber Example version: 1.0.1

 

Thank you very much for reporting.
Have the projects imported only "OpenCV for Unity" work without any problem? Does "OpenCVForUnity / Examples / Basic / Texture2DToMatExample / Texture2DToMatExample.unity" work?
Also,
Are you using Windows10?

 

The project imported only "OpenCV for Unity" display the warnings and DllNotFound Exception as well.
And I'm using Window 7.

 

Unfortunately, I still can not determine the cause of the error.
Could you send me the project you tested for further investigation?
https://enoxsoftware.com/opencvforunity/contact/technical-inquiry/

 

What is the lowest version of iOS and Android that this plugin supports?

 

@EnoxSoftware
Hi, I have submitted the error logs at the technical inquiry on your site. Hope to hear from you soon.

 

iOS：iOS 8.0 or higher.
Android：OS 4.1 or higher.

 

Hi,

Just wondering if anyone can help me out with some ArUco tracking I have the markers fixed to an object, but not at the object's centre. I want to be able to align a GameObject with its real-world counterpart, but I can't get the GameObject to rotate around it's centre. I also can't figure out how best to apply the offset of the marker.

Not the best posed question for sure, but if anyone could help me out that would be great.

Thanks,

 

Hi, I don't have the answer, but in the mean time I recommend checking out https://assetstore.unity.com/packages/templates/tutorials/markerbased-ar-example-29678 it's by the same developer and might give you some hints on how to go about doing this.

 

I bought this asset several months ago, Comodo is now alerting that a part of this asset contains AdWare Adposhel file is opencv_optflow341.dll here is the screen shot: https://ibb.co/mvXt20 What should I do?

 

Hello！
I use OpenCVForUnity but I have one question.
A red rectangle is displayed when tracking with CamShiftExample, but how to display CamShiftExample.cs to display arbitrary images (such as allowing easy image exchange with inspector) at the center of the detection area Should I rewrite it?

 

Hi, I have been facing this error in Xcode 10.1 with your latest Update


Undefined symbols for architecture armv7:

"_features2d_FeatureDetector_detect_13", referenced from:

_FeatureDetector_detect_m1145136506 in Bulk_Assembly-CSharp_8.o

_FeatureDetector_features2d_FeatureDetector_detect_13_m1389406130 in Bulk_Assembly-CSharp_8.o

(maybe you meant: _FeatureDetector_features2d_FeatureDetector_detect_13_m1389406130)
.
.
.
.
"_ml_ANN_1MLP_1ANNEAL_setAnnealFinalT_10", referenced from:

_ANN_MLP_ANNEAL_setAnnealFinalT_m1467379725 in Bulk_Assembly-CSharp_5.o

_ANN_MLP_ANNEAL_ml_ANN_1MLP_1ANNEAL_setAnnealFinalT_10_m2035420374 in Bulk_Assembly-CSharp_5.o

(maybe you meant: _ANN_MLP_ANNEAL_ml_ANN_1MLP_1ANNEAL_setAnnealFinalT_10_m2035420374)

ld: symbol(s) not found for architecture armv7

clang: error: linker command failed with exit code 1 (use -v to see invocation)


I have already added
AssetsLibrary.framework
CoreMedia.framework
CoreImage.framework
AVFoundation.framework

 

I think that this alert is probably false detection.
Since opencv_optflow341.dll is used only on the UWP platform, if you do not need the UWP platform build, please delete "OpenCVForUnity / Plugins / WSA / UWP" folder.

 

It may be the cause of the error that the files of the old version of OpenCVForUnity remain.
Please delete "OpenCVForUnity" folder once and re-import.
Alternatively,
Please import "OpenCVForUnity" into an empty project.

 

Hi guys, A while ago I reported the bug Plugins: Failed to load 'Assets/OpenCVForUnity/Plugins/Windows/x86_64/opencvforunity.dll' with error 'The specified module could not be found.
In case anyone got the same bug, it's either because of your VC++2013 or lacking DLL file in System 32.\
Use Dependency Walker to open your openCVforUnity.dll to see if you lack the system file to open the dll.
In my case, it was missing several DLL in System32

After downloading the missing dlls it fixed the problems

 

Here we go. I didn't plug the webcam in yet. But the dllNotFoundException is gone.

 

It is possible to get the center of RotatedRect with the following code.
https://github.com/EnoxSoftware/Ope...video/CamShiftExample/CamShiftExample.cs#L231
http://enoxsoftware.github.io/OpenC...d_rect.html#a8c60aeb84a51d441f67e103f86f5966e

Code (CSharp):

1. RotatedRect r = Video.CamShift (backProj, roiRect, termination);
2. Point center = r.center;

 

Hello, I am a student studying Opencv.
I'd like to change Mat made from CvType.CV_8UC3 to CvType.CV_8UC1 Mat.

I just try cvtColor(mat, mat, Imgproc.COLOR_RGB2GRAY); but didn't work.
And I tried to make it CvType.CV_8UC1 from the beginning, but it failed.
maybe cause is using RGB camera.

What should I do?
Thank you for reading

 

I have the same issue, would be great to get OpenCV working with Lumin OS. There is an emulator where you should be able to test it out even without device.

 

I'm having this exact same error, using unity 2018.2.16f1 and the latest plugin version: "OpenCV for Unity 2.3.2". I don't understand what the person I quote here did to fix his issues. Reimporting the plugin did not help.

What I did:
- Create empty project
- Import plugin
- Change OpenCVForUnity/Plugins/WebGL/2018.2/opencvforunity.bc settings to WebGL
- Turn the above setting off for the Unity 5.6 version
- Build for WebGL

 

I couldn't find anything in documentation or examples; How I can implement IP Camera display, and get data from IP camera ?

 

https://github.com/EnoxSoftware/Ope...nced/ComicFilterExample/ComicFilterExample.cs
Is ComicFilterExample working without problems? This code is an example of converting WebCam's Mat to grayscale Mat.

 

1)Download "OpenCV for Windows Version 3.4.2"(http://opencv.org/downloads.html).
2)Rename opencv_ffmpeg342_64.dll to opencv_ffmpeg400_64.dll.
3)Copy to Project Folder.

4)
capture.open("http://iris.not.iac.es/axis-cgi/mjpg/video.cgi?resolution=320x240?x.mjpeg");

http://answers.opencv.org/question/79001/connect-to-public-ip-camera-in-java/

 

When I tested WebGL Build in my environment, the build succeeded without problem.
Unity 2018.2.16f1
OpenCV for Unity 2.3.2
macOS 10.11.6

- Create empty project
- Import OpenCV for Unity
- Select MenuItem[Tools/OpenCV for Unity/Set Plugin Import Settings].
- Build for WebGL

Also,
The build may fail when the remaining space on the HDD is not enough.

 

Sorry for another question but, What is proper way to reflect camera(IP Cam) onto object(GUI or Plane etc) I'm using picture GUI element , but it causes little freezes in whole game like it locks main thread for small amount of times( Laggy feedback from interface )

 

Hi, does this library natively provide an efficient unity webcamtexture -> opencv mat pipeline for real time computer vision in android / ios usage cases? i.e. update the mat every time the webcamtexture updates. Preferentially even in low / mid range phones.

 

What's the status and/or plan for UWP HoloLens support in Unity 2018.2 and IL2CPP?

I can get your HoloLens examples to build just fine, but it fails to launch on the HoloLens ... not even a Unity splash screen.

• HoloLensCameraStream works with IL2CPP as of Unity 2018.2.16, which I'm using and can confirm it builds.
• HoloToolkit is up to version 2017.4.3 with a 2018 version (renamed Mixed Reality Toolkit) in Beta. It's unclear from their release notes of their 2017 releases are incompatible with Unity 2018 or not, but I've been able to build successfully using Unity 2018.2.16 using both the current release and the beta.
• I am on Windows 10 version 1803 as listed in your requirements (OS builds 17134.407)
• I have Windows 10 SDK 10.0.17134 installed and selected in Unity as the target

Again to clarify - using Unity 2018.2.16 *builds* and Visual Studio 2017 *deploys* to my HoloLens, seemingly without issue, but when it tries to launch on the device it instantly crashes/hangs without ever appearing. Switching from IL2CPP to .NET scripting backend does NOT make a difference.

 

Hi
I need to detect thumbs up gestures from hd video when a person is fully visible in the camera. Can this plugin accomplish this? Are there any demos specifically for 'thumbs up' gestures?

 

Hi,
I have questions! Help me T T
Question 1.

Code (CSharp):

1.         uvmap = new Texture2D(width, height, TextureFormat.RGFloat, false, true)
2.         {
3.             wrapMode = TextureWrapMode.Clamp,
4.             filterMode = FilterMode.Point,
5.         };

Code (CSharp):

1.                     Mat mat = new Mat(uvmap .width, uvmap .height, CvType.CV_32FC2);
2.                     Utils.fastTexture2DToMat(uvmap, mat);

This code does not work.
Error messsage is "CvException: Mat data type is not compatible: 13"
CV_16UC4 does not generate this error but has an invalid value.

How do I convert that texture into a mat and then back to a texture?

Question 2.
When working with OpenCV and RealSence D415 cameras,
I felt that I needed to understand a lot about color formats.

Do you happen to know a site where I can study?
I want to study enough to change most supported formats freely!

Thank you for reading