[RELEASED] OpenCV for Unity

Hello,
i have a problem i am hoping you can give a solution for it.
i am using color detection and i want the user to select the color.
when the player select a color using camera i make a new ColorObject the problem is that i dont know which value the Scalar uses RGB or HSV values and i don't know how to find max and min values of that color.
So I have RGB values and from it i want find max and min and assign them to the ColorObject.
Thanks.

 

Cannot seem to run the yolov4 example without crashing in Unity 2020.3 LTS

 

So the links to tinyyolo3 do not work for tinyyolo4 ... this is causing crashes. What are the correct links to the v4?

 

I just tested it in the same environment as yours, and it worked fine in both yolov3-tiny and yolov4-tiny.

My environment :
Unity2020.3.0f1(LTS)
OpenCVForUnity2.4.3

yolov3-tiny files:
https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-tiny.cfg
https://pjreddie.com/media/files/yolo3-tiny.weights

yolov4-tiny files:
https://github.com/AlexeyAB/darknet/raw/master/cfg/yolov4-tiny.cfg
https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights

 

Hi there,

I can't find the scene for AR face tracking. I need to create some AR content like snapchat filters.

Where can I find it?

 

dyld: Library not loaded: @ rpath / opencv2.framework

I'm getting exactly the same error on unity 2020.2.1f1 using xcode 12.4 and ios 14.4
Any suggestions?

Thank you very much!

Michal

 

Edit: Okay was me being a noob, use the Tools > OpenCVForUnity > Set plugin import settings menu item to fix this

Same here on 2020.3 in an IOS project too, using Xcode 12.4


UnityFramework, 265): Library not loaded: @rpath/opencv2.framework/opencv2

 

libiconv.2.dylib not found

ran into this issue while trying to build to iOS XCode an old OpenCVForUnity project from 2019.

 

Hi, I was wondering if anyone has managed to use the dnn module in OpenCV for Unity, specifically a yolov4-tiny model, with GPU and OpenGL. I believe the OpenCV wrapper currently uses OpenCL, but this is rather limiting when running a model on a mobile device. We have already developed an entire app and interface for a customer project using this wrapper, but have ran into some performance limitations that we can't seem to overcome without having access to utilizing GPU for these dnn algorithms.

 

Ignore OpenCV if its Yolo implementation is CPU only, there is an implementation of Tiny Yolo v3 here which I have recently ported from using AR (which is obviously limited to CPU only on Android with ARCore as ARCore does kot support Vulkan and Unity Barracuda only really uses Vulkan on Android). IOS uses GPU with metal which means Iphone 6 and onwards you are good.

Also important to note that Unity Barracuda 1.3.2 (I think) also has some drastic speed improvements for CPU inference.

If your use case is for non-AR, feel free to DM me I can share what I did to get it working with webcamtexture and also you can pass any kind of texture to it if you needed to run inference on a video or similar.

All you need to do is remove the AR stuff in the scene and where the script for detection uses XR CPU image just provide it with a texture from elsewhere

https://github.com/derenlei/Unity_Detection2AR

 

https://forum.unity.com/threads/no-reference-to-libiconv-2-dylib-on-xcode-7.353999/

 

Hello. I have a question. I want to pass data in Mat format captured through this asset from C# to Java. . . How can I convert C#'s Mat to Android Java's Mat??

In general, an attempt is made to insert Intptr in C# Mat into long in Java Mat by calling Java Class existing in Java AAR Plugin File, but it fails.

 

Hello I have a question.Problem with processing Mat into Imgproc.floodfill.Mat is from WebCamTexture.All process works except floodfill function,it doesnt give me back exact Point where he should floodfill.Could you help me which paramaters of Point and Rect should be for the floodfill to work(because it's WebCamTexture that is real time process per frame)?

 

There is probably no way to convert it directly. Once you get the data array from the C# mat, you need to set the data array in the Java mat.

The following code can be used to retrieve data from Mat in C#.

Code (CSharp):

1. byte[] data = new byte[ csharpMat.rows() * csharpMat.cols() * csharpMat.elemSize()];
2. csharpMat.get( 0, 0, data);

 

Hi,
I’m working on a project in Unity with the HTC Vive Pro Eye headset.
I have tried the openCV for Unity example scene and the front facing filter fun provided here Augmented Virtuality. (more on “mixed reality”) | by Dario Laverde | Medium and everything works.

Now, starting from the front facing filter fun example, I want to do the same in the Test Tracked Camera scene provided by SteamVR, working on the SteamVR_TrackedCamera and the SteamVR_TestTrackedCamera scripts.
Do you know how I could convert to OpenCV the video stream from the OpenGL or the DirectX textures in the SteamVR_TrackedCamera script?

I tried to apply to the texture in the Test Tracked Camera scene a gaussian filter. I worked in the Update of the SteamVR_TestTrackedCamera script. Writing something like this

Code (CSharp):

1. byte[] buffer = newbyte[source.nWidth * source.nHeight *4];
2. Marshal.Copy(texture.GetRawTextureData(),buffer,0,source.n nWidth * source.nHeight *4);
3. Mat rgbaMat = new Mat(texture.height, texture.width, CvType.CV_8UC4);
4. rgbaMat.put(0,0,texture.GetRawTextureData());
5. Imgproc.GaussianBlur(rgbaMat, rgbaMat, new Size(51,51),0);
6. OpenCVForUnity.UnityUtils.Utils.matToTexture2D(rgbaMat,texture);
7.  

after the material.main texture = texture of SteamVR_TestTrackedCamera script it tells me that rgbaMat and texture don’t have the same size or directly crushes and exits from Unity.

Any possible way to solve these problems is more than welcome.

Thank you.

 

Hi,

I am working on a project using the MarkerBased AR Reader from Open CV for Unity. All of my builds have worked fine both in Android and iOS in testing, but once I try to upload my project into the App Store for iOS through XCode, I am receiving the error message "No suitable application records were found. Verify your bundle identifier 'org.opencv' is correct."

I have
1. Checked my import setting in Unity so that the plugins are in the plugin folder
2. Turned of Enable Bitcode in Xcode
3. Turned on Verify Workspace in Xcode
4. Embed and Signed the opencv2 framework
5. Added a script in the build phase to override the Unsupported Architecture Error.

Code (CSharp):

1. APP_PATH="${TARGET_BUILD_DIR}/${WRAPPER_NAME}"
2.  
3. # This script loops through the frameworks embedded in the application and
4. # removes unused architectures.
5. find "$APP_PATH" -name '*.framework' -type d | while read -r FRAMEWORK
6. do
7.     FRAMEWORK_EXECUTABLE_NAME=$(defaults read "$FRAMEWORK/Info.plist" CFBundleExecutable)
8.     FRAMEWORK_EXECUTABLE_PATH="$FRAMEWORK/$FRAMEWORK_EXECUTABLE_NAME"
9.     echo "Executable is $FRAMEWORK_EXECUTABLE_PATH"
10.  
11.     EXTRACTED_ARCHS=()
12.  
13.     for ARCH in $ARCHS
14.     do
15.         echo "Extracting $ARCH from $FRAMEWORK_EXECUTABLE_NAME"
16.         lipo -extract "$ARCH" "$FRAMEWORK_EXECUTABLE_PATH" -o "$FRAMEWORK_EXECUTABLE_PATH-$ARCH"
17.         EXTRACTED_ARCHS+=("$FRAMEWORK_EXECUTABLE_PATH-$ARCH")
18.     done
19.  
20.     echo "Merging extracted architectures: ${ARCHS}"
21.     lipo -o "$FRAMEWORK_EXECUTABLE_PATH-merged" -create "${EXTRACTED_ARCHS[@]}"
22.     rm "${EXTRACTED_ARCHS[@]}"
23.  
24.     echo "Replacing original executable with thinned version"
25.     rm "$FRAMEWORK_EXECUTABLE_PATH"
26.     mv "$FRAMEWORK_EXECUTABLE_PATH-merged" "$FRAMEWORK_EXECUTABLE_PATH"
27.  
28. done
29.  

I have built and upload an application using the OpenCv plugin before, but I cannot get his particular project working.

Is there anyway to solve this problem?

I am working on
XCode : 12.3
Unity : 2019.2.15f1
macOS : Catalina 10.15.5

Thank you,
Panu

 

SOLVED

I reimported all the open cv asset plugins and setting the import setting again and it now works

 

Currently having a serious issue of being unable to upload or verify an IOS archive build of Unity apps using OpenCV that work fine when built and installed to ipad/iphone

We get the error unexpected Mach-O header code: 0x72613c21 and it doesn't go away until I go to the Build Phases for the app target in Xcode - Embed frameworks and remove libopencvforunity.a as well as OpenCVForUnityAppController.mm which probably has broken OpenCV in my app

Using OpenCV 2.4.3, Xcode 12.4 and Unity 2020.3.0f1

Issue link on github:
https://github.com/EnoxSoftware/OpenCVForUnity/issues/103

 

I'll reply to this issue on github.

 

Hi @EnoxSoftware and all,

do you know how I can delete the 'OpenCV For Unity' sentence that appears on the textures (as in the picture attached) when clicking play button?

Thank you

 

It is a watermark that appears only when using the trial version.

 

Thank you @EnoxSoftware. So there is no way to eliminate it when using the free trial version?

Another question. Is the price for which the package is sold in the Asset Store una tantum? I mean, I pay 95$ once or do I have to renew the purchase periodically?

Thank you

 

Of course, once you purchase the software, you can use it permanently, and you can also download updated versions based on AssetStore's terms and conditions.

 

Thank you very much for your reply!

 

Hi @EnoxSoftware, first of all thank you for the great work, I purchased your plugin in 2017 and happily used it since then!

One short question: We urgently need one feature from the OpenCV 4.5.1 release (from Dec 2020) or even better 4.5.2 from this month. Can you say approximately when you are going to integrate these two releases into your plugin?

 

hello, in cpp, i can directly edit the value using Mat.at() function,

how can i edit pixel value directly?

i found that there are utils.matToTexture2d().
beside convert the mat to texture2D and using getPixels function,

any other faster way?, ty

 

The next version of OpenCVForUnity, which integrates OpenCV 4.5.2, will be released this month.

 

The AccessingPixelValueExample of MatBasicProcessingExample is a good reference for manipulating Mat pixel values.
https://github.com/EnoxSoftware/Ope...mple/MatBasicProcessingExample.cs#L2010-L2320

 

I plan to purchase this asset but due to Covid 19 so I will be working on 2 computers, one is my personal and the other is from the company, do I need to purchase this asset 2 times to be able to use it in my 2 computers or as long as the two using the same account, I can use the OpenCVforUnity on both of them (1-time purchase for my account)?

 

Hello, could you please add Mat creation from native (user-allocated) data pointer? For example if user have color buffer stored somewhere copying it to the new mat is not efficient at all.

 

Unity seat license allows "one" user per seat to install and use Unity on up to "two" computers simultaneously.

https://unity3d.com/jp/legal/as_terms
2.3.2 END-USER is granted a single seat license to install and use any Asset categorized in the Asset Store as an "Editor Extension" “Scripting”, or “Services” (collectively, “Extension Asset”) only on a maximum of 2 computers. For the avoidance of doubt, Extension Assets are licensed on a per seat basis and may not be shared or used concurrently on more than 2 different computers. As an exception, build farm servers and virtual machine instances used only for running, testing, or building projects with Extension Assets do not require separate seat license(s) or constitute use on more than 2 different computers.

 

Original Post: 6740311

So I have the same crash (ok if created in Editor, crashes on reload - Windows 10, OpenCVForUnity 2.4.3, Unity 2020.3.4f1 + other 2020 & 2019 versions), but for a slightly different scenario in that the offending line was for MatOfKeyPoint which was created in the class constructor. This class would have been instantiated during the Editor startup as it was a Serializable property. Placing the MatOfKeyPoint creation in a separate Init() method which in turn was called from a Start() method solved the problem.

Code (CSharp):

1. // this class crashes
2. [Serializable]
3. public class abc
4. {
5.     MatOfKeyPoint _keypoints;
6.  
7.     public abc()
8.     {
9.         _keypoints = new MatOfKeyPoint(); // crash here
10.     }
11. }
12.  
13. // This class loads fine when Init() is called from Start()
14. [Serializable]
15. public class abc
16. {
17.     MatOfKeyPoint _keypoints;
18.  
19.     public void Init()
20.     {
21.         _keypoints = new MatOfKeyPoint();
22.     }
23. }

Hopefully this helps, confirms that it's a general problem (MatOfKeyPoint& MatOfDouble + others?) and not just MatOfDouble. Because the solution revolves around class construction and sequencing it could mean that the opencv libs are not fully loaded/initialized during the Editor load process, maybe?

Thanks for a great product.