Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

[RELEASED] OpenCV for Unity

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Oct 30, 2014.

  1. Bassem102

    Bassem102

    Joined:
    Oct 11, 2017
    Posts:
    2
    Hello,
    i have a problem i am hoping you can give a solution for it.
    i am using color detection and i want the user to select the color.
    when the player select a color using camera i make a new ColorObject the problem is that i dont know which value the Scalar uses RGB or HSV values and i don't know how to find max and min values of that color.
    So I have RGB values and from it i want find max and min and assign them to the ColorObject.
    Thanks.
     
  2. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    958
    Cannot seem to run the yolov4 example without crashing in Unity 2020.3 LTS
     
  3. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    958
    So the links to tinyyolo3 do not work for tinyyolo4 ... this is causing crashes. What are the correct links to the v4?
     
  4. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    I just tested it in the same environment as yours, and it worked fine in both yolov3-tiny and yolov4-tiny.

    My environment :
    Unity2020.3.0f1(LTS)
    OpenCVForUnity2.4.3

    yolov3-tiny files:
    https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-tiny.cfg
    https://pjreddie.com/media/files/yolo3-tiny.weights

    yolov4-tiny files:
    https://github.com/AlexeyAB/darknet/raw/master/cfg/yolov4-tiny.cfg
    https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights
     
  5. danidiazr

    danidiazr

    Joined:
    Dec 27, 2016
    Posts:
    19
    Hi there,

    I can't find the scene for AR face tracking. I need to create some AR content like snapchat filters.

    Where can I find it?
     
  6. VrischDev

    VrischDev

    Joined:
    Feb 11, 2020
    Posts:
    2
  7. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,108
    Edit: Okay was me being a noob, use the Tools > OpenCVForUnity > Set plugin import settings menu item to fix this

    Same here on 2020.3 in an IOS project too, using Xcode 12.4


    UnityFramework, 265): Library not loaded: @rpath/opencv2.framework/opencv2
     
    Last edited: Mar 18, 2021
    EnoxSoftware likes this.
  8. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    958
    libiconv.2.dylib not found

    ran into this issue while trying to build to iOS XCode an old OpenCVForUnity project from 2019.
     
  9. jevon-williams

    jevon-williams

    Joined:
    Nov 19, 2020
    Posts:
    3
    Hi, I was wondering if anyone has managed to use the dnn module in OpenCV for Unity, specifically a yolov4-tiny model, with GPU and OpenGL. I believe the OpenCV wrapper currently uses OpenCL, but this is rather limiting when running a model on a mobile device. We have already developed an entire app and interface for a customer project using this wrapper, but have ran into some performance limitations that we can't seem to overcome without having access to utilizing GPU for these dnn algorithms.
     
    ina and ROBYER1 like this.
  10. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,108
    Ignore OpenCV if its Yolo implementation is CPU only, there is an implementation of Tiny Yolo v3 here which I have recently ported from using AR (which is obviously limited to CPU only on Android with ARCore as ARCore does kot support Vulkan and Unity Barracuda only really uses Vulkan on Android). IOS uses GPU with metal which means Iphone 6 and onwards you are good.

    Also important to note that Unity Barracuda 1.3.2 (I think) also has some drastic speed improvements for CPU inference.

    If your use case is for non-AR, feel free to DM me I can share what I did to get it working with webcamtexture and also you can pass any kind of texture to it if you needed to run inference on a video or similar.

    All you need to do is remove the AR stuff in the scene and where the script for detection uses XR CPU image just provide it with a texture from elsewhere

    https://github.com/derenlei/Unity_Detection2AR
     
  11. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
  12. kim-min-soo

    kim-min-soo

    Joined:
    Jan 3, 2016
    Posts:
    3
    Hello. I have a question. I want to pass data in Mat format captured through this asset from C# to Java. . . How can I convert C#'s Mat to Android Java's Mat??

    In general, an attempt is made to insert Intptr in C# Mat into long in Java Mat by calling Java Class existing in Java AAR Plugin File, but it fails.
     
  13. NewCo-Tech

    NewCo-Tech

    Joined:
    Jan 28, 2020
    Posts:
    1
    Hello I have a question.Problem with processing Mat into Imgproc.floodfill.Mat is from WebCamTexture.All process works except floodfill function,it doesnt give me back exact Point where he should floodfill.Could you help me which paramaters of Point and Rect should be for the floodfill to work(because it's WebCamTexture that is real time process per frame)?
     
  14. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    There is probably no way to convert it directly. Once you get the data array from the C# mat, you need to set the data array in the Java mat.

    The following code can be used to retrieve data from Mat in C#.
    Code (CSharp):
    1. byte[] data = new byte[ csharpMat.rows() * csharpMat.cols() * csharpMat.elemSize()];
    2. csharpMat.get( 0, 0, data);
     
  15. Federica_96

    Federica_96

    Joined:
    Feb 19, 2021
    Posts:
    22
    Hi,
    I’m working on a project in Unity with the HTC Vive Pro Eye headset.
    I have tried the openCV for Unity example scene and the front facing filter fun provided here Augmented Virtuality. (more on “mixed reality”) | by Dario Laverde | Medium and everything works.

    Now, starting from the front facing filter fun example, I want to do the same in the Test Tracked Camera scene provided by SteamVR, working on the SteamVR_TrackedCamera and the SteamVR_TestTrackedCamera scripts.
    Do you know how I could convert to OpenCV the video stream from the OpenGL or the DirectX textures in the SteamVR_TrackedCamera script?

    I tried to apply to the texture in the Test Tracked Camera scene a gaussian filter. I worked in the Update of the SteamVR_TestTrackedCamera script. Writing something like this

    Code (CSharp):
    1. byte[] buffer = newbyte[source.nWidth * source.nHeight *4];
    2. Marshal.Copy(texture.GetRawTextureData(),buffer,0,source.n nWidth * source.nHeight *4);
    3. Mat rgbaMat = new Mat(texture.height, texture.width, CvType.CV_8UC4);
    4. rgbaMat.put(0,0,texture.GetRawTextureData());
    5. Imgproc.GaussianBlur(rgbaMat, rgbaMat, new Size(51,51),0);
    6. OpenCVForUnity.UnityUtils.Utils.matToTexture2D(rgbaMat,texture);
    7.  
    after the material.main texture = texture of SteamVR_TestTrackedCamera script it tells me that rgbaMat and texture don’t have the same size or directly crushes and exits from Unity.

    Any possible way to solve these problems is more than welcome.

    Thank you.
     
  16. popwithap

    popwithap

    Joined:
    Jan 23, 2020
    Posts:
    2
    Hi,

    I am working on a project using the MarkerBased AR Reader from Open CV for Unity. All of my builds have worked fine both in Android and iOS in testing, but once I try to upload my project into the App Store for iOS through XCode, I am receiving the error message "No suitable application records were found. Verify your bundle identifier 'org.opencv' is correct."

    I have
    1. Checked my import setting in Unity so that the plugins are in the plugin folder
    2. Turned of Enable Bitcode in Xcode
    3. Turned on Verify Workspace in Xcode
    4. Embed and Signed the opencv2 framework
    5. Added a script in the build phase to override the Unsupported Architecture Error.

    Code (CSharp):
    1. APP_PATH="${TARGET_BUILD_DIR}/${WRAPPER_NAME}"
    2.  
    3. # This script loops through the frameworks embedded in the application and
    4. # removes unused architectures.
    5. find "$APP_PATH" -name '*.framework' -type d | while read -r FRAMEWORK
    6. do
    7.     FRAMEWORK_EXECUTABLE_NAME=$(defaults read "$FRAMEWORK/Info.plist" CFBundleExecutable)
    8.     FRAMEWORK_EXECUTABLE_PATH="$FRAMEWORK/$FRAMEWORK_EXECUTABLE_NAME"
    9.     echo "Executable is $FRAMEWORK_EXECUTABLE_PATH"
    10.  
    11.     EXTRACTED_ARCHS=()
    12.  
    13.     for ARCH in $ARCHS
    14.     do
    15.         echo "Extracting $ARCH from $FRAMEWORK_EXECUTABLE_NAME"
    16.         lipo -extract "$ARCH" "$FRAMEWORK_EXECUTABLE_PATH" -o "$FRAMEWORK_EXECUTABLE_PATH-$ARCH"
    17.         EXTRACTED_ARCHS+=("$FRAMEWORK_EXECUTABLE_PATH-$ARCH")
    18.     done
    19.  
    20.     echo "Merging extracted architectures: ${ARCHS}"
    21.     lipo -o "$FRAMEWORK_EXECUTABLE_PATH-merged" -create "${EXTRACTED_ARCHS[@]}"
    22.     rm "${EXTRACTED_ARCHS[@]}"
    23.  
    24.     echo "Replacing original executable with thinned version"
    25.     rm "$FRAMEWORK_EXECUTABLE_PATH"
    26.     mv "$FRAMEWORK_EXECUTABLE_PATH-merged" "$FRAMEWORK_EXECUTABLE_PATH"
    27.  
    28. done
    29.  
    I have built and upload an application using the OpenCv plugin before, but I cannot get his particular project working.

    Is there anyway to solve this problem?

    I am working on
    XCode : 12.3
    Unity : 2019.2.15f1
    macOS : Catalina 10.15.5

    Thank you,
    Panu
     

    Attached Files:

  17. popwithap

    popwithap

    Joined:
    Jan 23, 2020
    Posts:
    2
    SOLVED

    I reimported all the open cv asset plugins and setting the import setting again and it now works
     
    EnoxSoftware likes this.
  18. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,108
    Currently having a serious issue of being unable to upload or verify an IOS archive build of Unity apps using OpenCV that work fine when built and installed to ipad/iphone

    We get the error unexpected Mach-O header code: 0x72613c21 and it doesn't go away until I go to the Build Phases for the app target in Xcode - Embed frameworks and remove libopencvforunity.a as well as OpenCVForUnityAppController.mm which probably has broken OpenCV in my app

    Using OpenCV 2.4.3, Xcode 12.4 and Unity 2020.3.0f1

    Issue link on github:
    https://github.com/EnoxSoftware/OpenCVForUnity/issues/103
     
    Last edited: Apr 7, 2021
  19. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    I'll reply to this issue on github.
     
  20. Federica_96

    Federica_96

    Joined:
    Feb 19, 2021
    Posts:
    22
    Hi @EnoxSoftware and all,

    do you know how I can delete the 'OpenCV For Unity' sentence that appears on the textures (as in the picture attached) when clicking play button?

    Thank you
     

    Attached Files:

  21. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    It is a watermark that appears only when using the trial version.
     
  22. Federica_96

    Federica_96

    Joined:
    Feb 19, 2021
    Posts:
    22
    Thank you @EnoxSoftware. So there is no way to eliminate it when using the free trial version?

    Another question. Is the price for which the package is sold in the Asset Store una tantum? I mean, I pay 95$ once or do I have to renew the purchase periodically?

    Thank you
     
    Last edited: Apr 8, 2021
  23. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    Of course, once you purchase the software, you can use it permanently, and you can also download updated versions based on AssetStore's terms and conditions.
     
    Federica_96 likes this.
  24. Federica_96

    Federica_96

    Joined:
    Feb 19, 2021
    Posts:
    22
    Thank you very much for your reply!
     
  25. daniel-md

    daniel-md

    Joined:
    Jan 26, 2015
    Posts:
    6
    Hi @EnoxSoftware, first of all thank you for the great work, I purchased your plugin in 2017 and happily used it since then!

    One short question: We urgently need one feature from the OpenCV 4.5.1 release (from Dec 2020) or even better 4.5.2 from this month. Can you say approximately when you are going to integrate these two releases into your plugin?
     
  26. kenman

    kenman

    Joined:
    Mar 3, 2014
    Posts:
    14
    hello, in cpp, i can directly edit the value using Mat.at() function,

    how can i edit pixel value directly?

    i found that there are utils.matToTexture2d().
    beside convert the mat to texture2D and using getPixels function,

    any other faster way?, ty
     
  27. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    The next version of OpenCVForUnity, which integrates OpenCV 4.5.2, will be released this month.
     
  28. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    The AccessingPixelValueExample of MatBasicProcessingExample is a good reference for manipulating Mat pixel values.
    https://github.com/EnoxSoftware/Ope...mple/MatBasicProcessingExample.cs#L2010-L2320
     
  29. phamhung0306

    phamhung0306

    Joined:
    Oct 1, 2020
    Posts:
    2
    I plan to purchase this asset but due to Covid 19 so I will be working on 2 computers, one is my personal and the other is from the company, do I need to purchase this asset 2 times to be able to use it in my 2 computers or as long as the two using the same account, I can use the OpenCVforUnity on both of them (1-time purchase for my account)?
     
  30. Meetem

    Meetem

    Joined:
    Feb 10, 2014
    Posts:
    12
    Hello, could you please add Mat creation from native (user-allocated) data pointer? For example if user have color buffer stored somewhere copying it to the new mat is not efficient at all.
     
  31. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,328
    Unity seat license allows "one" user per seat to install and use Unity on up to "two" computers simultaneously.

    https://unity3d.com/jp/legal/as_terms
    2.3.2 END-USER is granted a single seat license to install and use any Asset categorized in the Asset Store as an "Editor Extension" “Scripting”, or “Services” (collectively, “Extension Asset”) only on a maximum of 2 computers. For the avoidance of doubt, Extension Assets are licensed on a per seat basis and may not be shared or used concurrently on more than 2 different computers. As an exception, build farm servers and virtual machine instances used only for running, testing, or building projects with Extension Assets do not require separate seat license(s) or constitute use on more than 2 different computers.
     
  32. PhillVixx

    PhillVixx

    Joined:
    Jan 6, 2021
    Posts:
    1
    Original Post: 6740311

    So I have the same crash (ok if created in Editor, crashes on reload - Windows 10, OpenCVForUnity 2.4.3, Unity 2020.3.4f1 + other 2020 & 2019 versions), but for a slightly different scenario in that the offending line was for MatOfKeyPoint which was created in the class constructor. This class would have been instantiated during the Editor startup as it was a Serializable property. Placing the MatOfKeyPoint creation in a separate Init() method which in turn was called from a Start() method solved the problem.

    Code (CSharp):
    1. // this class crashes
    2. [Serializable]
    3. public class abc
    4. {
    5.     MatOfKeyPoint _keypoints;
    6.  
    7.     public abc()
    8.     {
    9.         _keypoints = new MatOfKeyPoint(); // crash here
    10.     }
    11. }
    12.  
    13. // This class loads fine when Init() is called from Start()
    14. [Serializable]
    15. public class abc
    16. {
    17.     MatOfKeyPoint _keypoints;
    18.  
    19.     public void Init()
    20.     {
    21.         _keypoints = new MatOfKeyPoint();
    22.     }
    23. }
    Hopefully this helps, confirms that it's a general problem (MatOfKeyPoint& MatOfDouble + others?) and not just MatOfDouble. Because the solution revolves around class construction and sequencing it could mean that the opencv libs are not fully loaded/initialized during the Editor load process, maybe?

    Thanks for a great product.
     
unityunity