Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] OpenCV for Unity

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Oct 30, 2014.

  1. alexmccraw

    alexmccraw

    Joined:
    Oct 31, 2019
    Posts:
    1
    @EnoxSoftware I can't get the bilateral filter operation to actually work, I'm trying to use it in a realtime image processing pipeline but to verify it was not performing the operation I created an editor script that loads an image, attempts to apply the filter, and saves a copy of the image. I verified that the input and output images are identical, I'm not sure why this operation in particular is failing as I'm using plenty of other Imgproc methods succesfully. Below is my test code as well as the input and output image.

    Code (CSharp):
    1.     [ContextMenu("Test")]
    2.     void Test()
    3.     {
    4.         Mat image = Imgcodecs.imread("[OBFUSCATED]\\cat.jpg");
    5.         Imgproc.bilateralFilter(image, image, 9, 150, 150);
    6.         Imgcodecs.imwrite("[OBFUSCATED]\\bilateral.jpg", image);
    7.     }
    cat.jpg
    Original image

    bilateral.jpg
    Filtered image
     
  2. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    100
    Last test, it's a mistery :D:D:D

    Just new photo (not trained) converted to jpg and voila...
    I don't know what happen with all other photo, format ? compression ? or ???
    Image_10-02-2023_à_10.39.jpg
     
  3. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Just to be sure, I have to ask, are the defects you are currently encountering that only occur on specific models the result of testing the YOLOv5 example (v1.0.2) on a clean project?
    Several of your recent posts seem to be about modified code.
     
  4. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Curiously, OpenCV seems to generate an internal error if the src and dst of the bilateralFilter function are the same mat.
    It is necessary to temporarily prepare a mat of the same size for output.
    Code (CSharp):
    1.  
    2.     [ContextMenu("Test")]
    3.     void Test()
    4.     {
    5.         Mat image = Imgcodecs.imread("[OBFUSCATED]\\cat.jpg");
    6.         Mat dst = image.clone();
    7.         Imgproc.bilateralFilter(image, dst, 9, 150, 150);
    8.         dst.copyTo(image);
    9.         Imgcodecs.imwrite("[OBFUSCATED]\\bilateral.jpg", image);
    10.     }
    11.  
    This error can be confirmed by enabling debug mode.
    Code (CSharp):
    1.  
    2. //if true, The error log of the Native side OpenCV will be displayed on the Unity Editor Console.
    3. Utils.setDebugMode(true);
    4.  
    5. Imgproc.bilateralFilter(image, image, 9, 150, 150);
    6.  
    7. Utils.setDebugMode(false);
    8. //
    9. imgproc::bilateralFilter_11() : OpenCV(4.7.0-dev) C:\Users\satoo\Desktop\opencv\modules\imgproc\src\bilateral_filter.dispatch.cpp:166: error: (-215:Assertion failed) (src.type() == CV_8UC1 || src.type() == CV_8UC3) && src.data != dst.data in function 'cv::bilateralFilter_8u'
    10. //
    11.  
     
    Last edited: Feb 10, 2023
  5. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    100
    Clean project or modified code is same problem with the model, like I said in my previous post all photo used in model not work, only new photo... (my last post with good result is modified code and new photo)
     
  6. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Hmmm, that is really strange behavior.
    I can't think of a cause or solution...
     
  7. WayneVenter

    WayneVenter

    Joined:
    May 8, 2019
    Posts:
    56
    @ibompuis don't do this
    Code (CSharp):
    1. Imgcodecs.imread(Application.persistentDataPath+"/test.jpg");
    Do this
    Code (CSharp):
    1. Imgcodecs.imread(Path.Combine(Application.persistentDataPath, "test.jpg"));
     
    ibompuis likes this.
  8. gromiczek

    gromiczek

    Joined:
    Jul 19, 2019
    Posts:
    25
    Hi - thanks for all your work and support on OpenCV for Unity!

    I am working with a RealSense 455d camera and have used the RealSense SDK to do some image processing on the camera input and I now want to pass that to the OpenCV of Unity SDK to do perform blob detection. Right now the output from the RealSense pipeline is a binding to a RawImage.texture on a canvas like so:

    texture_binding_in_unity.jpg

    However, now I'd like to send this output to something like your `
    MultiObjectTrackingBasedOnColorExample` script, but I'm unsure how to connect the dots between the `RawImage.texture` I have and the process in your script. It appears that the `
    WebCamTextureToMatHelper` is a key part of this interface, but how could I connect my `RawImage.texture` output to this or some other part of the script? Thanks!
     
  9. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Unfortunately I don't have a RealSence so I haven't tried it, but I did find the following examples.
    https://qiita.com/UnagiHuman/items/f877da98d7600e3d579c
    According to this example article, integration with OpenCVForUnity seems to be possible.
     
    gromiczek likes this.
  10. Quadropups

    Quadropups

    Joined:
    May 23, 2017
    Posts:
    18
    Hello! Quick question. Does this API allow turn on/off phone flashlight while AR is active?
     
  11. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    OpenCV does not have the ability to control mobile phone flashlight.
    Third party assets must be linked.
     
  12. WayneVenter

    WayneVenter

    Joined:
    May 8, 2019
    Posts:
    56
    @EnoxSoftware hi guys, back on the DNN Object Detection track, thank you for adding Yolov7, but you did not include Yolov5 in the distribution / release version, also you are using .weights file in the Yolov7 example and not ONNX, the main purpose of my original request was to use ONNX files.

    Is the issues that you can't covert the weigts file to ONNX, because the main use case is to take Pytorch .pt files to .ONNX, this works 100% for Yolov5 example you created.

    I did some digging on this and found the following:
    How export to onnx be used by opencv?
    https://github.com/WongKinYiu/yolov7/issues/49

    I also read your notes about ONNX in the example, if you can have a look at this alternative solution as per the above mentioned issue.

    Opencv dnn success run custom yolov7
    1. reparam trained best.pt
    https://github.com/WongKinYiu/yolov7/blob/main/tools/reparameterization.ipynb
    nc = 1 #change with your nc
    model = Model('cfg/deploy/yolov7.yaml', ch=3, nc=nc).to(device)
    for i in range( (nc + 5) * 3):

    2 use branch https://github.com/WongKinYiu/yolov7/tree/u5 to export onnx model
    3 run with the same as yolov5 6.1
    https://github.com/VITA-Alchemy/yolov5_6.0_opencvdnn_python/blob/main/main_dnn.py
     
    Last edited: Mar 1, 2023
  13. thePostFuturistUnity

    thePostFuturistUnity

    Joined:
    Nov 16, 2012
    Posts:
    53
    Hello, I'm getting an almost black screen from a WebGL build on Chrome. To be specific, the texture shows one frame that was captured, then just goes black. Building HandPoseEstimationExample using SimpleWebServer if it matters.
     
  14. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    There seems to be a bug in Unity 2022.2.0f1 and above where webcamTexture.didUpdateThisFrame only updates the first frame in WebGL build.
    Could you try a build with Unity2021?
     
  15. Rayo3

    Rayo3

    Joined:
    May 21, 2018
    Posts:
    1
    Hello, the SimpleBlobExample doesn't seem to work. In Unity and the OpenGL example I get an error: "error: (-5:Bad argument) minDistBetweenBlobs>0 in function 'validateParameters'". When setting minDistBetweenBlobs=10, minArea=1, maxArea=100000, minCircularity=1, maxCircularity=100000, minInertiaRatio=1, maxInertiaRatio=100000, minConvexity=1, maxConvexity= 100000 in code I don't get an error anymore.
     
  16. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Thank you very much for reporting. This problem will be fixed in the next version.
     
  17. Zalosath

    Zalosath

    Joined:
    Sep 13, 2014
    Posts:
    687
    Hello, I'm having a problem with hconcat, it works for 2 Mats but not for 3.
    I'm using:
    Code (CSharp):
    1. Size max = new Size(Math.Max(one.size().width, Math.Max(two.size().width, colourMatch.size().width)), Math.Max(one.size().height, Math.Max(two.size().height, colourMatch.size().height)));
    2. Imgproc.resize(one, one, max);
    3. Imgproc.resize(two, two, max);
    4. Imgproc.resize(colourMatch, colourMatch, max);
    5.  
    6. Mat combined = new Mat();
    7. Core.hconcat(new List<Mat>() { one, two, colourMatch }, combined);
    If I don't include colourMatch it works, if I do it returns a Mat that is 0x0 in size, so matToTexture2D fails with GetPixels32 on degenerate image. I can use matToTexture2D on all three Mats (one, two, and colourMatch) individually and it works fine, but after combining it creates a Mat that is 0x0 and therefore causes an error when trying to use matToTexture2D.

    Note that it only fails with colourMatch so it's likely a problem with that Mat, it's created like so:
    Code (CSharp):
    1. Mat mat = new Mat(75, 50, CvType.CV_8UC3);
    2.         for (int x = 0; x < 50; x++)
    3.         {
    4.             for (int y = 0; y < 75; y++)
    5.             {
    6.                 if (x <= 24 && y <= 24)
    7.                 {
    8.                     // top left corner
    9.                     mat.put(y, x, new double[] { color1.r, color1.g, color1.b });
    10.                 }
    11.                 else if (x <= 24 && y <= 49)
    12.                 {
    13.                     // middle left
    14.                     mat.put(y, x, new double[] { color2.r, color2.g, color2.b });
    15.                 }
    16.                 else if (x <= 24 && y <= 74)
    17.                 {
    18.                     // bottom left corner
    19.                     mat.put(y, x, new double[] { color3.r, color3.g, color3.b });
    20.                 }
    21.                 else if (x > 24 && y <= 24)
    22.                 {
    23.                     // top right corner
    24.                     mat.put(y, x, new double[] { color4.r, color4.g, color4.b });
    25.                 }
    26.                 else if (x > 24 && y <= 49)
    27.                 {
    28.                     // middle right
    29.                     mat.put(y, x, new double[] { color5.r, color5.g, color5.b });
    30.                 }
    31.                 else if (x > 24 && y <= 74)
    32.                 {
    33.                     // bottom right corner
    34.                     mat.put(y, x, new double[] { color6.r, color6.g, color6.b });
    35.                 }
    36.             }
    37.         }
    38.         return mat;
    Why is this happening and how do I fix it? Thanks.

    EDIT: Fixed, one and two were 8UC4 and colourMatch was 8UC3.
     
    Last edited: Mar 13, 2023
  18. rstorm000

    rstorm000

    Joined:
    Aug 19, 2010
    Posts:
    229
    What's the expected framerate on the lightweight pose estimation? I'm only getting 15fps on a good computer with the trial version
     
  19. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Our simple test code did not generate any errors.

    Code (CSharp):
    1.  
    2. using OpenCVForUnity.CoreModule;
    3. using OpenCVForUnity.ImgprocModule;
    4. using OpenCVForUnity.UnityUtils;
    5. using System;
    6. using System.Collections.Generic;
    7. using UnityEngine;
    8.  
    9. public class HconcatTest : MonoBehaviour
    10. {
    11.  
    12.     // Start is called before the first frame update
    13.     void Start()
    14.     {
    15.         //if true, The error log of the Native side OpenCV will be displayed on the Unity Editor Console.
    16.         Utils.setDebugMode(true);
    17.  
    18.  
    19.         Mat one = new Mat(100, 100, CvType.CV_8UC3);
    20.         Mat two = new Mat(200, 200, CvType.CV_8UC3);
    21.         Mat colourMatch = new Mat(300, 300, CvType.CV_8UC3);
    22.  
    23.         //
    24.         Size max = new Size(Math.Max(one.size().width, Math.Max(two.size().width, colourMatch.size().width)), Math.Max(one.size().height, Math.Max(two.size().height, colourMatch.size().height)));
    25.         Imgproc.resize(one, one, max);
    26.         Imgproc.resize(two, two, max);
    27.         Imgproc.resize(colourMatch, colourMatch, max);
    28.  
    29.         Mat combined = new Mat();
    30.         Core.hconcat(new List<Mat>() { one, two, colourMatch }, combined);
    31.         //
    32.  
    33.  
    34.         Debug.Log("one:" + one);
    35.         Debug.Log("two:" + two);
    36.         Debug.Log("colourMatch:" + colourMatch);
    37.  
    38.         Debug.Log("combined:" + combined);
    39.  
    40.  
    41.         Utils.setDebugMode(false);
    42.     }
    43. }
    44.  
    Enabling Utils.setDebugMode(true); may help identify the cause of the error.
     
  20. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    That was about the same FPS on my PC.
    It may be possible to increase the FPS if we find another even lighter model.
    Although not related to OpenCV, a plugin for easy Human Pose Estimation in Unity is available here.
    I think this is faster since it uses Barracuda for inference.
    https://github.com/keijiro/BodyPixBarracuda
     
  21. joaomlo2

    joaomlo2

    Joined:
    Jan 3, 2017
    Posts:
    1
    Hi, I'm currently attempting to implement a WebGL webcam video recorder. The problem I'm having so far happens right on the Editor where for some reason I am unable to save both video (with VideoWriter) and images (with imwrite) to the Application.persistentDataPath folder.
    Saving on other folders, such as Desktop, works completely fine.
    Also, I tested creating a simple text file on that folder using File.WriteAllText and it works correctly, which lead me to believe it has something to do with the plugin itself.

    The code I wrote for the camera operation is as follows:
    Code (CSharp):
    1.     private IEnumerator _CameraOperationCoroutine(){
    2.         VideoWriter writer = new VideoWriter();
    3.         string save_path = Application.persistentDataPath;
    4.         do{
    5.             switch (this.camStatus){
    6.                 case CameraStatus.NoDevice:
    7.                 case CameraStatus.Unauthorized:
    8.                 case CameraStatus.Unavailable:
    9.                     ToggleErrorMessage(true);
    10.                     break;
    11.                 case CameraStatus.Initializing_Feed:
    12.                     _webcamDisplay.material.mainTexture = _webCamTexture;
    13.                     _webCamTexture.Play();
    14.                     this.camStatus = !_webCamTexture.isPlaying ? CameraStatus.Unavailable : CameraStatus.Idle;
    15.                     break;
    16.                 case CameraStatus.Starting_Recording:
    17.                     string filename = DateTime.Now.ToString("ddMMyyyyHHmmssff") + ".mp4";
    18.                     writer = new VideoWriter(Path.Combine(save_path,filename),VideoWriter.fourcc('h','2','6','4'),30, new Size(_webCamTexture.width,_webCamTexture.height));
    19.                     this.camStatus = CameraStatus.Recording;
    20.                     break;
    21.                 case CameraStatus.Recording:
    22.                     if (_webCamTexture.didUpdateThisFrame){
    23.                         Mat frame = new Mat(_webCamTexture.height, _webCamTexture.width, CvType.CV_8UC3);
    24.                         Utils.webCamTextureToMat(_webCamTexture,frame);
    25.                         Imgproc.cvtColor(frame,frame,Imgproc.COLOR_BGR2RGB);
    26.                         writer.write(frame);
    27.                     }
    28.                     break;
    29.                 case CameraStatus.Stopping_Recording:
    30.                     writer.release();
    31.                     string[] files = Directory.GetFiles(Application.persistentDataPath);
    32.                     foreach (string file in files)
    33.                     {
    34.                         Debug.Log("FILE: "+file);
    35.                     }
    36.                     this.camStatus = CameraStatus.Idle;
    37.                     break;
    38.                 case CameraStatus.Idle:
    39.                 default:
    40.                     break;
    41.             }
    42.             yield return null;
    43.         }
    44.         while (this.camStatus != CameraStatus.NoDevice && this.camStatus != CameraStatus.Unauthorized && this.camStatus != CameraStatus.Unavailable);
    45.     }
     
  22. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    When the Imgcodecs.imwrite() method was added to VideoWriterExample, the video and images were saved without problems.
    Windows10
    Unity 2021.1.0f1
    OpenCVForUnity 2.5.3

    Imgcodecs.imwrite(Application.persistentDataPath + "/VideoWriterExample_image"+frameCount+".png", recordingFrameRgbMat);

    https://github.com/EnoxSoftware/Ope...WriterExample/VideoWriterExample.cs#L148-L175
    Code (CSharp):
    1.         void OnPostRender()
    2.         {
    3.             if (isRecording)
    4.             {
    5.                 if (frameCount >= maxframeCount ||
    6.                     recordingFrameRgbMat.width() != Screen.width || recordingFrameRgbMat.height() != Screen.height)
    7.                 {
    8.                     Debug.LogError("Please fix the screen ratio of the Game View to recognize the recording area.");
    9.                     OnRecButtonClick();
    10.                     return;
    11.                 }
    12.  
    13.                 frameCount++;
    14.  
    15.                 // Take screen shot.
    16.                 screenCapture.ReadPixels(new UnityEngine.Rect(0, 0, Screen.width, Screen.height), 0, 0);
    17.                 screenCapture.Apply();
    18.  
    19.                 Utils.texture2DToMat(screenCapture, recordingFrameRgbMat);
    20.                 Imgproc.cvtColor(recordingFrameRgbMat, recordingFrameRgbMat, Imgproc.COLOR_RGB2BGR);
    21.  
    22.                 Imgproc.putText(recordingFrameRgbMat, frameCount.ToString(), new Point(recordingFrameRgbMat.cols() - 70, 30), Imgproc.FONT_HERSHEY_SIMPLEX, 1.0, new Scalar(255, 255, 255), 2, Imgproc.LINE_AA, false);
    23.                 Imgproc.putText(recordingFrameRgbMat, "SavePath:", new Point(5, recordingFrameRgbMat.rows() - 30), Imgproc.FONT_HERSHEY_SIMPLEX, 0.8, new Scalar(0, 0, 255), 2, Imgproc.LINE_AA, false);
    24.                 Imgproc.putText(recordingFrameRgbMat, savePath, new Point(5, recordingFrameRgbMat.rows() - 8), Imgproc.FONT_HERSHEY_SIMPLEX, 0.5, new Scalar(255, 255, 255), 0, Imgproc.LINE_AA, false);
    25.  
    26.                 writer.write(recordingFrameRgbMat);
    27.  
    28.                 Imgcodecs.imwrite(Application.persistentDataPath + "/VideoWriterExample_image"+frameCount+".png", recordingFrameRgbMat);
    29.             }
    30.         }
     
  23. kwkw

    kwkw

    Joined:
    Mar 3, 2014
    Posts:
    12
    Hi, is it possible to use human segmentation as a virtual background like in Zoom?

    i using
    Core.bitwise_not(rgbaMat, rgbaMat); 

    but it turned out to be like the picture I attached below. How do I remove the white masking?
     

    Attached Files:

    Last edited: Mar 20, 2023
  24. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    If you are interested in doing a Zoom-like screen composite, the following KeyFrameGreenScreenExample code may be helpful.
    How to do image compositing using a mask image:
    https://github.com/EnoxSoftware/Ope...5bd4fcc6227ec/Assets/OpenCVForUnity/Examples/ Advanced/KeyFrameGreenScreenExample/KeyFrameGreenScreenExample.cs#L228-L232
     
  25. vetronus

    vetronus

    Joined:
    Jan 21, 2016
    Posts:
    11
    OpenCV for Unity is not working on my M1 Mac. It is giving this error:

    DllNotFoundException: opencvforunity assembly:<unknown assembly> type:<unknown type> member:(null)
    OpenCVForUnity.CoreModule.Mat..ctor (System.Int32 rows, System.Int32 cols, System.Int32 type) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:516)
    CamApp.Update () (at Assets/CamApp.cs:23)

    Can someone help me out?
     
  26. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    M1Mac ( Apple Sillicon ) is supported with OpenCVForUnity 2.4.7 or higher.
     
  27. unity_02PK1Lep1qhCqw

    unity_02PK1Lep1qhCqw

    Joined:
    Mar 28, 2023
    Posts:
    3
    Hi, Is there some plan to support Windows arm64 platform (not UWP)? Seeing this error when ran on Windows 11 arm64

    DllNotFoundException: Unable to load DLL 'opencvforunity'. Tried the load the following dynamic libraries: Unable to load dynamic library 'opencvforunity' because of 'Failed to open the requested dynamic library (0x06000000) - The specified module could not be found. (WinError:0000007e)
    at OpenCVForUnity.CoreModule.Mat..cto

    Thank you! Great plugin BTW
     
  28. kwkw

    kwkw

    Joined:
    Mar 3, 2014
    Posts:
    12
    Thank you for replying. I still haven't found a solution on how to create a virtual background without using a greenscreen. woul you give me some sample
     
  29. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    I understand that UnityEditor itself does not currently support the Windows 11 ARM64 platform. Is this error displayed when you run a pre-built app?
    https://docs.unity3d.com/2023.2/Documentation/Manual/system-requirements.html
     
  30. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    We have modified HumanSegmentationExample to add a background composite function, please try it out.
     

    Attached Files:

  31. unity_02PK1Lep1qhCqw

    unity_02PK1Lep1qhCqw

    Joined:
    Mar 28, 2023
    Posts:
    3
    I used x86 PC to compile for arm64. Then ran on arm64. That's where the error shows up. App runs fine otherwise, just the OpenCV part throws this error.

    https://docs.unity3d.com/2023.2/Documentation/Manual/system-requirements.html#desktop

    In the compiled app opencvforunity.dll is placed under ..\Plugins\x86_64 (which is the same as x86). I tried changing dll's CPU to Arm64 in the Inspector but when the compile completes it goes back to x64.
     
  32. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    I did not know that windows ARM64 is supported in Unity 2023.2. Unfortunately, OpenCVForUnity does not currently support windows ARM64.
     
  33. H_v_H

    H_v_H

    Joined:
    May 21, 2018
    Posts:
    4
    Hello and good day to you.
    I'm using OpenCV and Yolov4 for real-time object detection with my custom dataset, and I need to add a UI button above each 3D item detected. I attempted to tweak the drawPred method to meet my requirements but became perplexed. I'm aware of an older question that is similar to mine and has been answered here: https://forum.unity.com/threads/released-opencv-for-unity.277080/page-52#post-6745243
    However, when I attempted to use the attached file for that answer, I received the following error:

    Code (CSharp):
    1. CvException: An error occurred on the C++ side, causing the class initialization to fail.Enclose the point where the error occurs in Utils.setDebugMode() method, and the error on the C++ side will be displayed on the console.
    2. OpenCVForUnity.DisposableObject.ThrowIfNullIntPtr (System.IntPtr ptr) (at Assets/OpenCVForUnity/org/opencv/DisposableObject.cs:58)
    3. OpenCVForUnity.ObjdetectModule.CascadeClassifier..ctor (System.String filename) (at Assets/OpenCVForUnity/org/opencv/objdetect/CascadeClassifier.cs:76)
    4. OpenCVForUnitySample.DetectFace2DTo3DExample.Start () (at Assets/OpenCVForUnity/Examples/DetectFace2DTo3DExample/DetectFace2DTo3DExample.cs:31)

    Could you perhaps assist me with placing a UI button above every 3D item detected by the webcam?
     
  34. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Hi, H_v_H
    It seems that the error was caused by a recent change in the placement path of the StreamingAssets file in OpenCVForUnity.
    Please try the corrected example.
     

    Attached Files:

    H_v_H likes this.
  35. H_v_H

    H_v_H

    Joined:
    May 21, 2018
    Posts:
    4
    thank you very much :)
     
  36. patrickcatanzariti

    patrickcatanzariti

    Joined:
    Oct 9, 2014
    Posts:
    2
    Hi all!

    I'm developing on Unity 2021.3.10f1, on a Mac M1 laptop.

    When running the examples, I'm finding I regularly get a "OnWebCamTextureToMatHelperErrorOccurred TIMEOUT" error and the webcam view doesn't load. It seems to work on first load, and then if I load a different example without stopping and restarting in Play mode, it has a timeout?

    When I've separated one example into its own scene and opened the scene directly, it seems to be timing out too.

    Any advice on managing the timeouts better? Is this a common issue? I haven't spotted too many mentions of it in the thread here.

    Thank you for your help!
    Patrick
     
  37. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Thank you very much for reporting.
    I have not received any reports of similar problems.
    Does changing the value of the "TImeout Frame Count" property change the behavior?
    WebCamTextureHelper_DeviceName.PNG
     
  38. nativehfQ

    nativehfQ

    Joined:
    Nov 1, 2017
    Posts:
    11
    Hi, is this something that you are would be supporting in future? Really interested in Win ARM64 support.

    Thank you!
     
  39. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    nativehfQ likes this.
  40. patrickcatanzariti

    patrickcatanzariti

    Joined:
    Oct 9, 2014
    Posts:
    2
    So far increasing that seems to have solved it... I'll keep an eye on it and report back if I find it reoccurring - but that Timeout Frame Count field looks to be the solution! Must be taking longer than usual to get access to the webcam on my laptop?
     
  41. PaavoHapponen

    PaavoHapponen

    Joined:
    Sep 10, 2014
    Posts:
    6
    After executing

    Mat opencvmaterial=Imgcodecs.imread(@"exrfile.exr", Imgcodecs.IMREAD_COLOR );

    I get
    Code (CSharp):
    1. CvException: imgcodecs::imread_10() : OpenCV(4.7.0-dev) C:\Users\satoo\Desktop\opencv\modules\imgcodecs\src\grfmt_exr.cpp:103: error: (-213:The function/feature is not implemented) imgcodecs: OpenEXR codec is disabled. You can enable it via 'OPENCV_IO_ENABLE_OPENEXR' option. Refer for details and cautions here: https://github.com/opencv/opencv/issues/21326 in function 'cv::initOpenEXR'
    Is this a compilation oversight or is exr support unavailable?
    Exr is marked as supported in documentation.
     
  42. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Please enable OPENCV_IO_ENABLE_OPENEXR as per the error message.
    https://github.com/opencv/opencv/issues/21326

    https://support.unity.com/hc/en-us/...environment-variable-s-set-how-can-I-do-that-
    Variable name OPENCV_IO_ENABLE_OPENEXR
    Variable value 1
     
  43. PaavoHapponen

    PaavoHapponen

    Joined:
    Sep 10, 2014
    Posts:
    6
    Apologies, after trying the script preprocessor defines I assumed it was a compilation parameter.

    Adding
    System.Environment.SetEnvironmentVariable("OPENCV_IO_ENABLE_OPENEXR", "1", EnvironmentVariableTarget.Process);
    before imread call seems to work on android builds, but for windows, after a long while debugging, I found I had to modify the build script to not preload the dll;
    pluginImporter.isPreloaded = false;

    Something to consider.
     
    EnoxSoftware likes this.
  44. wechat_os_Qy02fzyOGnzWoIhqek1ZhmB5s

    wechat_os_Qy02fzyOGnzWoIhqek1ZhmB5s

    Joined:
    Aug 6, 2020
    Posts:
    4
    Hello, may I ask why OpenCVForUnity can only use the CascadeClassifier detector for face detection on hololens2, but not Dnn deep learning algorithm? The performance of Dnn deep learning algorithm is better than the CascadeClassifier detector. How can I use Dnn deep learning algorithm (e.g. Dnn.readNet) on hololens2 ? Thank you very much!
     
  45. nativehfQ

    nativehfQ

    Joined:
    Nov 1, 2017
    Posts:
    11
  46. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    The OpenCV Dnn module is not supported by the UWP platform of OpenCV C++, so there is no plan to be implemented by OpenCVforUnity for now.
    https://github.com/opencv/opencv/blob/master/modules/dnn/CMakeLists.txt#L1-L3
    https://github.com/opencv/opencv/issues/9177
     
  47. drewwise

    drewwise

    Joined:
    Jun 16, 2016
    Posts:
    5
    Asset validation failed (90085)

    When I run Distribute App from Xcode, I get an error and can't upload to App Store Connect.

    Asset validation failed No architectures in the binary. Lipo failed to detect any architectures in the bundle executable. (ID: ****-****-****-****-****)


    How can I resolve this error?
    The development environment is as follows.

    macOS Version : Monterey 12.6.3
    Xcode Version : Version 14.2 (14C18)
    Unity Version : 2019.3.2.f1
    Framework used: OpenCVforUnity2.3.3

    Thank you.
     
  48. fredmouse

    fredmouse

    Joined:
    Oct 22, 2017
    Posts:
    5
    I'm trying to improve color correction on face swapper and face masking. It seems that the function has limited tolerance of light and color difference. Face color can easily go to infidelity turning green or purple or too bright or too dark. I've found that specifyHistogram in facechanger codes doing all the tricks but is there any way I can do with it to improve the color fidelity?
     
  49. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Older versions may cause problems.
    Could you try the latest version of OpenCVforUnity?
     
  50. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Unfortunately, we could not find a better color correction algorithm. It has been some years since we created the code, though, so we may find a new way.