Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice

[RELEASED] OpenCV for Unity

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Oct 30, 2014.

  1. VitekCapS

    VitekCapS

    Joined:
    Jul 26, 2017
    Posts:
    3
  2. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    The asmdef file seems to be the cause of the error.
    The build will succeed if the following asmdef file is removed.
    Assets\OpenCVForUnity\EnoxSoftware.OpenCVForUnity.asmdef
    Assets\OpenCVForUnity\Editor\EnoxSoftware.OpenCVForUnity.Editor.asmdef
    Assets\OpenCVForUnityPlayMakerActions\EnoxSoftware.OpenCVForUnityPlayMakerActions.asmdef

    The solution to the problem without deleting the asmdef file is under investigation.
     
  3. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    60
    Hello,
    I am trying to remove background from recorded video using background segmentation example, its provide output with human body as a white and background to be in black colour, here i attached images

    - i want to get output in real world human body, only background to be black but human body have same rgb as reference image attached, how to achieve this
    - does it work on .mp4 video
     

    Attached Files:

  4. Philkrom

    Philkrom

    Joined:
    Dec 26, 2015
    Posts:
    87
    It works, thanks a lot !

    Sorry for this newbie question, but I just want to define a zone in the camera viewport and detect when a significant amount of pixels change within this zone (for instance 30%).

    If possible with Playmaker as my C# knowledge is very basic.

    Your asset is very powerful asset, but it has so many features and examples that I don’t know where to start. About the Playmaker actions, only in the category “ObjDetect” there are 229 actions, so I absolutely don’t know which one to use for my (pretty simple) project.

    If you could make some simple and short tutorials on the fondamentals, it would be great !

    Best regards, Philippe
     
  5. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468

    Here's how to get the results you want.
    Please remove line 278 of the BackgrownSubtractorComparisonExample.cs and insert the following line.

    Code (CSharp):
    1.  
    2. //Imgproc.cvtColor(fgmaskMat, rgbMat, Imgproc.COLOR_GRAY2RGB);
    3.  
    4. Core.bitwise_not(fgmaskMat, fgmaskMat); // Invert mask image.
    5. rgbMat.setTo(new Scalar(0, 0, 0), fgmaskMat); // Black fill with mask image.
    6.  
     
  6. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    60
    Thanks, yes its working on this "768x576_mjpeg.mjpeg" video file which is used in example scene but after changing any other mp4 video it is not accurate as much on that video,
     
  7. Packedbox

    Packedbox

    Joined:
    Jun 20, 2013
    Posts:
    20
    Hi
    I have problem with an Android 6.0.1 device running ARM V7

    Using Unity 2021.3.5 with OpenCv for Unity 2.4.8

    It works fine on most devices but not on this one

    08/31 19:26:10.413 30901 30938 Error Unity DllNotFoundException: Unable to load DLL 'opencvforunity'. Tried the load the following dynamic libraries: Unable to load dynamic library 'opencvforunity' because of 'Failed to open the requested dynamic library (0x06000000) dlerror() = dlopen failed: library "opencvforunity" not found

    I unziped the APK and libopencvforunity.so is present in the armeabi-v7a folder

    Any idea why the library cannot be loaded ?
     

    Attached Files:

    Last edited: Aug 31, 2022
  8. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    OpenCVForUnity2.4.8 supports API level 24 or higher and requires Android 7 or higher.
     
  9. Shellcial

    Shellcial

    Joined:
    Jan 9, 2021
    Posts:
    12
    Hi, I am not familiar with opencv, I am using OpenCV for Unity 2.4.8, wondering how to use GPU to accelerate the object detection process. I am using DnnObjectDetectionWebCamTextureExample script to get the detected object position and make my stuff regarding to those value. I used and trained yolov4 model and build opencv with CUDA, CUDNN, those things should have all properly built. While when I run the detection in unity, it only has 6 FPS and CPU is running at 100%, comparing to directly run darknet.exe to operate same webcam detection, the FPS has 15 and the CPU is running at a low percentage so I think it is using CUDA(GPU). I think I need to configure something so that it will use opencv with GPU in unity?

    just some extra information for my opencv built environment:
    Previously I have successfully set opev_for_dir path in my environment variable and used Cmake to build and install opencv with CUDA. While This time I followed draknet github instruction and used vcpkg. it seems it has handled all those manual configuration, doesn't know whether it will bring difference when dealing with opencv for unity, anyway, thanks for reading my question.

    Edit: I have uncomment out the two lines in yolo detection script, but seems no performance changes
    net.setPreferableBackend(Dnn.DNN_BACKEND_CUDA);
    net.setPreferableTarget(Dnn.DNN_TARGET_CUDA);
     
    Last edited: Sep 9, 2022
  10. VincentMarnier

    VincentMarnier

    Joined:
    Dec 13, 2017
    Posts:
    6
    Hello,

    First of all thanks for your hard work :)

    I have an issue with FLANN and hoped you could help me.
    You talked about it earlier:
    Excuse me if I'm wrong but the FLANN ctor does not allow me to pass the said dictionaries so there is no way to use FLANN with ORB at the moment in OpenCVForUnity wheras it is possible with OpenCV.

    --------------------------------------------------------------------------------

    See: https://docs.opencv.org/4.6.0/dc/dc3/tutorial_py_matcher.html
    While using ORB, you can pass the following. The commented values are recommended as per the docs, but it didn't provide required results in some cases. Other values worked fine.:
    Code (python):
    1. index_params= dict(algorithm = FLANN_INDEX_LSH,[/INDENT]
    2.                    table_number = 6, # 12
    3.                    key_size = 12,     # 20
    4.                    multi_probe_level = 1) #2
     
  11. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    The following article is a good reference on how to use CUDA with OpenCVForUnity.
    The native library included in OpenCVForUnity by default is built with the OPENCV_DNN_CUDA flag disabled. To use CUDA backend, rebuild OPENCV library with OPENCV_DNN_CUDA enabled.
    https://jamesbowley.co.uk/buildcompile-opencv-v3-3-on-windows-with-cuda-8-0-and-intel-mkltbb/
    https://qiita.com/utibenkei/items/3d9ce5c30ef666e14f44
    For more details, see the section on “How to use OpenCV Dynamic Link Library with customized build settings” in ReadMe.pdf.
     
  12. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Could you try the attachment?
    I have converted the Java code on this page to C# code.
    https://docs.opencv.org/3.4/d5/d6f/tutorial_feature_flann_matcher.html
     

    Attached Files:

  13. Shellcial

    Shellcial

    Joined:
    Jan 9, 2021
    Posts:
    12
    thanks for answering, I encountered the below errors when following the instruction.

    Code (CSharp):
    1. DllNotFoundException: opencvforunity assembly:<unknown assembly> type:<unknown type> member:(null)
    2. OpenCVForUnity.UnityUtils.Utils.setDebugMode (System.Boolean debugMode, System.Boolean throwException) (at Assets/OpenCVForUnity/org/opencv/unity/Utils.cs:1189)
    3. OpenCVForUnity.OpenCVForUnityResetDebugMode.InitializeOnEnterPlayMode () (at Assets/OpenCVForUnity/Editor/OpenCVForUnityResetDebugMode.cs:16)
    I am using window10, opencv4.6.0 matching opencvforUnity 2.4.8.
    this is the opencv I built and directory I set, one thing I found different from the Japanese reference link you provided is there is a bunch of dll file after his opencv built, while my build only contains the below files. Does it mean I built a static library instead of dynamic library?


    I also copy the opencvForUnity.dll from extra folder to the plugin folder and set the setting like this.
     
    Last edited: Sep 11, 2022
  14. jk-acrodea

    jk-acrodea

    Joined:
    Jun 15, 2022
    Posts:
    3
    Hi, I'm trying to convert out project to OpenCVForUnity(2.4.8) from OpenCV+Unity and having problem that OpenCVForUnity crashes in some library calls but some work.
    Not sure what's causing difference but Imgproc.circle() and Imgproc.resize() crash with EXC_BAD_ACCESS. I can reproduce same error with CircleDetection example scene. It seems like it cashes right after calling cv::Circle() inside Imgproc.circle() as attached. If I comment Imgproc.circle() line out then it goes through.
    The Unity version I'm using is 2021.3.6.f1 and I imported OpenCVForUnity from asset store, followed instruction to setup except StreamingAssets din thing since the script download_dnn_models.py didn't work so.
    Please help.

    Thanks,

    JK
     

    Attached Files:

  15. jk-acrodea

    jk-acrodea

    Joined:
    Jun 15, 2022
    Posts:
    3
    Forgot to write. It only crashes on iOS. Android works fine(so far).
     
  16. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    In order to build the dynamic library, the build parameters of cmake must be as follows.
    BUILD_SHARED_LIBS:BOOL=ON

    If the build is successful, many dll files are generated as shown below.
    dll_dynamic.PNG
     
  17. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Could you tell me the environment you tested?
    OpenCVforUnity version:
    Unity version:
    Xcode version:
     
  18. Shellcial

    Shellcial

    Joined:
    Jan 9, 2021
    Posts:
    12
    I do turn on the BUILD_SHARED_LIBS:BOOL, here is my build setting using Cmake.
    Code (CSharp):
    1. Commandline options:
    2. -DCUDA_ARCH_BIN:STRING="6.1" -DCUDA_FAST_MATH:BOOL="1" -DOPENCV_EXTRA_MODULES_PATH:PATH="C:/opencv_version_4.6.0/opencv_contrib-4.6.0/modules" -DENABLE_FAST_MATH:BOOL="1" -DWITH_CUDA:BOOL="1" -DBUILD_SHARED_LIBS:BOOL="1" -DBUILD_opencv_world:BOOL="1" -DCMAKE_CONFIGURATION_TYPES:STRING="Release" -DOPENCV_DNN_CUDA:BOOL="1"
    3.  
    4. Cache file:
    5. CUDA_ARCH_BIN:STRING=6.1
    6. CUDA_FAST_MATH:BOOL=1
    7. OPENCV_EXTRA_MODULES_PATH:PATH=C:/opencv_version_4.6.0/opencv_contrib-4.6.0/modules
    8. ENABLE_FAST_MATH:BOOL=1
    9. WITH_CUDA:BOOL=1
    10. BUILD_SHARED_LIBS:BOOL=1
    11. BUILD_opencv_world:BOOL=1
    12. CMAKE_CONFIGURATION_TYPES:STRING=Release
    13. OPENCV_DNN_CUDA:BOOL=1
    There is only one warning during my configuration part.
    Code (CSharp):
    1. CMake Warning at cmake/OpenCVGenSetupVars.cmake:54 (message):
    2.   CONFIGURATION IS NOT SUPPORTED: validate setupvars script in install
    3.   directory
    4. Call Stack (most recent call first):
    5.   CMakeLists.txt:1039 (include)
    Below is the my final configuration before building
    Code (CSharp):
    1. General configuration for OpenCV 4.6.0 =====================================
    2.   Version control:               unknown
    3.  
    4.   Extra modules:
    5.     Location (extra):            C:/opencv_version_4.6.0/opencv_contrib-4.6.0/modules
    6.     Version control (extra):     unknown
    7.  
    8.   Platform:
    9.     Timestamp:                   2022-09-12T09:13:27Z
    10.     Host:                        Windows 10.0.19044 AMD64
    11.     CMake:                       3.24.1
    12.     CMake generator:             Visual Studio 16 2019
    13.     CMake build tool:            C:/Program Files/Microsoft Visual Studio/2019/Community/MSBuild/Current/Bin/MSBuild.exe
    14.     MSVC:                        1929
    15.     Configuration:               Release
    16.  
    17.   CPU/HW features:
    18.     Baseline:                    SSE SSE2 SSE3
    19.       requested:                 SSE3
    20.     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
    21.       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
    22.       SSE4_1 (18 files):         + SSSE3 SSE4_1
    23.       SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
    24.       FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
    25.       AVX (5 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
    26.       AVX2 (33 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
    27.       AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
    28.  
    29.   C/C++:
    30.     Built as dynamic libs?:      YES
    31.     C++ standard:                11
    32.     C++ Compiler:                C:/Program Files/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe  (ver 19.29.30146.0)
    33.     C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP  /MD /O2 /Ob2 /DNDEBUG
    34.     C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP  /MDd /Zi /Ob0 /Od /RTC1
    35.     C Compiler:                  C:/Program Files/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
    36.     C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /MP   /MD /O2 /Ob2 /DNDEBUG
    37.     C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:fast     /MP /MDd /Zi /Ob0 /Od /RTC1
    38.     Linker flags (Release):      /machine:x64  /INCREMENTAL:NO
    39.     Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL
    40.     ccache:                      NO
    41.     Precompiled headers:         NO
    42.     Extra dependencies:          cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64
    43.     3rdparty dependencies:
    44.  
    45.   OpenCV modules:
    46.     To be built:                 aruco barcode bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab wechat_qrcode world xfeatures2d ximgproc xobjdetect xphoto
    47.     Disabled:                    -
    48.     Disabled by dependency:      -
    49.     Unavailable:                 alphamat cvv freetype java julia matlab ovis python2 python2 sfm viz
    50.     Applications:                tests perf_tests apps
    51.     Documentation:               NO
    52.     Non-free algorithms:         NO
    53.  
    54.   Windows RT support:            NO
    55.  
    56.   GUI:
    57.     Win32 UI:                    YES
    58.     VTK support:                 NO
    59.  
    60.   Media I/O:
    61.     ZLib:                        build (ver 1.2.12)
    62.     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
    63.     WEBP:                        build (ver encoder: 0x020f)
    64.     PNG:                         build (ver 1.6.37)
    65.     TIFF:                        build (ver 42 - 4.2.0)
    66.     JPEG 2000:                   build (ver 2.4.0)
    67.     OpenEXR:                     build (ver 2.3.0)
    68.     HDR:                         YES
    69.     SUNRASTER:                   YES
    70.     PXM:                         YES
    71.     PFM:                         YES
    72.  
    73.   Video I/O:
    74.     DC1394:                      NO
    75.     FFMPEG:                      YES (prebuilt binaries)
    76.       avcodec:                   YES (58.134.100)
    77.       avformat:                  YES (58.76.100)
    78.       avutil:                    YES (56.70.100)
    79.       swscale:                   YES (5.9.100)
    80.       avresample:                YES (4.0.0)
    81.     GStreamer:                   NO
    82.     DirectShow:                  YES
    83.     Media Foundation:            YES
    84.       DXVA:                      YES
    85.  
    86.   Parallel framework:            Concurrency
    87.  
    88.   Trace:                         YES (with Intel ITT)
    89.  
    90.   Other third-party libraries:
    91.     Intel IPP:                   2020.0.0 Gold [2020.0.0]
    92.            at:                   C:/opencv_version_4.6.0/build/3rdparty/ippicv/ippicv_win/icv
    93.     Intel IPP IW:                sources (2020.0.0)
    94.               at:                C:/opencv_version_4.6.0/build/3rdparty/ippicv/ippicv_win/iw
    95.     Lapack:                      NO
    96.     Eigen:                       NO
    97.     Custom HAL:                  NO
    98.     Protobuf:                    build (3.19.1)
    99.  
    100.   NVIDIA CUDA:                   YES (ver 11.6, CUFFT CUBLAS FAST_MATH)
    101.     NVIDIA GPU arch:             61
    102.     NVIDIA PTX archs:
    103.  
    104.   cuDNN:                         YES (ver 8.4.1)
    105.  
    106.   OpenCL:                        YES (NVD3D11)
    107.     Include path:                C:/opencv_version_4.6.0/opencv-4.6.0/3rdparty/include/opencl/1.2
    108.     Link libraries:              Dynamic load
    109.  
    110.   Python 3:
    111.     Interpreter:                 C:/Users/turtl/anaconda3/python.exe (ver 3.9.12)
    112.     Libraries:                   C:/Users/turtl/anaconda3/libs/python39.lib (ver 3.9.12)
    113.     numpy:                       C:/Users/turtl/anaconda3/lib/site-packages/numpy/core/include (ver 1.21.5)
    114.     install path:                C:/Users/turtl/anaconda3/Lib/site-packages/cv2/python-3.9
    115.  
    116.   Python (for build):            C:/Users/turtl/anaconda3/python.exe
    117.  
    118.   Java:                        
    119.     ant:                         NO
    120.     JNI:                         NO
    121.     Java wrappers:               NO
    122.     Java tests:                  NO
    123.  
    124.   Install to:                    C:/opencv_version_4.6.0/build/install
    125. -----------------------------------------------------------------
    One thing to note, no error during building while this warning comes out frequently, doesn't know if this relates to not building dynamic library
    Code (CSharp):
    1. warning : field of class type without a DLL interface used in a class with a DLL interface
     
    Last edited: Sep 12, 2022
  19. VincentMarnier

    VincentMarnier

    Joined:
    Dec 13, 2017
    Posts:
    6
  20. jk-acrodea

    jk-acrodea

    Joined:
    Jun 15, 2022
    Posts:
    3
    Thank you for your reply. The problem has gone after recreated project from scratch that removed unused packages. It was probably some symbol confliction with OpenCVPuls libs. Thanks anyways.
     
    EnoxSoftware likes this.
  21. zkaka

    zkaka

    Joined:
    May 30, 2013
    Posts:
    5
    Hi, I want to buy this plugin but I'm afraid that maybe it lacks of some functionalities so my question is: does this plugin support images stitching? I saw ImageStitcher on origin OpenCV but not in this plugin. Thanks!
     
  22. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Unfortunately, Currently, "stitching module" has not been implemented.
    https://enoxsoftware.com/opencvforunity/documentation/support-modules/
     
    zkaka likes this.
  23. Shimizoki

    Shimizoki

    Joined:
    Feb 2, 2013
    Posts:
    5
    Do you have an example of how to build a custom board? For my application I would like several fiducials wrapped around a cube. I could not find any particular example on how to build the board from scratch. At the moment I am doing the below (only showing 1 face for the sake of brevity).

    The error I get is:
    CvException: calib3d::undistort_11() : OpenCV(4.6.0-dev) C:\Users\satoo\Desktop\opencv\modules\calib3d\src\undistort.dispatch.cpp:299: error: (-215:Assertion failed) dst.data != src.data in function 'cv::undistort'


    I'd appreciate it if you could point me in the correct direction

    Code (CSharp):
    1. Dictionary dictionary = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_50);
    2.  
    3. Mat ids = new Mat(1,6, CvType.CV_32F);
    4. ids.put(0, 0, 0); // ID 0
    5. // ids.put(0, 1, 1); // ID 1
    6.  
    7. List<Mat> objPoints = new List<Mat>();
    8.  
    9. // Marker Width = 25mm, Cube = 64mm
    10. Mat p1 = new Mat(4,3, CvType.CV_32F);
    11. p1.put(0, 0, -12.5); // NW Corner X
    12. p1.put(0, 1, 32.0);  // NW Corner Y
    13. p1.put(0, 2, -12.5); // NW Corner Z
    14. p1.put(1, 0, -12.5); // NE Corner X
    15. p1.put(1, 1, 32.0);  // NE Corner Y
    16. p1.put(1, 2, 12.5);  // NE Corner Z
    17. p1.put(2, 0, 12.5);  // SE Corner X
    18. p1.put(2, 1, 32.0);  // SE Corner Y
    19. p1.put(2, 2, 12.5);  // SE Corner Z
    20. p1.put(3, 0, 12.5);  // SW Corner X
    21. p1.put(3, 1, 32.0);  // SW Corner Y
    22. p1.put(3, 2, -12.5); // SW Corner Z
    23.  
    24. objPoints.Add(p1);
    25. // objPoints.Add(p2);
    26.  
    27. board = .Board.create(objPoints, dictionary, ids); // This line throws the error
     
    Last edited: Sep 15, 2022
  24. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468


    I have tried to correct the cause of the error in your code.
    It appears that the type of ids was incorrect.

    Code (CSharp):
    1.  
    2.             //if true, The error log of the Native side OpenCV will be displayed on the Unity Editor Console.
    3.             Utils.setDebugMode (true);
    4.  
    5.  
    6.             // Create a GridBoard to know the Mat type of arguments to pass to the Board.create method.
    7.             //
    8.             GridBoard gridBoard = GridBoard.create(1, 1, 0.04f, 0.01f, Aruco.getPredefinedDictionary(Aruco.DICT_6X6_50));
    9.  
    10.             Debug.Log("gridBoard: " + gridBoard.get_objPoints().Count); // 1
    11.             Debug.Log("gridBoard: " + gridBoard.get_objPoints()[0]); // Mat [ 4*1*CV_32FC3
    12.             Debug.Log("gridBoard: " + gridBoard.get_ids()); // Mat [ 1*1*CV_32SC1
    13.             //
    14.  
    15.  
    16.             Dictionary dictionary = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_50);
    17.             Mat ids = new Mat(1, 1, CvType.CV_32SC1);
    18.             ids.put(0, 0, 0); // ID 0
    19.  
    20.             List<Mat> objPoints = new List<Mat>();
    21.  
    22.             // Marker Width = 25mm, Cube = 64mm
    23.             Mat p1 = new Mat(4, 1, CvType.CV_32FC3);
    24.             p1.put(0, 0, -12.5, 32.0, -12.5); // NW Corner X, NW Corner Y. NW Corner Z
    25.             p1.put(1, 0, -12.5, 32.0, 12.5); // NE Corner X, NE Corner Y, NE Corner Z
    26.             p1.put(2, 0, 12.5, 32.0, 12.5);  // SE Corner X, SE Corner Y, SE Corner Z
    27.             p1.put(3, 0, 12.5, 32.0, -12.5);  // SW Corner X, SW Corner Y, SW Corner Z
    28.  
    29.             objPoints.Add(p1);
    30.  
    31.             Board board = Board.create(objPoints, dictionary, ids);
    32.  
    33.             Debug.Log("board: " + board.get_objPoints().Count); // 1
    34.             Debug.Log("board: " + board.get_objPoints()[0]); // Mat [ 4*1*CV_32FC3
    35.             Debug.Log("board: " + board.get_ids()); // Mat [ 1*1*CV_32SC1
    36.  
    37.  
    38.             Utils.setDebugMode (false);
    39.  
    I have found several sites that discuss ARUCO's custom board creation and I hope this helps.
    https://forum.opencv.org/t/how-to-create-a-customized-non-square-aruco-board/7948

    https://github.com/karlun/gramods/blob/main/modules/gmTrack/src/ArucoBoardComplex.cpp
     
  25. VincentMarnier

    VincentMarnier

    Joined:
    Dec 13, 2017
    Posts:
    6
  26. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    The KeyPointsFilter class is not implemented in OpenCVForUntiy2.4.8.
     
  27. infra_unity459

    infra_unity459

    Joined:
    Nov 14, 2019
    Posts:
    2
    Hi, after switching to Mac M1, the console shows
    Code (CSharp):
    1. DllNotFoundException: opencvforunity assembly:<unknown assembly> type:<unknown type> member:(null)
    2. OpenCVForUnity.CoreModule.Mat..ctor (System.Int32 rows, System.Int32 cols, System.Int32 type) (at Assets/OpenCVForUnity/org/opencv/core/Mat.cs:512)
    I'm using unity 2021.3.11 Silicon
    OpenCV 2.4.3
    Does OpenCV for unity have support for M1 ?
     
    Last edited: Oct 3, 2022
  28. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    M1Mac ( Apple Sillicon ) is supported with OpenCVForUnity 2.4.7 or higher.
     
  29. kimvasquez17

    kimvasquez17

    Joined:
    Dec 19, 2017
    Posts:
    17
    Btw they already fixed this and I confirmed it was fixed. But I'm currenlty facing issue building webgl in Unity 2022.1.19f1 and Unity 2021.3.11f1. What I did was. Import the opencv to my project then Tools > OpenCv for Unity > Set plugin import settings. The compression format is Brotli.

    Building Library\Bee\artifacts\WebGL\build\debug_WebGL_wasm\build.js failed with output:
    wasm-ld: error: duplicate symbol: crc32_combine_op
    >>> defined in C:\Users\kim_v\AppData\Local\Temp\tmpx0zgm7qfWebGLSupport_ImageConversionModule_Dynamic.a(crc32_5s9a7.o)
    >>> defined in C:\Users\kim_v\AppData\Local\Temp\tmpjm_5zc7eopencvforunity.a(opencvforunity.bc)
    emcc2: error: 'D:/Unity/Hub/Editor/2021.3.11f1/Editor/Data/PlaybackEngines/WebGLSupport/BuildTools/Emscripten/llvm\wasm-ld.exe -o Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.wasm C:\Users\kim_v\AppData\Local\Temp\tmp_ahpl_xpGameAssembly.a C:\Users\kim_v\AppData\Local\Temp\tmptt8h5920WebGLSupport_AccessibilityModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpbrwbx3pxWebGLSupport_AIModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmprg_idrzdWebGLSupport_AndroidJNIModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpfjwl6kjdWebGLSupport_AnimationModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpzrca_qa5WebGLSupport_AssetBundleModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpcco2_1_aWebGLSupport_AudioModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpr8cd8586WebGLSupport_ClothModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpkgux3lsnWebGLSupport_CoreModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpyhlqin88WebGLSupport_CrashReportingModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmplsanljhiWebGLSupport_DirectorModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpfg3rg15jWebGLSupport_DSPGraphModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpz75z08crWebGLSupport_GameCenterModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphyvrummnWebGLSupport_GIModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpgfcw0mzcWebGLSupport_GridModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp78w6zqtkWebGLSupport_HotReloadModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpx0zgm7qfWebGLSupport_ImageConversionModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpc6lufgtlWebGLSupport_IMGUIModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpz6yqmwcbWebGLSupport_InputLegacyModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp5jhmqq6jWebGLSupport_InputModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphet7ghxwWebGLSupport_JSONSerializeModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpabno6e4cWebGLSupport_LocalizationModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpuhft22bpWebGLSupport_ParticleSystemModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphf52w7dtWebGLSupport_PerformanceReportingModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpzj9n_s_cWebGLSupport_Physics2DModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp1_th41kmWebGLSupport_PhysicsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp34a2uvnqWebGLSupport_ProfilerModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpqwxwqc5kWebGLSupport_RuntimeInitializeOnLoadManagerInitializerModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpoqohni82WebGLSupport_ScreenCaptureModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpnrhktahuWebGLSupport_SharedInternalsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpede_shwhWebGLSupport_SpriteMaskModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphxy_agq0WebGLSupport_SpriteShapeModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp96m0rx7_WebGLSupport_StreamingModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpbh157qg5WebGLSupport_SubstanceModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpn18prjd_WebGLSupport_SubsystemsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpr608rwc9WebGLSupport_TerrainModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpjt5z8_c3WebGLSupport_TerrainPhysicsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpyw8ex39gWebGLSupport_TextCoreFontEngineModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphqia0jj5WebGLSupport_TextCoreTextEngineModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpeo05fgrzWebGLSupport_TextRenderingModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp63iw39tlWebGLSupport_TilemapModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp0hinsefzWebGLSupport_TLSModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp0zb6swj8WebGLSupport_UIElementsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmprf0wum97WebGLSupport_UIElementsNativeModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpn1lu2p7jWebGLSupport_UIModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp385tx8dyWebGLSupport_UmbraModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp5sm0pz2dWebGLSupport_UNETModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpbh75qk2kWebGLSupport_UnityAnalyticsModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpzi43iyjdWebGLSupport_UnityConnectModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp9jvkavq1WebGLSupport_UnityCurlModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpqrtfp6_5WebGLSupport_UnityTestProtocolModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpihch0nieWebGLSupport_UnityWebRequestAssetBundleModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpy6u8v9bsWebGLSupport_UnityWebRequestAudioModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmphqdgdtp6WebGLSupport_UnityWebRequestModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp7n1y42j0WebGLSupport_UnityWebRequestTextureModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp928n8gb9WebGLSupport_UnityWebRequestWWWModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpwixwj2raWebGLSupport_VehiclesModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpo5z3xz2sWebGLSupport_VFXModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp0_fwzg0zWebGLSupport_VideoModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp9v_eqruvWebGLSupport_VRModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp7jychui7WebGLSupport_WebGLModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpnfnbc8glWebGLSupport_WindModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmp0wr36fo9WebGLSupport_XRModule_Dynamic.a C:\Users\kim_v\AppData\Local\Temp\tmpjm_5zc7eopencvforunity.a -LD:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libgl-webgl2-full_es3.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libal.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libhtml5.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libc.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libcompiler_rt.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libc++.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libc++abi.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libdlmalloc.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libc_rt_wasm.a D:\Unity\Hub\Editor\2021.3.11f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten\cache\sysroot\lib\wasm32-emscripten\libsockets.a -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined --strip-debug --export main --export stackSave --export stackRestore --export stackAlloc --export __wasm_call_ctors --export __errno_location --export malloc --export free --export __cxa_is_pointer_type --export __cxa_can_catch --export setThrew --export ntohs --export strlen --export htonl --export htons --export memset --export _get_tzname --export _get_daylight --export _get_timezone --export memalign --export emscripten_main_thread_process_queued_calls --export emscripten_webgl_make_context_current --export emscripten_webgl_get_current_context --export-table --export __start_em_asm --export __stop_em_asm -z stack-size=5242880 --initial-memory=33554432 --no-entry --max-memory=2147483648 --global-base=1024' failed (1)
    UnityEngine.GUIUtility: ProcessEvent (int,intptr,bool&)
     
  30. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Thank you very much for reporting.
    This error seems to occur with Unity 2021.3.9f1 and above. I am currently fixing this issue.
     
    kimvasquez17 likes this.
  31. kimvasquez17

    kimvasquez17

    Joined:
    Dec 19, 2017
    Posts:
    17
    No worries. Thank you too. Can't wait to use it in the PWA WebGL template.
     
  32. kukewilly

    kukewilly

    Joined:
    Jan 3, 2019
    Posts:
    44
    Hello,

    I'm having trouble getting OpenCV for Unity to work on on android builds with IL2PCC and ARM64 in the settings. I've confirmed this is causing the problem because it works fine when set to Mono and ARM7, but I need my project set to IL2PCC/ARM64 for another feature. This is the error I was getting from Logcat:

    2022-10-16 09:29:10.486 8619 8642 Error Unity DllNotFoundException: Unable to load DLL 'opencvforunity'. Tried the load the following dynamic libraries: Unable to load dynamic library 'opencvforunity' because of 'Failed to open the requested dynamic library (0x06000000) dlerror() = dlopen failed: library "opencvforunity" not found
    2022-10-16 09:29:10.486 8619 8642 Error Unity at OpenCVForUnity.CoreModule.Mat.core_Mat_n_1Mat__ () [0x00000] in <00000000000000000000000000000000>:0
    2022-10-16 09:29:10.486 8619 8642 Error Unity at OpenCVForUnity.CoreModule.Mat..ctor () [0x00000] in <00000000000000000000000000000000>:0
    2022-10-16 09:29:10.486 8619 8642 Error Unity at OpenCVForUnityExample.ArUcoWebCamTextureExample.Update () [0x00000] in <00000000000000000000000000000000>:0
    2022-10-16 09:29:10.486 8619 8642 Error Unity

    I assumed I was missing an updated plug-in so I updated/reimported OpenCV for Unity (2.4.9) from the package manager. But I only imported the android plugin folder because Ive made significant made changes to the ArUco scripts and didn't want it to be overwritten. This is the contents of the folder in my project:

    upload_2022-10-16_9-45-28.png


    I built and tested it and now I'm getting a crash as soon as the camera sees the marker and tries to execute the single marker pose estimation:

    0001-01-01 00:00:00.000 -1 -1 Info --------- beginning of crash
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime FATAL EXCEPTION: UnityMain
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Process: com.DefaultCompany.MetaplexTest, PID: 20327
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Version '2021.3.2f1 (d6360bedb9a0)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Build fingerprint: 'oculus/hollywood/hollywood:10/QQ3A.200805.001/47421700667400000:user/release-keys'
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Revision: '0'
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime ABI: 'arm64'
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Timestamp: 2022-10-16 10:06:43-0600
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime pid: 20327, tid: 20351, name: UnityMain >>> com.DefaultCompany.MetaplexTest <<<
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime uid: 10190
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime Cause: null pointer dereference
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x0 00000000ffffffff x1 00000000ffffffff x2 00000000ffffffff x3 00000074dbcc3340
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x4 0000000000000000 x5 00000074dbcc3318 x6 3fb307ae77c2658b x7 3fb42a1f7315485e
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x8 0000000000000000 x9 00000075db1fe018 x10 0000000002010000 x11 00000074fe69b93c
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x12 00000074fee60430 x13 0000000000000001 x14 ffffffffffffffff x15 0000000000000008
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x16 00000074ff23c628 x17 00000074fe69b8f8 x18 0000000000000001 x19 00000075db1fc4d0
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x20 00000075db1fc690 x21 0000000000000005 x22 00000075db1fe018 x23 0000000000000015
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x24 00000075db1fe018 x25 0000000000000001 x26 0000000000000001 x27 00000075db1fe018
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime x28 00000075de074300 x29 00000075db1fbf70
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime sp 00000075db1fbf40 lr 00000074fe60c124 pc 00000074fe69b940
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime backtrace:
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #00 pc 0000000000ddb940 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libopencvforunity.so (cv::_InputArray::type(int) const+72) (BuildId: d626f89a395e5d79163310033364d9c7f9a179f7)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #01 pc 0000000000d4c120 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libopencvforunity.so (cv::Mat::copyTo(cv::_OutputArray const&) const+68) (BuildId: d626f89a395e5d79163310033364d9c7f9a179f7)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #02 pc 0000000000d4223c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libopencvforunity.so (cv::Mat::convertTo(cv::_OutputArray const&, int, double, double) const+260) (BuildId: d626f89a395e5d79163310033364d9c7f9a179f7)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #03 pc 000000000037780c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libopencvforunity.so (cv::aruco::estimatePoseSingleMarkers(cv::_InputArray const&, float, cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::ptr<cv::aruco::EstimateParameters>)+832) (BuildId: d626f89a395e5d79163310033364d9c7f9a179f7)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #04 pc 0000000000317e14 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libopencvforunity.so (aruco_Aruco_estimatePoseSingleMarkers_11+244) (BuildId: d626f89a395e5d79163310033364d9c7f9a179f7)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #05 pc 0000000000c290cc /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libil2cpp.so (BuildId: 585770805546817ca728c98fc3e940937b5ccc38)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #06 pc 0000000000b39324 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libil2cpp.so (BuildId: 585770805546817ca728c98fc3e940937b5ccc38)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #07 pc 0000000000b390f0 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libil2cpp.so (BuildId: 585770805546817ca728c98fc3e940937b5ccc38)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #08 pc 0000000000983c2c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libil2cpp.so (BuildId: 585770805546817ca728c98fc3e940937b5ccc38)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #09 pc 0000000000983a8c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libil2cpp.so (BuildId: 585770805546817ca728c98fc3e940937b5ccc38)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #10 pc 00000000003032c8 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #11 pc 0000000000312184 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #12 pc 000000000031fd0c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #13 pc 0000000000192a20 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #14 pc 0000000000236690 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #15 pc 00000000002366d0 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #16 pc 0000000000236908 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #17 pc 0000000000390ea0 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #18 pc 00000000003a67b8 /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/lib/arm64/libunity.so (BuildId: f96f56f15144c26c63a3881af41e93d2b71bb6c2)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime #19 pc 000000000001078c /data/app/com.DefaultCompany.MetaplexTest-DtI3ffGiqMhTnEPMiFPO3A==/oat/arm64/base.odex
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::_InputArray::type(int) const(type:72)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::Mat::copyTo(cv::_OutputArray const&) const(copyTo:68)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::Mat::convertTo(cv::_OutputArray const&, int, double, double) const(convertTo:260)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::aruco::estimatePoseSingleMarkers(cv::_InputArray const&, float, cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::ptr<cv::aruco::EstimateParameters>)(estimatePoseSingleMarkers:832)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.aruco_Aruco_estimatePoseSingleMarkers_11(aruco_Aruco_estimatePoseSingleMarkers_11:244)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libil2cpp.0xc290cc(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libil2cpp.0xb39324(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libil2cpp.0xb390f0(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libil2cpp.0x983c2c(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libil2cpp.0x983a8c(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x3032c8(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x312184(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x31fd0c(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x192a20(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x236690(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x2366d0(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x236908(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x390ea0(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libunity.0x3a67b8(Native Method)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at base.0x1078c(Native Method)


    I know it's ideal that I just fully reimport the updated version of OpenCVforUnity, but I'm really worried about my script modifications getting erased and having to re-integrate them. I know eventually I'll have to segregate them out into their own scripts for the future, but I'd like to avoid this in the short term?

    I'm using Unity 2021.3.2f1, JDK 1.8, NDK 21, Gradle 6.9.2, Minimum Android 7 - Target Android 8
     

    Attached Files:

  33. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Is ImportSettings of "arm64-v8a / libopencvforunity.so" set correctly?
    unnamed (1).png

    unnamed (3).png

    Set the minimum API level to 24 or higher.
    unnamed (4).png
     
  34. kukewilly

    kukewilly

    Joined:
    Jan 3, 2019
    Posts:
    44
    Hey thanks for your reply,

    Can confirm my settings are as shown
    upload_2022-10-17_13-10-18.png

    upload_2022-10-17_13-11-4.png
    upload_2022-10-17_13-11-30.png

    I added some debug messages in ArUcoWebcamTextureExample.cs for more info. It breaks trying to run Aruco.estimatePoseSingleMarkers()

    upload_2022-10-17_13-14-5.png

    I have no custom edits in ArUco.cs so anything pushed through that method will deal with things as Enox software intended. I'm sending a texture2D produced by a UVC camera, then turned into a Mat class through the scripts. You can see the format of the texture2D as well as the Mat I push through below:

    upload_2022-10-17_13-9-49.png

    Again, everything works perfectly running in Mono and arm7! The most specific cause I can get reported is an Int value in the Mat class getting put into the pose method, although the mat seems to run through any of the script leading up to that without any problems:

    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::_InputArray::type(int) const(type:72)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::Mat::copyTo(cv::_OutputArray const&) const(copyTo:68)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::Mat::convertTo(cv::_OutputArray const&, int, double, double) const(convertTo:260)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.cv::aruco::estimatePoseSingleMarkers(cv::_InputArray const&, float, cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::ptr<cv::aruco::EstimateParameters>)(estimatePoseSingleMarkers:832)
    2022-10-16 10:06:44.077 20327 20351 Error AndroidRuntime at libopencvforunity.aruco_Aruco_estimatePoseSingleMarkers_11(aruco_Aruco_estimatePoseSingleMarkers_11:244)
     
  35. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    There may be an inconsistency between the OpenCVForUntiy version 2.4.9 plugin files and the older version of the script. Could you test with OpenCVForUnity 2.4.9 re-imported once you have deleted the OpenCVForUnity folder?
     
  36. kukewilly

    kukewilly

    Joined:
    Jan 3, 2019
    Posts:
    44
    [SOLVED] So I completely removed OpenCV for Unity from my project and reimported it, then took the time to splice in my customizations tot he reimported scripts. It's working perfect now! The updated version of the main script wasn't far off from the version i had previously, so i only had to make minor adjustments after I spliced in my changes.

    Thanks for the help!
     
    EnoxSoftware likes this.
  37. kimvasquez17

    kimvasquez17

    Joined:
    Dec 19, 2017
    Posts:
    17
    This is fixed in webgl, I can build a webgl project now. But I am facing another problem, which is the performance. I tested it on my android and getting only 5-10fps when using the Markerless AR example. Do you have a workaround in this. I tried lowering my camera reso to below 480p but still lagging.
     
  38. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    MarkerLessARExample Code is a rewrite of https://github.com/MasteringOpenCV/code/tree/master/Chapter3_MarkerlessAR using “OpenCV for Unity”.
    Since this example is a tutorial code, I recommend using Vuforia etc. for more advanced functions.
    I think that this page is helpful for hints on performance improvement.
    https://github.com/MasteringOpenCV/code/issues/52

    It may also be possible to improve processing speed by changing the detector or extractor of feature points to a different algorithm.
    https://github.com/EnoxSoftware/Mar...mple/MarkerLessAR/PatternDetector.cs#L97-L116
     
    kimvasquez17 likes this.
  39. wenqian157

    wenqian157

    Joined:
    Aug 10, 2021
    Posts:
    2
    WebCam mobile browser cant detect rear camera
    Hi I am trying WebGl example on mobile. on android it only detect front camera on ios no camera at all. but your example webgl player works perfectly fine. Any idea? Thanks
     
  40. kimvasquez17

    kimvasquez17

    Joined:
    Dec 19, 2017
    Posts:
    17
    Hi, unity provided documentation about that. Take a look here at how to properly execute the camera.
    https://docs.unity3d.com/2022.1/Documentation/Manual/webgl-browser-access-device.html
     
  41. wenqian157

    wenqian157

    Joined:
    Aug 10, 2021
    Posts:
    2
    how did the problem get fixed? I am experience the same with 2021 above....
     
  42. MatthewBodnerBoeing

    MatthewBodnerBoeing

    Joined:
    Jan 5, 2022
    Posts:
    1
    Unity 2020.3.28f1
    Import OpenCV 2.4.9
    Set build platform to UWP.

    Get multiple errors saying the DnnModule namespace does not exist.

    Assets\OpenCVForUnity\Examples\MainModules\dnn\LightweightPoseEstimationExample\LightweightPoseEstimationWebCamExample.cs(4,22): error CS0234: The type or namespace name 'DnnModule' does not exist in the namespace 'OpenCVForUnity' (are you missing an assembly reference?)

    This problem doesn't happen in older versions. But I guess older version didn't have LightweightPoseEstimation.

    Deleting Examples/Dnn removes the errors.
     
  43. Zalosath

    Zalosath

    Joined:
    Sep 13, 2014
    Posts:
    634
    Hi,
    I'm working on a project that detects ArUco markers in a picture and perspective warps the image based on their positions in the photo. The final goal is to have a "top down" almost view of a sheet of paper, here's some examples:

    Is transformed into:
    The resulting image is for some reason flipped vertically; this should be noted for all examples I'll show, I do not know why.
    This is less of a perspective fix and more of a rotational fix, but it works fine as you can see.

    This is a more extreme example of a more distorted image:


    But as you can see:
    It works. Bar of course some scale issues which if you have any additional advice for that'd be greatly appreciated.

    It starts to break down when I move away from computer generated images though, if I take one from my camera it just doesn't really do much. E.g.:


    Supposedly corrected:
    You can see that it flipped the image vertically, and rotated it a bit, but that's about it. In terms of actually perspective correcting, there's not much really.

    My code is a bit all over the place at the moment, this area is really not my forte, but I would really like it to be.

    Here's the code that I'm using for this:
    Code (CSharp):
    1.         // params and identification dictionary
    2.         DetectorParameters detectorParams = DetectorParameters.create();
    3.         Dictionary dictionary = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_250);
    4.  
    5.         // define new mat with texture dimensions
    6.         Mat mat = new Mat(texture.height, texture.width, CvType.CV_8UC3);
    7.         Utils.texture2DToMat(texture, mat);
    8.  
    9.         // declare out variables for detectMarkers
    10.         Mat ids = new Mat();
    11.         List<Mat> corners = new List<Mat>();
    12.         List<Mat> rejectedCorners = new List<Mat>();
    13.  
    14.         // detect the markers
    15.         Aruco.detectMarkers(mat, dictionary, corners, ids, detectorParams, rejectedCorners);
    16.         // draw markers onto mat
    17.         Aruco.drawDetectedMarkers(mat, corners, ids, new Scalar(0, 255, 0));
    18.         Aruco.drawDetectedMarkers(mat, rejectedCorners, ids, new Scalar(0, 255, 0));
    19.  
    20.         // loop the corners, pick 0,0 index of mat, corners is not always presorted
    21.         List<Point> coords = new List<Point>();
    22.         foreach (Mat m in corners)
    23.         {
    24.             coords.Add(new Point(m.get(0, 0)[0], m.get(0, 0)[1]));
    25.         }
    26.         coords = coords.OrderByDescending(y => y.y).ToList();
    27.         // first two values should be the bottom, second two should be the top
    28.         // now we just have to filter each one for left and right
    29.         Point temp;
    30.         if (coords[0].x > coords[1].x)
    31.         {
    32.             // if the first item is larger then it's actually bottom right, and we need to swap
    33.             temp = coords[1];
    34.             coords[1] = coords[0];
    35.             coords[0] = temp;
    36.             // if the first x is greater than the second x, swap them, so we get BL BR
    37.         }
    38.  
    39.         if (coords[2].x > coords[3].x)
    40.         {
    41.             // if the first top is larger than the second top then swap them
    42.             temp = coords[3];
    43.             coords[3] = coords[2];
    44.             coords[2] = temp;
    45.         }
    46.  
    47.         // ordered BR BL TR TL (i tested every combination of these, only 2 of the combinations worked, one flipped horizontally and the other flipped vertically, is this wrong?)
    48.         List<Point> points = new List<Point>()
    49.         {
    50.             coords[1],
    51.             coords[0],
    52.             coords[3],
    53.             coords[2]
    54.         };
    55.    
    56.         Mat src_mat = new Mat(4, 1, CvType.CV_32FC2);
    57.         Mat dst_mat = new Mat(4, 1, CvType.CV_32FC2);
    58.         src_mat.put(0, 0, 0.0, 0.0, 1360.0, 0.0, 0.0, 1024.0, 1360.0, 1024.0);
    59.         dst_mat.put(0, 0, points[0].x, points[0].y, points[1].x, points[1].y, points[2].x, points[2].y, points[3].x, points[3].y);
    60.  
    61.         Mat perspectiveTransform = Imgproc.getPerspectiveTransform(src_mat, dst_mat);
    62.  
    63.         Imgproc.warpPerspective(mat, mat, perspectiveTransform, new Size(texture.width, texture.height));
    It took me many attempts to get getPerspectiveTransform working, and sadly it could still be wrong. I'd love some insight into this if anyone has any idea.

    Cheers!
     
  44. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Thank you very much for reporting.
    The code was missing #if !UNITY_WSA_10_0 and an error occurred.
    Replacing it with the corrected attached code works fine.
     

    Attached Files:

  45. Zalosath

    Zalosath

    Joined:
    Sep 13, 2014
    Posts:
    634
    I fixed this, src and dst were the wrong way around, whoops.
     
  46. vice39

    vice39

    Joined:
    Nov 11, 2016
    Posts:
    102
    How can I convert an Texture that's R16_UNorm to OpenCV Mat? The function Utils.texture2DToMat won't do it. Any ideas?
     
  47. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    Code (CSharp):
    1.             Texture2D texture = new Texture2D(512, 512, GraphicsFormat.R16_UNorm, TextureCreationFlags.None);
    2.  
    3.             //GraphicsFormat.R16_UNorm : A one-component, 8-bit unsigned normalized format that has a single 8-bit R component.
    4.             ushort[] data = new ushort[texture.width * texture.height];
    5.             for (int i = 0; i < data.Length; i++)
    6.             {
    7.                 data[i] = 65535;
    8.             }
    9.             Debug.Log("data.Length " + data.Length);
    10.  
    11.             //Copy data to textures.
    12.             GCHandle arrayHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
    13.             texture.LoadRawTextureData(arrayHandle.AddrOfPinnedObject(), data.Length * 2);
    14.             arrayHandle.Free();
    15.  
    16.             //Initialize a mat of the same size and data type as texture.
    17.             Mat mat = new Mat(texture.height, texture.width, CvType.CV_16UC1);
    18.             Debug.Log("mat.total() " + mat.total());
    19.  
    20.             //The Utils.texture2DToMat() method requires the Mat data type to be CvType.CV_8U.
    21.             //However, the Utils.fastTexture2DToMat() method simply copies data without checking size or data type.
    22.             Utils.fastTexture2DToMat(texture, mat);
    23.  
    24.             Debug.Log("mat.dump() " + mat.dump());
     
  48. vice39

    vice39

    Joined:
    Nov 11, 2016
    Posts:
    102
    How do I copy an image into roi of another image?
    Here is some C++ code that does that:

    Code (CSharp):
    1. How do copy an image into roi of another image.
    2. Something like this:
    3. Mat img = cv::imread(...);
    4. Rect roi_input(x,y,w,h);
    5. Rect roi_output(x2,y2,w,h);
    6.  
    7. Mat sub_img= new Mat(img, roi_input); // take sub_img
    8. Mat new_img= Mat.zeros(img.size(), img.type); // create new img
    9.  
    10. sub_img.copyTo(newImage(roi_output)); // this line is what I can't find
    That last line, I can't find a way to do that with this asset. Any ideas?
     
  49. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,468
    You need to create a new_img submat.
    Code (CSharp):
    1.             Texture2D imgTexture = Resources.Load("face") as Texture2D;
    2.  
    3.             Mat img = new Mat(imgTexture.height, imgTexture.width, CvType.CV_8UC4);
    4.  
    5.             Utils.texture2DToMat(imgTexture, img);
    6.             Debug.Log("imgMat.ToString() " + img.ToString());
    7.  
    8.             Rect roi_input = new Rect(0, 0, 100, 100);
    9.             Rect roi_output = new Rect(300, 300, 100, 100);
    10.  
    11.             Mat sub_img = new Mat(img, roi_input); // take sub_img
    12.             Mat new_img = Mat.zeros(img.size(), img.type()); // create new img
    13.  
    14.             Mat sub_new_img = new Mat(new_img, roi_output);
    15.  
    16.             sub_img.copyTo(sub_new_img);
    17.  
    18.             Texture2D texture = new Texture2D(new_img.cols(), new_img.rows(), TextureFormat.RGBA32, false);
    19.  
    20.             Utils.matToTexture2D(new_img, texture);
    21.  
    22.             gameObject.GetComponent<Renderer>().material.mainTexture = texture;
     
unityunity