Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

OpenCVSharp for Unity

Discussion in 'Assets and Asset Store' started by NWHCoding, Oct 8, 2017.

  1. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    If you have 8UC3 mat you can convert it to 8UC4 by using:

    Cv2.CvtColor(mat, mat, ColorConversionCodes.BGR2BGRA);

    Only RGBA and 8UC4 are supported when converting since most functions use memory copy of some kind, and Unity only supports Texture32 and the default number of channels is 4 (RGBA) so many more functions are available for RGBA than RGB. 8UC3 to RGB texture conversion would be possible but much slower, even when you factor in using CvtColor().
     
  2. Grubheadhunter

    Grubheadhunter

    Joined:
    Oct 4, 2018
    Posts:
    2
    I have the same problem with all the demos. Is there a solution?
     
  3. weareinteractive

    weareinteractive

    Joined:
    Apr 14, 2017
    Posts:
    12
    You moved the StreamingAssets folder into Asset folder?
     
  4. Grubheadhunter

    Grubheadhunter

    Joined:
    Oct 4, 2018
    Posts:
    2
    yes
     
  5. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I assume you are using the latest version? Could you also write down the version of Unity you are using and I will check what might be happening. Also - are you on Windows, Linux or MacOS?
    I always check my assets before publishing for errors and every scene throwing errors would not make it into the store for sure.
     
  6. vicenziano

    vicenziano

    Joined:
    Sep 11, 2018
    Posts:
    4
    Hi! I've been trying to compile for iOS using Unity 2018.2.10. Problem is that I get the following error:

    Code (CSharp):
    1. Undefined symbols for architecture arm64:
    2.   "cv::createBackgroundSubtractorMOG2(int, double, bool)", referenced from:
    3.       _video_createBackgroundSubtractorMOG2 in OpenCvSharpExtern(video.o)
    4.   "cv::boundingRect(cv::_InputArray const&)", referenced from:
    5.       _imgproc_boundingRect_Point in OpenCvSharpExtern(imgproc.o)
    6.   "cv::findContours(cv::_InputOutputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, int, int, cv::Point_<int>)", referenced from:
    7.       _imgproc_findContours1_vector in OpenCvSharpExtern(imgproc.o)
    8.   "cv::moments(cv::_InputArray const&, bool)", referenced from:
    9.       _imgproc_moments in OpenCvSharpExtern(imgproc.o)
    10.   "cv::adaptiveThreshold(cv::_InputArray const&, cv::_OutputArray const&, double, int, int, int, double)", referenced from:
    11.       _imgproc_adaptiveThreshold in OpenCvSharpExtern(imgproc.o)
    12.   "cv::circle(cv::_InputOutputArray const&, cv::Point_<int>, int, cv::Scalar_<double> const&, int, int, int)", referenced from:
    13.       _imgproc_circle in OpenCvSharpExtern(imgproc.o)
    14.   "cv::threshold(cv::_InputArray const&, cv::_OutputArray const&, double, double, int)", referenced from:
    15.       _imgproc_threshold in OpenCvSharpExtern(imgproc.o)
    16.   "cv::ellipse(cv::_InputOutputArray const&, cv::Point_<int>, cv::Size_<int>, double, double, double, cv::Scalar_<double> const&, int, int, int)", referenced from:
    17.       _imgproc_ellipse1 in OpenCvSharpExtern(imgproc.o)
    18.   "cv::HoughLinesP(cv::_InputArray const&, cv::_OutputArray const&, double, double, int, double, double)", referenced from:
    19.       _imgproc_HoughLinesP in OpenCvSharpExtern(imgproc.o)
    20.   "cv::CascadeClassifier::CascadeClassifier(cv::String const&)", referenced from:
    21.       _objdetect_CascadeClassifier_newFromFile in OpenCvSharpExtern(objdetect.o)
    22.   "cv::GaussianBlur(cv::_InputArray const&, cv::_OutputArray const&, cv::Size_<int>, double, double, int)", referenced from:
    23.       _imgproc_GaussianBlur in OpenCvSharpExtern(imgproc.o)
    24.   "cv::Mat::create(int, int const*, int)", referenced from:
    25.       _core_Mat_new2 in OpenCvSharpExtern(core.o)
    26.       _core_Mat_new3 in OpenCvSharpExtern(core.o)
    27.   "cv::imread(cv::String const&, int)", referenced from:
    28.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    29.   "cv::dilate(cv::_InputArray const&, cv::_OutputArray const&, cv::_InputArray const&, cv::Point_<int>, int, int, cv::Scalar_<double> const&)", referenced from:
    30.       _imgproc_dilate in OpenCvSharpExtern(imgproc.o)
    31.   "cv::flip(cv::_InputArray const&, cv::_OutputArray const&, int)", referenced from:
    32.       _core_flip in OpenCvSharpExtern(core.o)
    33.   "cv::SimpleBlobDetector::Params::Params()", referenced from:
    34.       _features2d_SimpleBlobDetector_create in OpenCvSharpExtern(features2d.o)
    35.   "cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)", referenced from:
    36.       _imgproc_cvtColor in OpenCvSharpExtern(imgproc.o)
    37.   "cv::SimpleBlobDetector::create(cv::SimpleBlobDetector::Params const&)", referenced from:
    38.       _features2d_SimpleBlobDetector_create in OpenCvSharpExtern(features2d.o)
    39.   "cv::CascadeClassifier::detectMultiScale(cv::_InputArray const&, std::__1::vector<cv::Rect_<int>, std::__1::allocator<cv::Rect_<int> > >&, double, int, int, cv::Size_<int>, cv::Size_<int>)", referenced from:
    40.       _objdetect_CascadeClassifier_detectMultiScale1 in OpenCvSharpExtern(objdetect.o)
    41.   "cv::CascadeClassifier::~CascadeClassifier()", referenced from:
    42.       _objdetect_CascadeClassifier_delete in OpenCvSharpExtern(objdetect.o)
    43.   "cv::Mat::operator=(cv::Scalar_<double> const&)", referenced from:
    44.       _core_Mat_new3 in OpenCvSharpExtern(core.o)
    45.   "cv::drawKeypoints(cv::_InputArray const&, std::__1::vector<cv::KeyPoint, std::__1::allocator<cv::KeyPoint> > const&, cv::_InputOutputArray const&, cv::Scalar_<double> const&, int)", referenced from:
    46.       _features2d_drawKeypoints in OpenCvSharpExtern(features2d.o)
    47.   "cv::cuda::GpuMat::release()", referenced from:
    48.       _cuda_GpuMat_delete in OpenCvSharpExtern(cuda.o)
    49.   "cv::FAST(cv::_InputArray const&, std::__1::vector<cv::KeyPoint, std::__1::allocator<cv::KeyPoint> >&, int, bool)", referenced from:
    50.       _features2d_FAST1 in OpenCvSharpExtern(features2d.o)
    51.   "cv::line(cv::_InputOutputArray const&, cv::Point_<int>, cv::Point_<int>, cv::Scalar_<double> const&, int, int, int)", referenced from:
    52.       _imgproc_line in OpenCvSharpExtern(imgproc.o)
    53.   "cv::error(int, cv::String const&, char const*, char const*, int)", referenced from:
    54.       cv::Mat::Mat(int, int, int, void*, unsigned long) in OpenCvSharpExtern(core.o)
    55.   "cv::Canny(cv::_InputArray const&, cv::_OutputArray const&, double, double, int, bool)", referenced from:
    56.       _imgproc_Canny in OpenCvSharpExtern(imgproc.o)
    57.   "cv::String::deallocate()", referenced from:
    58.       cv::Mat::Mat(int, int, int, void*, unsigned long) in OpenCvSharpExtern(core.o)
    59.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    60.       _objdetect_CascadeClassifier_newFromFile in OpenCvSharpExtern(objdetect.o)
    61.   "cv::Mat::copySize(cv::Mat const&)", referenced from:
    62.       _core_Mat_clone in OpenCvSharpExtern(core.o)
    63.       entity(cv::Mat*) in OpenCvSharpExtern(features2d.o)
    64.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    65.   "cv::subtract(cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&, cv::_InputArray const&, int)", referenced from:
    66.       _core_subtract_InputArray2 in OpenCvSharpExtern(core.o)
    67.   "cv::fastFree(void*)", referenced from:
    68.       _core_Mat_delete in OpenCvSharpExtern(core.o)
    69.       _core_Mat_clone in OpenCvSharpExtern(core.o)
    70.       _features2d_Feature2D_detect_Mat1 in OpenCvSharpExtern(features2d.o)
    71.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    72.       _imgproc_boundingRect_Point in OpenCvSharpExtern(imgproc.o)
    73.   "cv::add(cv::_InputArray const&, cv::_InputArray const&, cv::_OutputArray const&, cv::_InputArray const&, int)", referenced from:
    74.       _core_add in OpenCvSharpExtern(core.o)
    75.   "cv::Mat::deallocate()", referenced from:
    76.       _core_Mat_delete in OpenCvSharpExtern(core.o)
    77.       _core_Mat_clone in OpenCvSharpExtern(core.o)
    78.       _features2d_Feature2D_detect_Mat1 in OpenCvSharpExtern(features2d.o)
    79.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    80.       _imgproc_boundingRect_Point in OpenCvSharpExtern(imgproc.o)
    81.   "cv::String::allocate(unsigned long)", referenced from:
    82.       cv::Mat::Mat(int, int, int, void*, unsigned long) in OpenCvSharpExtern(core.o)
    83.       _imgcodecs_imread in OpenCvSharpExtern(imgcodecs.o)
    84.       _objdetect_CascadeClassifier_newFromFile in OpenCvSharpExtern(objdetect.o)
    85.   "cv::redirectError(int (*)(int, char const*, char const*, char const*, int, void*), void*, void**)", referenced from:
    86.       _redirectError in OpenCvSharpExtern(core.o)
    87.   "cv::Mat::copyTo(cv::_OutputArray const&) const", referenced from:
    88.       _core_Mat_clone in OpenCvSharpExtern(core.o)
    89.   "cv::rectangle(cv::Mat&, cv::Rect_<int>, cv::Scalar_<double> const&, int, int, int)", referenced from:
    90.       _imgproc_rectangle_Mat in OpenCvSharpExtern(imgproc.o)
    91.   "cv::noArray()", referenced from:
    92.       _core_add in OpenCvSharpExtern(core.o)
    93.       _core_subtract_InputArray2 in OpenCvSharpExtern(core.o)
    94.       _imgproc_dilate in OpenCvSharpExtern(imgproc.o)
    95. ld: symbol(s) not found for architecture arm64
    96. clang: error: linker command failed with exit code 1 (use -v to see invocation)
    97.  
    How can I solve this?
     
  7. StaffanEk

    StaffanEk

    Joined:
    Jul 13, 2012
    Posts:
    380
    I'm having issues getting floating Points from the FindContours function. I can get the integer Point struct array but I can't get floating point values.

    I get this error message when I try to return a Mat for instance: Cannot convert from out 'OpenCvSharp.Mat[]' to 'OpenCvSharp.Point[][]' .

    There are two overloads for the FindContours function one that should return an array of Point arrays, and another that should return an array of Mat's.

    Are the overrides created incorrectly, or am I just being stupid?

    Code (CSharp):
    1.         Mat[] featureContours;
    2.  
    3.         Cv2.FindContours(image: mat, contours: out featureContours, hierarchy: m_Hierarchy, mode: RetrievalModes.External, method: ContourApproximationModes.ApproxTC89KCOS, offset: null);
    4.  
    Can you help me out?
     
  8. cjf-inc

    cjf-inc

    Joined:
    Aug 18, 2014
    Posts:
    10
    Hello,

    I bought the plugin recently and I'm trying to build demo scene for iOS.
    I get the following error in the Xcode

    Code (CSharp):
    1. /clang:-1: linker command failed with exit code 1 (use -v to see invocation)
    It makes this plugin useless :/
    I need assistance.

    Thanks in advance
     
  9. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    Hello,
    are you sure your platform is correctly set up? Does the problem appear when building for simulator or device itself?
    Could you also run "lipo -info [pathToFramework]/OpenCvSharpForUnity.framework/OpenCvSharpForUnity" to see if you have the correct architectures in the framework.
    Sorry for the complications.
     
  10. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    A few examples that might help:
    There are following overloads / functions available:
    • void FindContours(out Point[][] contours, out HierarchyIndex[] hierarchy, RetrievalModes mode, ContourApproximationModes method, Point? offset = null)
    • void FindContours(out Mat[] contours, OutputArray hierarchy,
      RetrievalModes mode, ContourApproximationModes method, Point? offset = null)
    • Point[][] FindContoursAsArray(RetrievalModes mode, ContourApproximationModes method, Point? offset = null)
    • MatOfPoint[] FindContoursAsMat(RetrievalModes mode, ContourApproximationModes method, Point? offset = null)
     
  11. cjf-inc

    cjf-inc

    Joined:
    Aug 18, 2014
    Posts:
    10
    Hi,

    Thank you for your reply.

    What do you mean by that?
    I created new project in unity 2018.2.14f , imported opencv plugin and trying to create an iOS build to run demo scene to run on the device.
    The architectures I can see in the Xcode project are armv7 and arm64.
    https://monosnap.com/file/LlVtMy5C86HF2iyWepxx92f2omTZhT

    For device.

    I guess you mean OpenCvSharpExtern.framework.
    Here is results:
    Code (CSharp):
    1. $ lipo -info open-cv-test/Assets/OpenCVSharpForUnity/Plugins/iOS/OpenCvSharpExtern.framework/OpenCvSharpExtern
    2. Architectures in the fat file: open-cv-test/Assets/OpenCVSharpForUnity/Plugins/iOS/OpenCvSharpExtern.framework/OpenCvSharpExtern are: armv7s armv7 i386 x86_64 arm64
    Also, I sent you a project archive you can test in the direct message.
     
  12. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    As you can see from the lipo data both device and simulator builds are in there, bundled into one framework. My guess is that, most likely due to incorrect Unity build settings, the required architectures do not get added to the build. I would suggest checking the framework file itself inside Unity and the platforms/architectures that are selected there and also build settings under target platform (or whatever the name might be).
    I am suspecting either some xcode compatibility or wrong setting in either Unity or Xcode since the demo was built from the exact same files that are provided in the asset.

    Now that I mention the demo, does it work for you on iOS?
     
  13. cjf-inc

    cjf-inc

    Joined:
    Aug 18, 2014
    Posts:
    10
    Here is the data from the Xcode project folder
    Code (CSharp):
    1. $ lipo -info open-cv-test/build/Frameworks/OpenCVSharpForUnity/Plugins/iOS/OpenCvSharpExtern.framework/OpenCvSharpExtern
    2. Architectures in the fat file: open-cv-test/build/Frameworks/OpenCVSharpForUnity/Plugins/iOS/OpenCvSharpExtern.framework/OpenCvSharpExtern are: armv7s armv7 i386 x86_64 arm64
    I have an armv7 and arm64 architectures selected in the Xcode project right now.


    I just dowloaded your iOS project from the asset store link https://drive.google.com/open?id=1uwAvJ3i-SyIJqHmbH3lJYgXLdNWcB2JV
    And I see the same error as before.
    I have Xcode v10.0
    What Xcode version do you use to create iOS build?
     
  14. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I will check out what might be happening but if the Unity version is correct it could be that someting with Xcode version mismatch might be the issue. Plugin was built with Xcode 9. After checking some other projects, one of which from Google, they seem to have had the same error for the package built in 9.x used in 10.x Xcode. Will fire up my MacOS machine and see what might be the issue. I do not have an iOS device itself but if it is the problem with those versions then it should happen in simulator too.
     
  15. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    There seems to be a bug in the current version of OpenCV that some people have encountered, while for the rest the asset worked fine. Some users got a warning about incorrect texture size being passed to the conversion function inside some of the example scenes. The problem here is with the Unity class WebCamTexture which (with some cameras, on some OSes) does not return the correct width and height if one frame was not taken previous to querying the dimensions. There are quite a few threads on this (such as this one) but since everything worked fine in Unity 2017.1.1 even without the fix I assumed Unity finally patched it and that we were good to go without the fix. Guess not.

    So if you are encountering this problem (which will be patched) it is nothing to worry about. WebCamTexture.width returns 16x16 (or sometimes 8x8) dimensions no matter what the camera resolution is and the Mat in the Start() function gets set to those dimensions. After first camera grab WebCamTexture.width jumps to correct resolution but leaves the Mat at 16x16 and you end up with mismatched Texture2D and Mat being passed to conversion function which then throws error to prevent memory overwrite/read.

    Solution
    One of solutions would be to simply wait for camera to initialize which can be up to 0.5s:

    Code (CSharp):
    1.  
    2. private bool initializedMat = false;
    3.  
    4. void Update()
    5. {
    6.         if(webCamTexture.width < 100f)
    7.         {
    8.             // Camera not yet initialized, wait.
    9.             return;
    10.         }
    11.         else if(!initializedMat && webCamTexture.width > 100f)
    12.         {
    13.             // Camera initialized, initialize mat to correct size.
    14.             mat = new Mat(webCamTexture.height, webCamTexture.width, MatType.CV_8UC4);
    15.  
    16.             // Set to false as to prevent mat from initializing every frame.
    17.             initializedMat = true;
    18.         }
    19.  
    20.        // Your code here...
    21. }
    22.  
    Or you could just initialize the Mat each frame to keep with the camera resulution change. Not really recommended for performance/memory reasons:


    Code (CSharp):
    1.  
    2. void Update()
    3. {
    4. mat = new Mat(webCamTexture.height, webCamTexture.width, MatType.CV_8UC4);
    5. // Your code here...
    6. }
    7.  
    Not really elegant but simple and effective.
     
    Last edited: Nov 5, 2018
    FlashyGoblin likes this.
  16. cjf-inc

    cjf-inc

    Joined:
    Aug 18, 2014
    Posts:
    10

    Hi again.
    The problem I faces is nothing related to warning about webcam texture size, but that I literally can't create a build for IOS in the Xcode.
    As I wrote earlier - I see the next error in the Xcode 10
    Code (CSharp):
    1. /clang:-1: linker command failed with exit code 1 (use -v to see invocation)
    Have you tried to create iOS build in the Xcode?
    Do you see the same error message?
     
    NWHCoding likes this.
  17. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I did understand and I am also talking with one more customer that has the same problem using Xcode 10. The asset was tested and working with Xcode 9 and Unity 2017.1.1 so something is most likely wrong with the versions.
    Will check with Xcode 10 and see what I can do about it.
     
    FlashyGoblin likes this.
  18. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Hi,

    Just to chime in: I'm also seeing the clang linker issue when I try to build the example project. Let me know if there's any info I can provide that might help track this down.
     
  19. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    The error is confirmed to be due to Xcode 10 / MacOS 10.14 (one or both) and the builds pass on Xcode 9.2 with MacOS High Sierra. My guess is they changed something between those versions but am having problems updating to 10.14 at the moment.

    Here are a few things you can try at the moment:
    - [Unity] Make sure your project is in Release (since all the libraries are Release-only)
    - [Unity] iOS target should be 9.0
    - Architecture one of the following
    • armv7s armv7 arm64 (device)
    • i386 x86_64 (simulator)
    - "lipo -info OpenCvSharp.framework/OpenCvSharp" returns a list of architectures (all 5 from point above)
    - Scripting backend - IL2CPP

    Those settings are confirmed to work on Xcode 9.2 / High Sierra.
     
  20. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Just tried the project using Release, 9.0, armv7s, IL2CPP, but still had the linker issue. If I have time, I'll try with Xcode 9.2 on Thursday
     
  21. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    Which MacOS version and does simulator build work?
    Also, make sure that Runtime Types inside Xcode are enabled ('rtti' under search). They should be by default in Xcode.
     
  22. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    I'm on 10.14.1. I just checked and Enable C++ Runtime Types was disabled. However enabling it did not fix the problem. The Simulator build did work, however it crashed on start up.

    The crash occurred in what the comment labeled as System.IntPtr OpenCvSharp.NativeMethods::core_Mat_new3(System.Int32,System.Int32,System.Int32,OpenCvSharp.Scalar)

    On a second attempt the simulator crashed due to a deadlock. The third time the crash was the same as the 1st. The project runs fine in Unity.

    Output from first Simulator crash:
     
    Last edited: Nov 15, 2018
  23. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    I've just tried building in Xcode 9.4.1 on MacOS 10.14.1 and the build failed with the same issue. I got far more detailed output though.

     
  24. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I do have an idea what might be happening there and I will see to it but had to clean reinstall MacOS High Sierra after update got me stuck in boot loop (might have modified a bit too much of the system there).
    And I do want to test with 10.14 because using Xcode 9.2 and MacOS High Sierra, just before I tried the update, it worked fine for simulator build:

    image.png

    One more thing that could cause problems is that the Xcode should be in Release mode because all the frameworks are built as Release so it may be searching for debug version of framework and not finding it.
     
  25. cjf-inc

    cjf-inc

    Joined:
    Aug 18, 2014
    Posts:
    10
    Hi again,

    I tried to enable rtti flag, but it doesn't help.
    I'm building in the Release mode.
    Build failing to create.

    Looking forward to solution.
     
  26. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I have just restored my backup after failed MacOS update and have been able to replicate the error by downloading Unity 2018.2.16f1 on MacOS 10.13.1 with XCode 9.2 which means that this is most likely not version specific. To replicate I created a new empty project and imported the asset. Selected target as Simulator and the error is there.
    At the same time the error does not happen using other project with the files that were used to create the build and Unity 2017.3.1f1. From this I would say the settings are the most likely culprit OR there is a wrong framework file included with the asset. Will investigate and hopefully have a solution in a few hours.
     
  27. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    Problem found and corrected. I will push the update tomorrow after I check that everything works fine. It will take two or three days to appear in the store.

    Now on to the problem. Well, I must admit I messed this one up in the last update and no amount of settings tweaking would have fixed it. In short - I have not included actual OpenCV library for iOS, only the wrapper. There was supposed to be a copy framework phase inside XCode build settings which did not happen for some reason or other and the test project on MacOS machine had the OpenCV.framework inside the folder as part of development. Later, when moving the files to the main Windows PC only the wrapper was moved.

    Sorry for all the problems this has caused. I know people in here are on tight schedule and things like this are not what should take up their time. If you need the fix sooner contact me over email and I will send it over.
     
  28. flintmech

    flintmech

    Joined:
    Sep 29, 2011
    Posts:
    32
    I'm trying to use this plugin with the latest OpenCVSharp DLL(s) from shimat's Github and I'm unable to do the Webcam texture to Mat conversion operation. Color32ArrayToMat calls mat.SetArray after forcing the mat to be type CV_8UC4, but OpenCVSharp prevents CV_8UC4 Mats from being converted with that method - the SetArray that's called invokes CheckArgumentsForConvert which specifies CV_8UC1 as the only valid type of Mat for this operation, and an OpenCVException is thrown.

    Is this Unity plugin based on shimat's library? (I assumed it was because of the matching DLL name). Or is it your own custom C# port of OpenCV? If this is built off of his library, how do you get around this MatType restriction? The reason I'm using shimat's OpenCVSharp DLLs is because I'm additionally building a class library outside of Unity, kind of a second plugin for my specific needs which relies on OpenCV, and my Unity game depends on both of these, so I'm trying to impose some synergy. If this Unity plugin is wholly incompatible with shimat's DLLs I'll understand and try to find a workaround.
     
  29. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    The plugin is definitely based on shimat's library and all the wrapped code is pretty much intact. Most of the changes are related to loading libraries and there are changes to the code in some areas. To get around this check the SetArray() function inside the wrapper and you will see that shimat is checking if conversion is valid, and by default only 8UC1 is. I do not remember which line of code was in question exactly but there are 4 lines or so.
    If you can I would recommend keeping the plugin close to stock if possible, especially if you are planning on doing multi-platform builds. Trust me, it will save you a lot of headaches.
     
  30. flintmech

    flintmech

    Joined:
    Sep 29, 2011
    Posts:
    32
    @NWHCoding Oh believe me, I'd love to avoid deviating from stock where possible. As things are currently, the examples you provide which make use of WebCamTexture no longer work with recent OpenCVSharp DLLs because CV_8UC4 is used. I was mainly curious if this was something you were aware of and/or knew of a workaround for, and if not, consider this a bug report I guess.

    (BTW, I tried inspecting the git history of shimat's Mat.cs on Github and, unless I missed something, the critical lines of code (specifying CV_8UC1 as the only valid type for the SetArray that takes an array of bytes) has not changed at all since the first commit, so I'm even more curious now where this disconnect is coming from.
     
  31. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    You misunderstood me there. The asset was never intended to be used with OpenCvSharp DLLs since it already contains them. You can not just take DLLs from one commit and use the rest of the code from other commit.

    And as I mentioned in the last message I dir modify OpenCvSharp dlls, and amongst things that were changed is the addition of 8UC4 as a valid SetArray() parameter.
     
  32. flintmech

    flintmech

    Joined:
    Sep 29, 2011
    Posts:
    32
    Gotcha, thanks for the clarification.
     
  33. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Has the StreamingAssets folder disappeared in the latest version?
     
  34. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    Yes it has, and not intentionally. Will push the fix in a few minutes.

    In the meantime here is the folder for anyone needing it.
     

    Attached Files:

  35. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Thanks! I thought it was odd that it had gone.

    Unfortunately I'm still seeing linker issues, but they have changed, which is good. It might be something wrong with my setup.

    The full linker error is below, but this seems to be the key line:

    Any idea what might be causing it? This Stack Overflow article suggests it might be because the Simulator libraries are being selected by the linker when trying to build for the device. That makes sense: the project is building fine for the Xcode Simulator. My Unity player settings are set to Device SDK, 9.0 ARM64.

    In full:
     
  36. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Is anyone else getting these linker errors with the new version or is it something particular to my setup?
     
  37. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    My guess is that it is setup related. Or at least I hope so. I made sure that all the files for iOS are there and intact when moving from MacOS machine on which it was tested to my PC from which it was published.
    If "lipo -info opencv2.framework/opencv2" returns all the 5 architectures it should be good to go. Have you tried with empty project and/or simulator build?

    I do not have actual iOS device so the testing was done on simulator but a customer I was in contact with when I released the update said he did not get any errors any more after updating, that is why I suspect it might be setup issue.
     
  38. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Oddly the lipo command returned:
    I'll try creating a new project and running the lipo command on that
     
  39. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    No luck. Just to check I am not going mad, here's the command I am running:
     
  40. personalnadir

    personalnadir

    Joined:
    Oct 22, 2013
    Posts:
    11
    Steps so far:
    - created a new project, reimported the package
    - updated Unity and created a new project
    - created a project, deleted the plugin and reimported it
    - deleted the downloaded version (the NWH Coding folder in ⁨Library⁩ ▸ ⁨Unity⁩ ▸ ⁨Asset Store-5.x⁩) of the plugin, created a new project, downloaded and imported the plugin

    All so far with the same results when running lipo
     
  41. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I managed to replicate the problem.
    This is the result I get with the original project:

    upload_2018-12-1_17-14-8.png

    But when I download the asset I get the same unknown architecture error.
    After doing some digging it seems that Windows version of Unity sees .framework file as folder (which it is) and therefore imports all the files as separate files and adds a lot of .meta files. This corrupts framework file and afterwards it can not be opened in MacOS again. The customer which said that the update fixed it for him got the compressed file before the release of the update and therefore did not have this problem.

    In the future iOS folder will come as .unitypackage to avoid errors such as this one.

    You seem to have PMs disabled so please send an email to nwhcoding@gmail.com and I wil send you the unitypackage with fixed iOS framework.

    Expect update in two to three days. Will upload it immediately but it takes some time for it to get reviewed.
     
  42. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,272
    That's not Windows version specific - Unity on mac adds (.)meta files as needed, as well
    And there's no way added (.)meta files would in any way "corrupt" the framework
    (I build and distribute mac native bundle in package and have never encountered something like that)

    Distributing (any) parts of the asset as .unitypackage might get you submission to be rejected (and rightfully so; yea I know _some_ assets do it, but it's not a rule)

    I don't know what the problem is here, but I'd start with building on a _real_ apple hardware, using macOS version of Unity, and testing on _real_ apple device as well
     
  43. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I never encountered it either but it could be that it got corrupt in some way while uploading the asset. There was no problem in previous versions with this.
    Actually, now that you mention it I will try to resubmit and import the package asset uploader creates directly into Unity on MacOS to make sure the corruption does not happen again.

    Testing on real Apple iPhone is out of question since the prices of those are astronomical and buying one just for testing one asset would not make much sense financially. Also, iPhones are quite rare around here (Croatia) so it is not just a matter of borrowing one.

    So hang in there, fix will be out in a few days. If you need it sooner send me an email and I will see that you get it in a few hours.
     
    Last edited: Dec 1, 2018
  44. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,272
    if you refuse to actually test on physical device then don't claim your asset supports it - it's as easy as that
    I'm writing this in good faith, but if you claim that your asset supports iOS while actually never run on it might be considered - deliberately - misleading - and I'm really sorry to write that

    claiming that iPhones are rare in Croatia is just ridiculous, sorry :)

    It's not worth pushing this further for me currently, so fix this and *strongly* consider physical device - it really makes a difference
     
    magicherb likes this.
  45. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    I will and thank you for suggestion.

    I have had previous version (1.3) tested on an actual device (by 3rd party) and not much has changed since then. Still, each version requires a complete rebuild of all libraries for all platforms and sometimes errors happen - such as this time. A day or two and you will have a working version.
    As for this asset and iOS, there have been 0 problems from 1.1 to now, and that is just because I managed to lose one quite important file in the copying of the files between OS-es. Completely a human error there and sorry for all the complications that has caused.
     
  46. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,272
    no worries - glad the outcome is positive :)
    ( apple stuff can break even with point updates sometimes, that's why it's good idea to actually use their gear )

    Cheers!
     
  47. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    The 1.4.3 (OpenCV 3.4.1) update is out!

    .framework for iOS reads correct architectures and runs as it should.
    Tested in empty scene with default settings and clean asset store import. Not making the same mistake again by assuming that the store version = local version.
     
  48. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,272
    What was the actual problem btw ?

    Do you intent to look at OpenCV 4.0 in the near future ?
     
  49. NWHCoding

    NWHCoding

    Joined:
    Jul 12, 2012
    Posts:
    1,656
    Problem was in .framework getting corrupted either due to being compressed before upload to cloud, being decompressed and imported into Unity on Windows (from where I publish the pakckage) or getting corrupt while uploading to the store. I tested it on MacOS and local copy worked fine while the asset store copy had a framework file that was not recognizable by the system as a framework and that is why I thought it was working while in fact it was not.

    As for 4.0 - I will give it a few months since it is relatively fresh out of the box and therefore be probably quite buggy. I will give the wrapper a few versions to make sure it is stable and then update. Updating takes me about 5 days of full-time work so I usually want to make sure there are no large bugs before I build all the libraries.
     
  50. r618

    r618

    Joined:
    Jan 19, 2009
    Posts:
    1,272
    Fair enough, thanks !