Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] OpenCV for Unity

Discussion in 'Assets and Asset Store' started by EnoxSoftware, Oct 30, 2014.

  1. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Thank you very much for reporting.
    Could you tell me about your test environment?
    Unity version :
    OpenCVForUnity version :
    MarkerLessARExample version :
    test browser :
     
  2. Idealisers

    Idealisers

    Joined:
    Jun 26, 2014
    Posts:
    10
    Hi,

    I'm trying to install and run the openpose example on a Mac : when retrieving the git example zip and importing the asset from the asset store I get the following error(s):
    Assets/OpenCVForUnity/Examples/MainModules/dnn/CaffeExample/MobileNetSSDWebCamTextureExample.cs(59,9): error CS0246: The type or namespace name `Net' could not be found. Are you missing an assembly reference?

    What can I do with this particular case? The examples that are present from the default asset store import are of an older version so I need to use the latest git repo version ( which has the openpose example )

    EDIT : Could be that the unity asset store version does not contain the dnn lib?
     
    Last edited: May 16, 2018
  3. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Unity version : 2017.4.3f1
    OpenCVForUnity version : OpenCV for Unity 2.2.8
    MarkerLessARExample version : MarkerLess AR Sample 1.0.3
    test browser : Firefox 59.0.2 (64-bit), Edge (41.16299.371.0)

    Couldn't test in chrome since it doesn't support non hosted unity apps.
     
  4. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Is the version of OpenCVForUnity 2.2.7 or higher?
    OpenPoseExample requires OpenCVForUnity 2.2.7 or higher.
     
  5. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    I tried MarkerLessExample in the same version as your environment, but it worked without problems.
    Is ImportSettings set correctly?
    MenuItem.png
    webgl_56_ImportSettings.png
     
  6. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    The build fails when I click to set plugin import settings:

    Code (CSharp):
    1. Exception: Failed building WebGL Player.
    2. UnityEditor.WebGL.ProgramUtils.StartProgramChecked (System.Diagnostics.ProcessStartInfo p) (at /Users/builduser/buildslave/unity/build/PlatformDependent/WebGL/Extensions/Unity.WebGL.extensions/ProgramUtils.cs:48)
    3. UnityEditor.WebGL.WebGlBuildPostprocessor.EmscriptenLink (BuildPostProcessArgs args, Boolean wasmBuild, System.String sourceFiles, System.String sourceFilesHash) (at /Users/builduser/buildslave/unity/build/PlatformDependent/WebGL/Extensions/Unity.WebGL.extensions/BuildPostprocessor.cs:408)
    4. UnityEditor.WebGL.WebGlBuildPostprocessor.LinkBuild (BuildPostProcessArgs args) (at /Users/builduser/buildslave/unity/build/PlatformDependent/WebGL/Extensions/Unity.WebGL.extensions/BuildPostprocessor.cs:451)
    5. UnityEditor.WebGL.WebGlBuildPostprocessor.PostProcess (BuildPostProcessArgs args) (at /Users/builduser/buildslave/unity/build/PlatformDependent/WebGL/Extensions/Unity.WebGL.extensions/BuildPostprocessor.cs:916)
    6. UnityEditor.PostprocessBuildPlayer.Postprocess (BuildTargetGroup targetGroup, BuildTarget target, System.String installPath, System.String companyName, System.String productName, Int32 width, Int32 height, BuildOptions options, UnityEditor.RuntimeClassRegistry usedClassRegistry, UnityEditor.BuildReporting.BuildReport report) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/PostprocessBuildPlayer.cs:272)
    7. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
    8.  
    Code (CSharp):
    1. Failed running "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten_Win\python\2.7.5.3_64bit\python.exe" "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc" @"F:\Lucas IF\Unity projects\Science\WebAR_OpenCV_u2017.4.3f1\Assets\..\Temp\emcc_arguments.resp"
    2.  
    3. stdout:
    4. stderr:WARNING:root:--separate-asm works best when compiling to HTML. otherwise, you must yourself load the '.asm.js' file that is emitted separately, and must do so before loading the main '.js` file
    5. emcc: warning: cannot represent a NaN literal '0x251c0848070' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    6.  in   store float 0x7FFFFFFFE0000000, float* %426, align 4, !tbaa !49 in _ZN2cv3powERKNS_11_InputArrayEdRKNS_12_OutputArrayE()
    7. emcc: warning: cannot represent a NaN literal '0x251c0848070' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    8.  in   store float 0x7FFFFFFFE0000000, float* %436, align 4, !tbaa !49 in _ZN2cv3powERKNS_11_InputArrayEdRKNS_12_OutputArrayE()
    9. emcc: warning: cannot represent a NaN literal '0x251c0847570' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    10.  in   store double 0x7FFFFFFFFFFFFFFF, double* %468, align 8, !tbaa !2 in _ZN2cv3powERKNS_11_InputArrayEdRKNS_12_OutputArrayE()
    11. emcc: warning: cannot represent a NaN literal '0x251c0847570' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    12.  in   store double 0x7FFFFFFFFFFFFFFF, double* %478, align 8, !tbaa !2 in _ZN2cv3powERKNS_11_InputArrayEdRKNS_12_OutputArrayE()
    13. emcc: warning: cannot represent a NaN literal '0x251c2aa1230' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    14. emcc: warning: cannot represent a NaN literal '0x251c2aa1230' with custom bit pattern in NaN-canonicalizing JS engines (e.g. Firefox and Safari) without erasing bits!
    15. warning: unresolved symbol: tracking_TrackerCSRT_deletewarning: unresolved symbol: _Z22Mat_to_vector_MatShapeRN2cv3MatERNSt3__26vectorINS3_IiNS2_9allocatorIiEEEENS4_IS6_EEEEwarning: unresolved symbol: dnn_Dnn_imagesFromBlob_10warning: unresolved symbol: tracking_TrackerCSRT_create_10warning: unresolved symbol: _Z15Mat_to_MatShapeRN2cv3MatERNSt3__26vectorIiNS2_9allocatorIiEEEEwarning: unresolved symbol: imgproc_Imgproc_cvtColorTwoPlane_10warning: unresolved symbol: _ZN2cv16phase_unwrapping24HistogramPhaseUnwrapping6createERKNS1_6ParamsEwarning: unresolved symbol: imgproc_Imgproc_HoughLinesPointSet_10warning: unresolved symbol: _ZN2cv16phase_unwrapping24HistogramPhaseUnwrapping6ParamsC1Evwrote symbol map file to F:\Lucas IF\Unity projects\Science\WebAR_OpenCV_u2017.4.3f1\Temp\StagingArea\Data\linkresult_asm\build.js.symbols
    16. Traceback (most recent call last):
    17.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc", line 13, in <module>
    18.    emcc.run()
    19.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc.py", line 1972, in run
    20.    JSOptimizer.flush()
    21.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc.py", line 1868, in flush
    22.    run_passes(chunks[0], title, just_split=False, just_concat=False)
    23.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc.py", line 1841, in run_passes
    24.    final = shared.Building.js_optimizer(final, passes, debug_level >= 4, JSOptimizer.extra_info, just_split=just_split, just_concat=just_concat)
    25.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\shared.py", line 1828, in js_optimizer
    26.    ret = js_optimizer.run(filename, passes, NODE_JS, debug, extra_info, just_split, just_concat)
    27.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\js_optimizer.py", line 559, in run
    28.    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
    29.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\tempfiles.py", line 78, in run_and_clean
    30.    return func()
    31.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\js_optimizer.py", line 559, in <lambda>
    32.    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
    33.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\js_optimizer.py", line 451, in run_on_js
    34.    filenames = pool.map(run_on_chunk, commands, chunksize=1)
    35.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten_Win\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 250, in map
    36.    return self.map_async(func, iterable, chunksize).get()
    37.  File "F:\Unity\2017.4.3f1\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten_Win\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 554, in get
    38.    raise self._value
    39. AssertionError: Error in optimizer (return code -1073740791):
    40. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
    41.  
     
  7. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Does the build succeed when OpenCVForUnity and MarkerLessARExample are not imported into the project?
     
  8. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Just tested here. Without MarkerLessARExample, it builds.
    With OpenCVForUnity it builds.

    Reimport and set the plugin after seems to have solved. Now it crashes when it recognizes a marker. I think it has to do with memory allocation but I need to test more on that.
     
    ina likes this.
  9. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Web cam is not working after authorized (in any examples) using firefox and locally hosted builds. It works on edge. In firefox the camera seems to turn off after 5 ot 8 seconds.
     
    ina likes this.
  10. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Code (CSharp):
    1. An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was:
    2. uncaught exception: abort(107) at jsStackTrace (public.asm.framework.unityweb:2:27460)
    3. stackTrace (public.asm.framework.unityweb:2:27631)
    4. abort (public.asm.framework.unityweb:4:88242)
    5. VUt (public.asm.code.unityweb:52:1)
    6. EMn (public.asm.code.unityweb:7:1)
    7. lrg (public.asm.code.unityweb:6:1)
    8. WIt (public.asm.code.unityweb:52:1)
    9. invoke_iiiii (public.asm.framework.unityweb:2:378724)
    10. T1e (public.asm.code.unityweb:16:1)
    11. J1e (public.asm.code.unityweb:16:1)
    12. F1e (public.asm.code.unityweb:16:1)
    13. FHq (public.asm.code.unityweb:10:1)
    14. jnq (public.asm.code.unityweb:11:1)
    15. Xmq (public.asm.code.unityweb:11:1)
    16. m_s (public.asm.code.unityweb:20:1)
    17. l_s (public.asm.code.unityweb:20:1)
    18. iuq (public.asm.code.unityweb:11:1)
    19. Epq (public.asm.code.unityweb:11:1)
    20. spq (public.asm.code.unityweb:11:1)
    21. sHt (public.asm.code.unityweb:52:1)
    22. browserIterationFunc (public.asm.framework.unityweb:2:223392)
    23. runIter (public.asm.framework.unityweb:2:205453)
    24. Browser_mainLoop_runner (public.asm.framework.unityweb:2:224929)
    25.  
    26. If this abort() is unexpected, build with -s ASSERTIONS=1 which can give more information.
     
  11. OSagioma

    OSagioma

    Joined:
    May 12, 2015
    Posts:
    13
    Increasing the memory in the player settings to 512 made it work in edge.
     
  12. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    This problem seems to occur with Unity version 2017.4.3f1 and Firefox version 60.0.1.
    The web camera starts successfully with Unity version 2018.1.1f1 and Firefox version 60.0.1.
     
    OSagioma likes this.
  13. Masegi

    Masegi

    Joined:
    May 24, 2014
    Posts:
    17
    Hi Enox,

    I bought your asset a few days ago, I switched from NHW Coding to yours, but I can't build for iOS with your version when I try to embed the opencv framework. Sure, I read the ReadMe.pdf, and I added the Run Script but I still get the same errors:

    Fullscreen
     
  14. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Thank you very much for reporting.
    Could you tell me about your test environment?
    Unity version :
    OpenCVForUnity version :
    Xcode version :
     
  15. Masegi

    Masegi

    Joined:
    May 24, 2014
    Posts:
    17
    Unity: 2017.4.0f1
    OpenCVForUnity: 2.2.8
    Xcode: I'm not sure what I had, cause I just tried to update to fix the problem (didnt worked out), now it's: 9.2
     
    Last edited: May 24, 2018
  16. mejor_purchase

    mejor_purchase

    Joined:
    May 11, 2018
    Posts:
    3
    Hello!
    I am using the WebCamTextureMarkerBasedARExample scene in the MarkerBasedAR Example. I just changed the Marker design of MarkerSettings1 in the hierarchy to be the same as my marker (created with Adobe Illustrator)
    marker.png



    Also, I make sure the "MarkerSettings1" is in the MarkerSettings array of the WebCamTextureMarkerBasedARExample script.



    I printed the original example marker and my marker on a paper. When I run the scene, it can detect the original one but cannot detect mine. I also tried several different 5x5 and 3x3 markers (all made in Adobe Illustrator) but none of them can be detectd.
    I was wondering whether I've missed any other settings? Or ONLY markers created with "ArUcoCreateMarkerExample" can be detected?
    Hope someone can shed some light on this! Thank you!
     
  17. Masegi

    Masegi

    Joined:
    May 24, 2014
    Posts:
    17
    This is just bullshit, I want a Refund, can't wait longer - I also wrote used your ticket system but u keep on ignoring?
     
  18. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Sorry for my late reply.
    I tested it in the same environment as your environment, but the build is complete without problems.
    Have you added a script at the end of the build phase?
    run_script.png

    Also, could you try another way?
    Please execute the following command on terminal. https://stackoverflow.com/questions...portedarchitecture-x86-64-i386-in-ipa-archive

    //remove i386 architectures.
    lipo -remove i386 opencv2.framework/opencv2 -o opencv2.framework/opencv2
    //remove x86_64 architectures.
    lipo -remove x86_64 opencv2.framework/opencv2 -o opencv2.framework/opencv2
    //check the architectures.
    lipo -info opencv2.framework/opencv2


    ticket system? Have you sent a message from this page?
    https://enoxsoftware.com/opencvforunity/contact/technical-inquiry/
     
  19. Masegi

    Masegi

    Joined:
    May 24, 2014
    Posts:
    17
    yes this technical inquiry ticket system - still no answer on my refund request - it's just not working as intended, yes I tried both ways.
     
  20. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    I'm sorry, I overlooked your technical inquiry. I just sent a message about the refund to your email address.
     
  21. mejor_purchase

    mejor_purchase

    Joined:
    May 11, 2018
    Posts:
    3
    The deadline of the project is very close. :( @EnoxSoftware I would really appreciate if you would help solve the problem I met. Thank you!
     
  22. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    It seems that white and black of the marker settings are reversed.

    markersettings.PNG
     
  23. mejor_purchase

    mejor_purchase

    Joined:
    May 11, 2018
    Posts:
    3
    You're right!! Embarrassed:oops: Thanks a lot!!!
     
  24. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    Has anyone tried interactively training the DNN's? Is this possible with Unity's StreamingAsset infrastructure (write to these dat files?)
     
  25. bastianmeyerbm3

    bastianmeyerbm3

    Joined:
    May 22, 2018
    Posts:
    15
    Hi, how can i replace the Lasers etc in the "FaceTrackerExample"? I want to import my own "effects" and use them but i dont really know how i could do this
     
  26. Totoro205

    Totoro205

    Joined:
    Dec 12, 2017
    Posts:
    18
    Hello, I'm working on a game where i detect real-life objects from the webcam on every frame, find their edges, draw them and then project them on top of the real objects. I was wondering what function are available that allow an exact camera-projector calibration, how can i find the real world coordinates to place both my camera and projector? I am fairly new to this field so i would appreciate your help.
     
  27. Manlok

    Manlok

    Joined:
    Nov 17, 2012
    Posts:
    10
    Hi, I would like to buy the plugin in asset store. So I just download the example in Google Play Store. I want to try the apps before I buy. I only use the yolo objectdetection function. I don't know why the fps is too slow (drop to the 3 fps) when I pressed the YoloObjectDetectionWebCamTextureExample.
     
  28. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    You can change to the gameObject you want to display.
    facetracker_ar_1.PNG
    facetracker_ar_2.PNG
     
  29. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Unfortunately, I am not familiar with that field.
    You can check the functions available on OpenCVForUnity on this page.
    http://enoxsoftware.github.io/OpenCVForUnity/3.0.0/doc/html/index.html
     
  30. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    The performance of the Dnn module does not seem to be high on the mobile device(Especially on Android).
     
  31. LAFI

    LAFI

    Joined:
    Sep 5, 2014
    Posts:
    47
    Hello
    i'm usig unity for Linux editor on android platform, when i test the YoloObjectDetectionWebCamTextureExample i get those error

    Code (CSharp):
    1. Plugins: Couldn't open Assets/OpenCVForUnity/Plugins/x86_64/libopencvforunity.so, error: libopencv_aruco.so.3.4: cannot open shared object file: No such file or directory
    2. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample:Run() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:166)
    3. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample:Run() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:135)
    4. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample:Start() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:98)
    5.  
    6.  
    7.  
    8. Plugins: Couldn't open Assets/OpenCVForUnity/Plugins/x86_64/libopencvforunity.so, error: libopencv_aruco.so.3.4: cannot open shared object file: No such file or directory
    9. OpenCVForUnity.Utils:setDebugMode(Boolean, Boolean) (at Assets/OpenCVForUnity/org/opencv/unity/Utils.cs:1116)
    10. OpenCVForUnity.Utils:setDebugMode(Boolean, Boolean) (at Assets/OpenCVForUnity/org/opencv/unity/Utils.cs:1106)
    11. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample:Run() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:133)
    12. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample:Start() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:98)
    13.  
    14.  
    15.  
    16.  
    17. DllNotFoundException: opencvforunity
    18. OpenCVForUnity.Utils.setDebugMode (Boolean debugMode, Boolean throwException) (at Assets/OpenCVForUnity/org/opencv/unity/Utils.cs:1104)
    19. OpenCVForUnityExample.YoloObjectDetectionWebCamTextureExample.OnDestroy () (at Assets/OpenCVForUnity/Examples/MainModules/dnn/YoloExample/YoloObjectDetectionWebCamTextureExample.cs:350)
    20.  
     
  32. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    It is probably an error caused by lack of dependency library file of libopencvforunity.so.
    Have you completed the ”Linux Setup Procedure” of ReadMe.pdf?
     
  33. cortinadev

    cortinadev

    Joined:
    Dec 11, 2012
    Posts:
    4
    I'm attempting to convert a 24 bit rgb image to a 32 bit RFloat. I keep getting a fully black texture as a result. Any suggestions?

    Code (CSharp):
    1. Texture2D rgbDepth = new Texture2D(croppedWidth, croppedHeight, TextureFormat.RGB24, false);
    2. rgbDepth.LoadRawTextureData(bytes);
    3. rgbDepth.Apply();
    4.  
    5. Mat mat = new Mat(rgbDepth.height, rgbDepth.width, CvType.CV_8UC3);
    6. Utils.texture2DToMat(rgbDepth, mat);
    7.  
    8. List<Mat> channels = new List<Mat>();
    9. Core.split(mat, channels);
    10. Mat red = channels[0];                          
    11. Imgproc.resize(red, red, new Size(512, 512));
    12.  
    13. Mat rFloat = new Mat(red.height(), red.width(), CvType.CV_32FC1);
    14. red.convertTo(rFloat, CvType.CV_32F, 1/255);
    15.                    
    16. Texture2D output = new Texture2D(rFloat.width(), rFloat.height(), TextureFormat.RFloat, false);
    17. Utils.matToTexture2D(rFloat, output);
     
  34. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    The Utils.matToTexture2D() method has limitations on Mat's CVType.
    The Mat object has to be of the types 'CV_8UC4' (RGBA) , 'CV_8UC3' (RGB) or 'CV_8UC1' (GRAY)
    Utils.fastMatToTexture2D () simply copies Mat's data to Texture using Texture2D.LoadRawTextureData().
    Code (CSharp):
    1. Texture2D rgbDepth = new Texture2D(croppedWidth, croppedHeight, TextureFormat.RGB24, false);
    2. rgbDepth.LoadRawTextureData(bytes);
    3. rgbDepth.Apply();
    4. Mat mat = new Mat(rgbDepth.height, rgbDepth.width, CvType.CV_8UC3);
    5. Utils.texture2DToMat(rgbDepth, mat);
    6. List<Mat> channels = new List<Mat>();
    7. Core.split(mat, channels);
    8. Mat red = channels[0];                      
    9. Imgproc.resize(red, red, new Size(512, 512));
    10. Mat rFloat = new Mat(red.height(), red.width(), CvType.CV_32FC1);
    11. //red.convertTo(rFloat, CvType.CV_32F, 1/255);
    12. red.convertTo(rFloat, CvType.CV_32F, 1.0/255.0);
    13.                
    14. Texture2D output = new Texture2D(rFloat.width(), rFloat.height(), TextureFormat.RFloat, false);
    15. //Utils.matToTexture2D(rFloat, output);
    16. Utils.fastMatToTexture2D (rFloat, output);
     
    Last edited: Jun 8, 2018
  35. charlee-qq

    charlee-qq

    Joined:
    Oct 2, 2012
    Posts:
    4
    Hello may i ask you something, i use ArUco for detected maker's position and calculate a distance between these markers, which if the marker has a same plane (Rotation 0,0,0) the result about 162 pixels, meanwhile i rotate a marker the distance is changed, Why?? , and how to calculate the distance if the marker change a position and rotation.
    please help, Thank you
     

    Attached Files:

  36. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    The calculation result of the distance between two markers is not wrong.
    The ArUco class uses the specified 2D image Mat and the camera matrix to estimate the position of the marker.Since the position of the marker is estimated from the 2D image, it is not always possible to estimate the exact depth.
     
  37. TaanielDS

    TaanielDS

    Joined:
    Aug 16, 2017
    Posts:
    1
    I would like to use Unity's Cloud build for iOS builds. Is there a workaround to avoid getting "Failed to verify bitcode" error?
     
    ina likes this.
  38. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Please see Q&A of ReadMe.pdf.
    I have not yet found a way to automatically solve this problem.
     
  39. citizen_12

    citizen_12

    Joined:
    Jun 21, 2013
    Posts:
    33
    I just got working with OpenCV Unity and am wondering if there will be Tesseract support? It's far better than the one currently included in OpenCV Unity.

    Thanks!
     
  40. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    For now, I have no plans to implement Tesseract.
     
  41. rickyYHY

    rickyYHY

    Joined:
    May 1, 2018
    Posts:
    1
    hi
    I'm trying to use the core.kmeans function to doing the clustering.
    But it seems not working.
    Please help~


    Code (CSharp):
    1.  // checking if the image can be find
    2. Mat src = Imgcodecs.imread(pca_test1_jpg_filepath);
    3. #if !UNITY_WSA_10_0
    4. if (src.empty())
    5. {
    6. Debug.LogError("pca_test1.jpg is not loaded. Please copy from “OpenCVForUnity/StreamingAssets/” to “Assets/StreamingAssets/” folder. ");
    7. }
    8. #endif
    9.  
    10. Debug.Log("src.ToString() " + src.ToString());
    11.  
    12. Mat bestLable = new Mat();
    13. TermCriteria termCriteria = new TermCriteria(TermCriteria.EPS | TermCriteria.COUNT, 10, 1);
    14.  
    15.  // k means
    16. double d = Core.kmeans(src, 3, bestLable, termCriteria, 3, 2);
    17.  
    18. Imgproc.cvtColor(bestLable, bestLable, Imgproc.COLOR_BGR2RGB);
    19. Texture2D texture = new Texture2D(bestLable.cols(), bestLable.rows(), TextureFormat.RGBA32, false);
    20. Utils.matToTexture2D(bestLable, texture);
    21. gameObject.GetComponent<Renderer>().material.mainTexture = texture;
     
    Last edited: Jun 19, 2018
  42. TrixiStudios

    TrixiStudios

    Joined:
    Sep 8, 2016
    Posts:
    1
    Hi- I've downloaded the following assets up still can't get OpenPose example to work- any suggestions?

    https://github.com/CMU-Perceptual-C.../examples/media/COCO_val2014_000000000589.jpg
    "http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/mpi/pose_iter_160000.caffemodel",
    "https://github.com/opencv/opencv_ex...nn/openpose_pose_mpi_faster_4_stages.prototxt".


    upload_2018-6-20_0-32-10.png



    model file is not loaded.The model and prototxt file can be downloaded here: "http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/mpi/pose_iter_160000.caffemodel","https://github.com/opencv/opencv_ex...nn/openpose_pose_mpi_faster_4_stages.prototxt".Please copy to “Assets/StreamingAssets/dnn/” folder.
    UnityEngine.Debug:LogError(Object)
    OpenCVForUnityExample.OpenPoseExample:Run() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/CaffeExample/OpenPoseExample.cs:202)
    OpenCVForUnityExample.OpenPoseExample:Start() (at Assets/OpenCVForUnity/Examples/MainModules/dnn/CaffeExample/OpenPoseExample.cs:135)
     
  43. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    I think that this page will be helpful.
    https://qiita.com/ud0nge/items/9c1045912dca0837aeb4
     
  44. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Could you check the file size?
    pose_iter_160000.caffemodel 205,950,363byte
    openpose_pose_mpi_faster_4_stages.prototxt 32,813byte
     
  45. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    858
    Hi Enox, and thank you so much for porting OpenCV!

    I am having trouble reading a depth texture from a RealSense camera (raw bytes) into a Mat. I'm getting the error:
    "Mat data type is not compatible: 2".

    The RealSense image pixels are single channel unsigned 16-bit integers, so I've tried with CvType.CV_16U and CvType.CV_16UC1. Both throw the error above. Reading the same data into a Texture2D with TextureFormat.R16 using LoadRawTextureData works fine.

    How can I read single channel 16-bit unsigned integer raw bytes into a Mat?

    EDIT: I am using mat.put to pass in the raw byte array. I found that using CV_8UC1 and CV_8UC2 is allowed but the values are wrong.

    EDIT2: I see that in Mat.cs line 3055 the put() method is only accepting depth types CV_8U and CV_8S. What is the reason for this? And how can I read a 16 bit image?

    All the best
    Carl Emil
     
    Last edited: Jun 21, 2018
  46. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    Since this asset is a clone of OpenCV Java, you are able to use the same API as OpenCV Java.http://enoxsoftware.github.io/OpenCVForUnity/3.0.0/doc/html/index.html
    The put() method of Mat has CvType restrictions. So, I recommend using the Utils.copyToMat () method instead of the Mat.put () method.
    Code (CSharp):
    1.             Mat mat = new Mat (512, 512, CvType.CV_16UC1);
    2.  
    3.             ushort[] ushortData = new ushort[mat.width () * mat.height ()];
    4.             for (int i = 0; i < ushortData.Length; i++) {
    5.                 ushortData [i] = 65535;
    6.             }
    7.  
    8. //            Utils.copyToMat<ushort> (ushortData, mat);
    9.  
    10.             byte[] byteData = new byte[mat.width () * mat.height () * 2];
    11.             for (int i = 0; i < byteData.Length / 2; i++) {
    12.                 byteData [i * 2 + 0] = BitConverter.GetBytes (ushortData [i]) [0];
    13.                 byteData [i * 2 + 1] = BitConverter.GetBytes (ushortData [i]) [1];
    14.             }
    15.             Utils.copyToMat<byte> (byteData, mat);
    16.  
    17.  
    18.             Texture2D texture = new Texture2D (mat.cols (), mat.rows (), TextureFormat.R16, false);
    19.          
    20.             Utils.fastMatToTexture2D (mat, texture);
    21.  
    22.             gameObject.GetComponent<Renderer> ().material.mainTexture = texture;
     
  47. cecarlsen

    cecarlsen

    Joined:
    Jun 30, 2006
    Posts:
    858
    Amazing. Thank you! I'm on a roll again.

    Two things:

    1) It seems that the OpenCvForUnity folder needs to be placed in the root of the asset folder for the "Set Plugin Import Settings" and possibly other things to work. This is a annoyance because I organise all my third party assets in 'Assets/ThirdParty'. Hereby a feature request; flexible location of the main folder.

    2) I noticed that 'OpenCvForUnity.dll' was not automatically copied to the build. I had to manually place it in the 'title_Data/Plugins' folder of the player. Is this an expected behaviour? Unity 2018.2.0b6, OpenCvForUnity 2.2.9.

    All the best
    Carl Emil
     
  48. EnoxSoftware

    EnoxSoftware

    Joined:
    Oct 29, 2014
    Posts:
    1,557
    1) Thank you for your proposal. It may be fixed in future version upgrade. For now, after calling "Set Plugin Import Settings", move the "OpenCVForUnity" folder to an arbitrary place.
    2) Is ImportSettings set like this?
    win_x86_64_importsettings.PNG
    win_x86_importsettings.PNG
     
  49. HeavyArmoredMan

    HeavyArmoredMan

    Joined:
    Apr 9, 2017
    Posts:
    8
    Hi EnoxSoftware,

    It seems the Utils.texture2DToMat() and Utils.fastTexture2DToMat() could generate huge amounts of garbage.

    To reproduce this issue, move the codes from Start() to Update() in Texture2DToMatExample.cs. Run this example scene and monitor every frame update.

    See this screenshot below:
    20180624141800.png


    [UPDATE]
    Now I have figured out what produces garbage is the texture2D.GetRawTextureData () call in the texture2DToMat() and fastTexture2DToMat() methods. Any workaround to avoid that?
     
    Last edited: Jun 24, 2018
    ina likes this.
  50. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    Actually, has this been resolved yet