Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Building OpenCV plugin for Mac Osx

Discussion in 'General Discussion' started by darshanpv, Feb 3, 2019.

  1. darshanpv


    Feb 1, 2019
    I have seen lot of interest amongst developer community who wants to use openCV with Unity. There has been very limited resources on this. This article is all about -
    How to use OpenCV functions in Unity Editor (on MAC OS) using native Plugin without using 3rd party plugin from assets store like openCVforUnity etc.

    Prerequisite – Xcode and CMake are installed on your machine. These are trivial installation.
    Xcode can be installed from App store
    CMake - follow steps provided at this site.​

    Build and Install OpenCV
    Download openCV and unzip it somewhere on your computer. (I hace tried it for 3.4.5) Create build folder inside it.
    Open CMake application
    Click Browse Source and navigate to your openCV folder.
    Click Browse Build and navigate to your build Folder.
    Click the configure button. You will be asked how you would like to generate the files.
    Choose Unix-Makefile from the Drop Down menu and Click OK.
    CMake will perform some tests and return a set of red boxes appear in the CMake Window.
    Click Configure again, then Click Generate.
    Go to build folder
    Code (JavaScript):
    1. # cd build
    2. # make
    3. # sudo make install
    This will install the opencv libraries on your computer.
    Create Plug-In using Xcode
    Now we will create a native Plugin for Mac to be used in Unity as Plugin.
    Please start Xcode
    Choose MacOS> Framework & Library> Bundle. "Product Name" is set to "UnityPlugin".
    Use packagae “com.hmi”
    Change bundle identifier to “com.hmi.opencvplugin”
    Select UnityPlugin in the navigation view

    Go to "Build Settings" → "Header Search Path" field (enter the details without collapsing the “Header Search Paths”) and enter "/usr/local/include" (the opencv header files are stored here as part of build that we performed in earlier step)

    Go to "Build Settings" → "Library Search Path" field (enter the details without collapsing the “Header Search Paths”) and enter "/usr/local/lib" (the opencv *opencv*.dyib files are stored here as part of build that we performed in earlier step)

    Go to "Build Settings" → "Other Linker Flags" field, enter “-lopencv_dnn -lopencv_ml -lopencv_objdetect -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_features2d -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_video -lopencv_photo -lopencv_imgproc -lopencv_flann -lopencv_core”. These are all the linked libraries of opencv. Validate this with following command on terminal window

    Code (CSharp):
    1. # pkg-config --libs --cflags opencv

    Next, create a new cpp file (OpenCVPlugin.cpp). Deslect “Also create header file” for use as a Native Plugin.
    Now create a sample opencv function as shown below
    Code (CSharp):
    1.     #include <stdio.h>
    2.     #include <string>
    4.     #include <opencv2/opencv.hpp>
    5.     #include <opencv2/core.hpp>
    6.     #include <opencv2/imgproc.hpp>
    8.     using namespace cv;
    10.     extern "C" {
    11.        int TestFunction_Internal();
    12.     }
    14.     int TestFunction_Internal() {
    15.         return 12345;
    16.     }
    Note - For simplicity I have just added TestFunction here to show how PlugIn functions. Please feel free to add your opencv image operation features as per your need. I can share more details on how to write these functions in my separate thread.
    Select "Product" → "Build" from the menu bar and build the project, UnityPlugin.bundle will be created in the Products folder.

    Installing plugin in Unity

    If you have reached this stage, Congratulations !!, are now few minutes away from your final test.
    Please create a demo project in Unity.
    Go to your project window and create a "Plugins" folder under the Asset.
    Create Osx folder under assets → Plugins folder. (you will have different OS types here.)
    Insert "UnityPlugin.bundle" that you had created in earlier steps into Osx folder. You can simply drag and drop directly from Xcode's Products folder to Unity. ( or go to Xcode editor and right click OpenCVPlugin.bundle and select show in folder. From there you can drag it into Unity)

    Create Scripts folder under assets.
    Under this Scripts folder you can create Debug and Native folders respectively. (Assets → Scripts → Debug and Assets → Scripts → Native)
    Now we create new c# controller.cs script in Debug folder. This script will consist of passing info back and forth between managed C# and native C++ code.
    Code (CSharp):
    1.     using UnityEngine;
    2.     using UnityEngine.UI;
    4.     using System.Runtime.InteropServices;
    5.     using System;
    7.     /*
    8.         Controller provides examples for passing pointers for byte arrays back and forth
    9.         between managed C# and native C++. When making your own classes, be sure to specify
    10.         the Texture2D texture format before attempting to load raw bytes into it.
    11.     */
    12.     public class Controller : MonoBehaviour
    13.     {
    14.         void Awake()
    15.         {
    16.         Debug.LogWarning("Test Value in C++ is " + NativeLibAdapter.Test());
    17.         }
    19.         void Update()
    20.         {
    21.         }
    22.     }
    You can see that it calls Test function from Native library. Now we create new C# script NativeLibAdapter.cs in Assets → Scripts → Native folder

    Code (CSharp):
    1.     using System.Runtime.InteropServices;
    2.     using System;
    4.     using UnityEngine;
    6.     /*
    7.         NativeLibAdapter is an example communication layer between managed C# and native C++
    8.     */
    9.     public class NativeLibAdapter
    10.     {
    11.     #if !UNITY_EDITOR
    12.     /*
    13.         [DllImport("native-lib")]
    14.         private static extern int TestFunction_Internal();
    15.     */
    16.     #elif UNITY_EDITOR
    17.         [DllImport("OpenCVPlugin")]
    18.         private static extern int TestFunction_Internal();
    19.     #endif
    22.         public static int Test()
    23.         {
    24.     #if !UNITY_EDITOR
    25.             return TestFunction_Internal();
    26.     #elif UNITY_EDITOR
    27.             return TestFunction_Internal();
    28.     #else
    29.             return -1;
    30.     #endif
    31.         }
    32.     }
    You will see that the script now has fine control on which library to link based on where you are running your player. (Unity Editor or Android i.e. Non Unity editor). Un-comment the code if you have build the native-lib for non Unity editor like Android.
    Create a cube 3D object under your scene and drop controller.cs script on to it.
    Now if you run this project in Unity editor, you should see in your console "Test Value in C++ is 12345"

    Please let me know if you are able to follow these steps. If not, I can edit this thread based on where you stumble upon.

    Happy coding !!​
    josephphilleo likes this.
  2. darshanpv


    Feb 1, 2019
    In case you are looking for complete code , here is a link to git repo.
  3. LucciVanSchandt


    Nov 6, 2019
    Thanks for the help! When I try to follow these steps with opencv4.1.2,
    Code (CSharp):
    1. pkg-config --libs --cflags opencv
    tells me:
    Code (CSharp):
    1. Package opencv was not found in the pkg-config search path.
    2. Perhaps you should add the directory containing `opencv.pc'
    3. to the PKG_CONFIG_PATH environment variable
    4. No package 'opencv' found
    Did you also find a way to use opencv4 in xcode/unity?
    josephphilleo likes this.
  4. josephphilleo


    Oct 29, 2019
    Found the issue. For the Header Search Paths key, use
    instead of the suggested