Building OpenCV plugin for Mac Osx

I have seen lot of interest amongst developer community who wants to use openCV with Unity. There has been very limited resources on this. This article is all about -
How to use OpenCV functions in Unity Editor (on MAC OS) using native Plugin without using 3rd party plugin from assets store like openCVforUnity etc.

Prerequisite – Xcode and CMake are installed on your machine. These are trivial installation.

Xcode can be installed from App store
CMake - follow steps provided at this site.​

Build and Install OpenCV

Download openCV and unzip it somewhere on your computer. (I hace tried it for 3.4.5) Create build folder inside it.
Open CMake application
Click Browse Source and navigate to your openCV folder.
Click Browse Build and navigate to your build Folder.
Click the configure button. You will be asked how you would like to generate the files.
Choose Unix-Makefile from the Drop Down menu and Click OK.
CMake will perform some tests and return a set of red boxes appear in the CMake Window.
Click Configure again, then Click Generate.
Go to build folder

Code (JavaScript):

1. # cd build
2. # make
3. # sudo make install

This will install the opencv libraries on your computer.
​

Create Plug-In using Xcode

Now we will create a native Plugin for Mac to be used in Unity as Plugin.
Please start Xcode
Choose MacOS> Framework & Library> Bundle. "Product Name" is set to "UnityPlugin".
Use packagae “com.hmi”
Change bundle identifier to “com.hmi.opencvplugin”
Select UnityPlugin in the navigation view

Go to "Build Settings" → "Header Search Path" field (enter the details without collapsing the “Header Search Paths”) and enter "/usr/local/include" (the opencv header files are stored here as part of build that we performed in earlier step)

Go to "Build Settings" → "Library Search Path" field (enter the details without collapsing the “Header Search Paths”) and enter "/usr/local/lib" (the opencv *opencv*.dyib files are stored here as part of build that we performed in earlier step)

Go to "Build Settings" → "Other Linker Flags" field, enter “-lopencv_dnn -lopencv_ml -lopencv_objdetect -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_features2d -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_video -lopencv_photo -lopencv_imgproc -lopencv_flann -lopencv_core”. These are all the linked libraries of opencv. Validate this with following command on terminal window

Code (CSharp):

1. # pkg-config --libs --cflags opencv

Next, create a new cpp file (OpenCVPlugin.cpp). Deslect “Also create header file” for use as a Native Plugin.
Now create a sample opencv function as shown below

Code (CSharp):

1.     #include <stdio.h>
2.     #include <string>
3.    
4.     #include <opencv2/opencv.hpp>
5.     #include <opencv2/core.hpp>
6.     #include <opencv2/imgproc.hpp>
7.    
8.     using namespace cv;
9.    
10.     extern "C" {
11.        int TestFunction_Internal();
12.     }
13.    
14.     int TestFunction_Internal() {
15.         return 12345;
16.     }

Note - For simplicity I have just added TestFunction here to show how PlugIn functions. Please feel free to add your opencv image operation features as per your need. I can share more details on how to write these functions in my separate thread.
Select "Product" → "Build" from the menu bar and build the project, UnityPlugin.bundle will be created in the Products folder.

Installing plugin in Unity

If you have reached this stage, Congratulations !!, ..you are now few minutes away from your final test.
Please create a demo project in Unity.
Go to your project window and create a "Plugins" folder under the Asset.
Create Osx folder under assets → Plugins folder. (you will have different OS types here.)
Insert "UnityPlugin.bundle" that you had created in earlier steps into Osx folder. You can simply drag and drop directly from Xcode's Products folder to Unity. ( or go to Xcode editor and right click OpenCVPlugin.bundle and select show in folder. From there you can drag it into Unity)

Create Scripts folder under assets.
Under this Scripts folder you can create Debug and Native folders respectively. (Assets → Scripts → Debug and Assets → Scripts → Native)
Now we create new c# controller.cs script in Debug folder. This script will consist of passing info back and forth between managed C# and native C++ code.

Code (CSharp):

1.     using UnityEngine;
2.     using UnityEngine.UI;
3.    
4.     using System.Runtime.InteropServices;
5.     using System;
6.    
7.     /*
8.         Controller provides examples for passing pointers for byte arrays back and forth
9.         between managed C# and native C++. When making your own classes, be sure to specify
10.         the Texture2D texture format before attempting to load raw bytes into it.
11.     */
12.     public class Controller : MonoBehaviour
13.     {
14.         void Awake()
15.         {
16.         Debug.LogWarning("Test Value in C++ is " + NativeLibAdapter.Test());
17.         }
18.    
19.         void Update()
20.         {
21.         }
22.     }

You can see that it calls Test function from Native library. Now we create new C# script NativeLibAdapter.cs in Assets → Scripts → Native folder

Code (CSharp):

1.     using System.Runtime.InteropServices;
2.     using System;
3.    
4.     using UnityEngine;
5.    
6.     /*
7.         NativeLibAdapter is an example communication layer between managed C# and native C++
8.     */
9.     public class NativeLibAdapter
10.     {
11.     #if !UNITY_EDITOR
12.     /*
13.         [DllImport("native-lib")]
14.         private static extern int TestFunction_Internal();
15.     */
16.     #elif UNITY_EDITOR
17.         [DllImport("OpenCVPlugin")]
18.         private static extern int TestFunction_Internal();
19.     #endif
20.    
21.    
22.         public static int Test()
23.         {
24.     #if !UNITY_EDITOR
25.             return TestFunction_Internal();
26.     #elif UNITY_EDITOR
27.             return TestFunction_Internal();
28.     #else
29.             return -1;
30.     #endif
31.         }
32.     }

You will see that the script now has fine control on which library to link based on where you are running your player. (Unity Editor or Android i.e. Non Unity editor). Un-comment the code if you have build the native-lib for non Unity editor like Android.
Create a cube 3D object under your scene and drop controller.cs script on to it.
Now if you run this project in Unity editor, you should see in your console "Test Value in C++ is 12345"

Please let me know if you are able to follow these steps. If not, I can edit this thread based on where you stumble upon.

Happy coding !!​

 

In case you are looking for complete code , here is a link to git repo.

 

Thanks for the help! When I try to follow these steps with opencv4.1.2,

Code (CSharp):

1. pkg-config --libs --cflags opencv

tells me:

Code (CSharp):

1. Package opencv was not found in the pkg-config search path.
2. Perhaps you should add the directory containing `opencv.pc'
3. to the PKG_CONFIG_PATH environment variable
4. No package 'opencv' found

Did you also find a way to use opencv4 in xcode/unity?

 

Found the issue. For the Header Search Paths key, use

/usr/local/include/opencv4

instead of the suggested

/usr/local/include

.