Video Input as Texture, then Interpreting Change

So now that I've determined that I can (in theory) port the Myron webcam computer vision system ( http://webcamxtra.sourceforge.net/ ) into unity, and have some Director functions that help me know how to call the Unity plugin I will (hopefully) create, how do I know that Unity will be able to handle the incoming data?

For instance, using the Director WebCam Xtra (based on Myron), I can find change in an image and draw globs based on the video like so:

Code (csharp):

1.  
2. //NOTE EVERYONE, THIS IS JAVASCRIPT LINGO, NOT UNITY JAVASCRIPT
3.  
4. function drawGlobPixels() {
5.   //draw pixels for each glob
6.   if (gGlobPixels == 1) {
7.      var gDrawColorRGB = color(255, 255, 204)
8.      var gpix = webcamxtra.globPixels();
9.      for (var i = 1; i <= gpix.count; i++) {
10.           var tg = gpix[i];
11.           for (var ii = 1; ii <= tg.count; ii++) {
12.               _movie.stage.image.setPixel(tg[ii],gDrawColorRGB);
13.       }
14.     }
15.   }
16. }

And then using the glob position, I can have a 3D object react to the position of the glob.

In Unity, can I take that incoming data, and create an XYZ point in space that can interact with the other objects in my gamespace?

 

Cool!
The first thing I have to do is figure out how to compile it. I have created an XCode Project and have followed the Unity tutorials for creating a plugin. I think I have everything set up correctly, but (not knowing the source code) when I try to build I get one error "storage class specified for field 'start'". (Screen grab attached).

Not being a C++ person, is this an obvious or easy thing to correct?

 

I added the Extern "C" around the class. I had thought that adding the Extern "C" was a requirement I read in the Unity tutorials about creating a plugin.

 

Extern "C" tells the compiler that what you are defining is C code and not C++. C does not have classes, so you are in effect trying to compile C++ code as if it were plain C. This is confusing the compiler causing it to report weird errors.

 

...in both the cexterms.cpp and cexterns.h files?

 

Right, so I declared the function prototpyes in the header, but now I have to expose the functions for each and every one in the cexterns.cpp...

 

So I edited the cextern.h and the cexterns.cpp and I now get a build error. So I guess that's progress anyway.

The error is when it tries to link to the Unity Plugin.
It is saying there are undefined symbols:
_DisposeUserData _EnterMovies etc.

Any ideas? I will of course re-read about building the plugin in XCode in case I've just missed a step...

 

Yep, not added...

 

Yeah!
I added the QT framework and the build was successful. There were 5 warnings, but I can probably track them down...
Now comes the next step. How do I apply the video input comping through the plugin to a texture in Unity?

 

Okay, I got the plugin into the Assets folder and wrote this little C# script:

Code (csharp):

1. using UnityEngine;
2. using System.Runtime.InteropServices;
3.  
4. class MyronScript : MonoBehaviour
5. {
6.    // This tells unity to look up the function FooPluginFunction inside the plugin named "PluginName"
7.    [DllImport ("Myron Unity Plugin")]
8.    private static extern float myron_start ();
9.  
10.    void Awake ()
11.    {
12.       int height = 320;
13.       int width = 240;
14.       // Calls the FooPluginFunction inside the PluginName plugin
15.       myron_start(height,width);
16.    }
17. }

but I'm gettting this error from Unity:

 

I am using a Panasonic industrial video camera, going through a Firewire bridge, but I also tried it with an iSight and it worked fine. If I remember correctly, think I couldn't get it to go beyond 320*240 resolution.

 

Yeah, this sounds really cool.

 

Here's where I am. I got it to compile, and I tried to get it to launch from within unity with this code in a file called myron.cs :

Code (csharp):

1. using UnityEngine;
2. using System.Runtime.InteropServices;
3.  
4. class MyronScript : MonoBehaviour
5. {
6.    // This tells unity to look up the function FooPluginFunction inside the plugin named "PluginName"
7.    [DllImport ("Myron Unity Plugin")]
8.    private static extern float myron_start ();
9.  
10.    void Awake ()
11.    {
12.       int height = 320;
13.       int width = 240;
14.       // Calls the FooPluginFunction inside the PluginName plugin
15.       myron_start(height,width);
16.    }
17. }
18.  

but I get the error from the Unity console which is confusing:

I supplied two arguements in the call...

Here's from the actual myron.cpp code that initializes myron:

Code (csharp):

1.  
2. void Myron::start(int w,int h){
3.  
4.    
5.     UserData mySGVideoSettings = NULL;
6.     //-- begin user code                                                                                 
7.     ComponentDescription theDesc;                                                
8.     //ComponentResult theresult;                                                    
9.     Component sgCompID ;                                                          
10.     Rect videoRect;                                                              
11.     int error;                                                                                                  
12. #if defined(Macintosh)
13. ...
14. ...

and from the cexterns.cpp file:

Code (csharp):

1. extern "C" void myron_start(int w,int h){
2.     if(m==0){
3.         m = new Myron();
4.         m->start(w,h);
5.     } else {
6.         m->stop();
7.         delete m;
8.         m = new Myron();
9.         m->start(w,h);
10.     }
11. }

And the camera is on and connected....

 

I just noticed that I had inverted width and height...I'll change those but it should still launch one would think since it's just two integers...

 

Don't you need to declare the parameters in your DllImport declaration, like this:

Code (csharp):

1.  
2. [DllImport ("Myron Unity Plugin")]
3.    private static extern float myron_start (int w, int h);
4.  

 

Thanks! That took care of the error. Now on to seeing if I can start getting motion data from the plugin!

 

I appologize, I've not really done much in the way of C++, so I'm still having some probs because I'm trying to figure out someone elses code.
I'm getting a Sytem:EntryPointNotFoundException:myron_start error.

Going backwards down the chain, I call from the plugin (... represents that code continues):

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3. using System.Runtime.InteropServices;
4.  
5. public class MyronScript : MonoBehaviour
6. {
7.    [DllImport ("Myron Unity Plugin")]  
8.    static extern void myron_start(int w, int h);
9.    void Awake ()
10.    {
11.      myron_start(320, 240);
12.    }
13. }

And the plugin itself has this at the top of it's cexterns.h file...

Code (csharp):

1.  
2. extern void myron_start(int w,int h);
3. extern void myron_stop();
4. extern void myron_update();
5. extern void myron_settings();
6. extern unsigned char*myron_version();
7. extern void myron_findGlobs();
8. ...

and this at the top of its cexterns.cpp file...

Code (csharp):

1. #include "myron.h"
2.  
3. static Myron *m = 0;
4.  
5. extern "C" void myron_start(int w,int h){
6.     if(m==0){
7.         m = new Myron();
8.         m->start(w,h);
9.     } else {
10.         m->stop();
11.         delete m;
12.         m = new Myron();
13.         m->start(w,h);
14.     }
15. }
16. ...
17.  

and this at the start of the myron.h file...

Code (csharp):

1. class Myron{
2.  public:
3.   Myron();
4.   ~Myron();
5.   void start(int w,int h);
6.   void stop();
7. ...
8. }

And finally, this at the start of the myron.cpp file...

Code (csharp):

1.  
2. #include "myron.h"
3. #include <math.h>
4.  
5.  
6. Myron::Myron(){
7. #if defined(Macintosh)
8. #else
9.    InitializeQTML(0);
10. #endif
11.  
12. }
13.  
14. Myron::~Myron(){
15. #if defined(Macintosh)
16. #else
17.   TerminateQTML();
18. #endif
19.  
20. }
21.  
22.  
23. //-----------------------------------------------------------------------------
24.  
25. void Myron::start(int w,int h){
26. ...
27. }

Any pointers as to what I'm not declaring correctly?

 

I don't know what the problem is, but here's some extra information. For the exception being thrown, see http://msdn.microsoft.com/library/d...stementrypointnotfoundexceptionclasstopic.asp.

It seems to be looking for an entry method in an assembly, which would imply (I think) that it is not yet trying to invoke the external function. It's as if there should be a C# assembly containing myron_start (maybe generated automatically) that would subsequently invoke the actual external function. This is all guesswork on my part, however, as I don't know the internals of .NET.

Tom