c++ managed to native issue (RakNet)

I am putting this in external tools because I have been working very hard at making RakNet a simple drag and drop library to be used for managed code objects, like Unity. I have ran into a snag that made me set the project aside for a little while, but now that I am back on it, it finally dawned on me what my problem is, but I have zero clue how to fix this.

That is my error in a nut shell, notice something? That the CLASS is being treated as a STRUCT in the object browser, but in fact the classes are defined as classes, this was a test interface that I had started after making all of the classes public and compilable under the /clr switch. This example test class is defined as follows:

Code (csharp):

1.  
2. public class ManagedRakPeerInterface
3. {
4. public:
5.     RakPeerInterface* pRakPeerInterface;
6.  
7.     // Constructor
8.     ManagedRakPeerInterface()
9.     {
10.    
11.     }
12.     // Destructor
13.     virtual ~ManagedRakPeerInterface()
14.     {
15.         delete pRakPeerInterface;
16.     }
17. };
18.  

I am totally at a loss, utterly and completely at a loss, why is it treating my classes as structures? I think if I can crack this problem and get either the project compiling options fixed or get the definitions fixed, everything else will just fall into place.

If you have any clue how to point me in the right direction, please help.

 

Any chance to see the binding CS file

 

This is a CPP project, compiled to managed, there is no c# file associated with this. I guess I am misunderstanding what you need. I can get you a list of all of the project properties if you need to see that, I have the project under source control and is 75 megs in size in total. I am not using DLLIMPORT in Unity, this is strictly going to be usable via using RakNet.

This is my command line build options:

Code (csharp):

1.  
2. /Od /Ob2 /I "./../../Source" /I "./../../DependentExtensions/openssl-0.9.8g" /I "./../../DependentExtensions/udt4/src" /D "_CRT_SECURE_NO_DEPRECATE" /D "WIN32" /D "_DEBUG" /D "_RAKNET_DLL" /D "_CRT_NONSTDC_NO_DEPRECATE" /D "_VC80_UPGRADE=0x0710" /D "_WINDLL" /D "_MBCS" /FD /EHa /MDd /Fo"Debug\\" /Fd"Debug/../../RakNetDebug" /W4 /nologo /c /Wp64 /Zi /clr /Gd /TP /errorReport:prompt
3.  

I can get you anything you need, even a copy of the project if you want, it is done in VS 2005 C++, I have attached just one of the modified header files that have the problem, this class should be a class in object browser but instead is a struct in object browser.

To further elaborate on the ease of use of this thing:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using System;
5. using RakNet;
6.  
7. public class zumRakNet : MonoBehaviour {
8.  
9.     // Use this for initialization
10.     void Start () {
11.         RakNetworkFactory pFactory;
12.         pFactory = new RakNetworkFactory();
13.     }
14.    
15.     // Update is called once per frame
16.     void Update () {
17.    
18.     }
19. }
20.  

That is all the code necessary to initialize a new RakNetworkFactory interface, you do not need to use DLLIMPORT with defining interface connections. AND this can be used via the web which means you do not need to put the DLL in the PLUGINS folder, and it will work for web games.

 

I might be wrong on that as I'm no managed C++ dev (I never liked it as it is ugly and hell to work with and normally only used to create binding wrappers between C++ and .NET on the unmanaged instead of the managed side) but as far as I'm aware, you need to declare all managed classes as __gc class XY

Also you need to have #using <mscorlib.dll> as it forms the base of anything in .NET (just as much as you need using System; on the C# side)


Your code there from what I see is just plain C / C++, not managed C++ code
If the class isn't managed, it would make sense that it lists it as struct as it is unmanaged memory that you can't pass around as ref but only by value.

 

You are correct about the mscorlib, see attached header for the windows build. RakNet is a collection of classes and sub classes and the core is inherited based on which base you need for which build, this is the -only- place I need to include the mscorlib file, and compile with the /clr option, that was phase 1 which I did months ago. If I then attempt to include either later in any class or sub class I get duplicate decleration errors like I should get since they are already declared and being used.

One such web article that discusses the methods I use to utilize managed to native calls is:
http://ondotnet.com/pub/a/dotnet/2003/03/03/mcppp2.html

Simply by using pointers to the unmanaged by the managed, I can hook into the unmanaged as if it was managed without any need to wrap everything.

 

Ok, I just created a test project to see what on earth is going on, I started a new VS 2005 C++ Project of type CLR Empty Project (always start simple eh?)

Now, I changed the type from .exe to .dll (configuration, properties, general, configuration type: dynamic link library), made sure that it was runtime linbrary as multhreaded debug dll, so far so good, nothing hard yet.

I created 3 files, 2 headers and 1 cpp files as follows:

functions.h

Code (csharp):

1.  
2. #include "WindowsIncludes.h"
3. namespace zumwalt
4. {
5.     public ref class cls1
6.     {
7.     public:
8.         cls1(){};
9.         virtual ~cls1(){};
10.         int addSomething(int v1,int v2);
11.     };
12.  
13.     public class cls2
14.     {
15.     public:
16.         cls2(){};
17.         virtual ~cls2(){};
18.         int subtractSomething(int v1, int v2);
19.     };
20. }
21.  

mainstuff.cpp

Code (csharp):

1.  
2. #include "Functions.h";
3. namespace zumwalt
4. {
5. #pragma managed
6.     int cls1::addSomething(int v1,int v2)
7.     {
8.         return v1+v2;
9.     }
10.  
11. #pragma unmanaged
12.     int cls2::subtractSomething(int v1, int v2)
13.     {
14.         return v1-v2;
15.     }
16. }
17.  

windowsincludes.h

Code (csharp):

1.  
2. #include <WinSock2.h>
3. #include <windows.h>
4. #using <mscorlib.dll>
5. using namespace System;
6.  

Now I compiled it, cleanly, into a DLL, then I simply dragged that DLL into the assets folder of my new empty test project, the namespace of zumwalt I can add in my using list, and I can see cls1 as a class, and cls2 as a struct, but from cls1 I can call cls2 objects and get pointers to the results. So why it is not working from RakNet I am at a loss, I tried with and without the ref key word in my managed code block, and neither make a difference on the raknet managed to native calls.

 

So just to get the situation straight:

You create managed wrapper classes for what you intend to use, that have a new name and hook up pointers the way page 2 does, right?

 

Correct

 

I am begining to think this is not entirely possible with Unity in its current release -at all-, I have made a completely managed C++ dll that is clr compliant and every time I try to make a new instance of the primary class, Unity simply crashes. Even with the posted code here, if you try to make a new instance of cls1, Unity will simply crash.

I think Mono can't handle managed assemblies but only unmanaged with using the dllimport and at that, it is buggy. This might have to be put on hold until 3.0 comes out and keep my fingers crossed.

 

Managed C++ stuff is just .net and it should work without problems.
But when I remember right, even if the unmanaged code is within a managed C++ dll, is still unmanaged. So in an environment that does not allow unmanaged code it will not work (webplayer, Unity nonPro)

Thats naturally assuming you build towards 2.0, not 3.0+ and that you only use whats available.

 

The target framework is 2.0 with x86, and drag the managed DLL into the assets folder, then try to use one of the managed classes, and boom. Unity dies with hitting the play, not even compiling it to an actual windows build (as in Unity scene only has a simple C# script that makes an instance of the class calling a method in the class). I know all to well about not trying to build against 3.0

Only thing I have yet to do is pull out an older laptop that is collecting dust that has VS 2003 on it and try it within that, since that machine only has framework 1.1 on it. I really think all of the limitations I have ever ran across are due to the version of Mono Unity is currently running on, which is very archaic compared to what is current released in stable build. This was just another failed attempt of mine it appears. I push the limits of the engine all the time on different levels for different reasons.

I have even built a managed C++ library that is a flat API wrapper, was hoping to have it walk existing unmanaged libraries and build up a static flat managed API interface, of course in that instance it -will not ever work- for web builds since that one requires both managed and unmanaged dll's. Until I can get a solid combined managed C++ with unmanaged C++ in the same DLL with the exposure of only the managed static interfaces, this is not going to work.

The ultimate Idea behind all of this, is that I could simply supply new headers and switch instruction sets for RakNet so that people just take the header files, add them to their RakNet, compily with the /clr switch out of the box, and it will work without any source modifications. All my headers are separate files to Jenkins files, so they are all backward compatible, doesn't matter how many new or changed things he does, as long as the inputs and outputs are the same, the thing JWW (Just Will Work). I am concededing to the point that I do not know of anyone else who can help with the managed to unmanaged piece, so I have signed up for a crash course in managed assemblies using C++ which will start in 4 weeks and lasts for 8 weeks in total. So sometime in the next 12 weeks I should have the missing link, that or I would have wasted the dough.

Well, off to dinner, thanks again for all the help you have provided.

 

I'm pretty sure that even if you had it all in a single managed dll it would still fail in the webplayer if you tried to access the unmanaged code as the webplayer is likely setup to not allow any unsafe pointers at all, which is what your unmanaged C++ "binding" code will use.

So even with all the heavy lifting, all you get around is the use of 2+ dlls.


The idea you have is pretty nice
Though out of experience with BM which has a similar kind of wrapper approach, I'm pretty sure you will have hell a lot of work to keep it updated ...
Looking forward though on what you achieve
I'll use Photon for my prototype as it suites my needs best. Potentially with a unity server node linked against it should I decide that I need "unity world simulation" on the server side.


and I agree on the mono thing, thats one of the reasons I kind of hope that 2.6.x is the last 2.x version so we can hope for the mono 2 upgrade happening earlier than later.
due to the outdated mono and the missing debugger, any work with unmanaged code it russian roulette ... took me quite some time to get NetDog work without blowing up unity as the slightest error in marshaling will kill the editor instantanous.

 

Very true that is, I have 2 builds right now of Photon, one local and one on a virtual server that I am also working with, trying to create some documentation for my thesis on my secondary degree, I just saw Unity utilizing RakNet as a good challenge for my current CCSI courseware with regards to live laser readings into a 3d scene for projectory determinators, I have way to much work done with RakNet over the past 3 years to just set it a side, the rooms, the dynamics, I had started with Newton physics then moved to PhysX before it sold out, with its SDK, each room had its own physical and physics layers, kept things running smoother that way. So I wanted to bring this forward from TV3D into Unity, not gone well at all sadly.

I ran into snags with 3Impact, even the wrapper I wrote to allow me to use 3Impact in Unity itself, that kept crashing, maybe by mid summer I will have converted what I need to convert into Unity and hopefully by June or July they will have 3.x.x out using updated Mono or I pray they use native .Net assemblies on Windows and stick with Mono on the Mac. I really hope they go down that path. Why fight native assemblies with Mono assemblies on Windows instead of using native .net framework on Windows, I think that is just too much work IMO.

I feel as if I have come to a screaching hult with the .Net RakNet managed code blocks that I have built, well with Unity anyway, and I don't want to man handle it into GG's T3D which is still in Beta, that would be a perverbial stitch in time. Sigh, this 3 year project, I only have 2 years left to complete it, rather it is due no later than March 1 of 2012. That is my ultimate deadline, all of my prototypes are in pieces, a module here, a project there, all proof of concepts, but before I can finish this degree, I have to have a viable working model.

 

OMFG, seriously I give up! *SIGH* in a major way!
Today I created a managed C++ library EXCEPT it was built via VS 2003 C++ within Parallels, as I stated before, it is nothing more than pointers to pointers and logging, I used it in Unity 2.6.1f3 on the Mac, and it...works... No crash, no problems. *cries* Windows DIES A HORRIBLE DEATH when I try the EXACT same code with MONO crying like it was just whipped like an oriental monkey by long bamboo strings.

How stupid easy is it to create a managed C++ library that works with Unity? Here is my simple stupid project that DIES on 3 different windows machines but works on the Mac. Just open the ZIP file, in the Debug folder there is a MCPPLIB.dll, grab that file. place it in your Assets folder, add a new CS file to your Unity project, call it MCPPLIBTEST to match the code block, here is the code to test this library:

Code (csharp):

1.  
2. using UnityEngine;
3. using System.Collections;
4. using MCPPLIB;
5.  
6. public class MCPPLIBTEST : MonoBehaviour {
7.     string t;
8.     // Use this for initialization
9.     void Start () {
10.         Debug.Log("Starting Managed C++ VS 2003 Test");
11.         // our public test variable is set right now
12.         // _msg=S"Not ready, iClass1 has not been initialized yet.";
13.         t = iClass1.Test();
14.         // now lets check that, throw that to the debug window
15.         Debug.Log(t);
16.         // now lets create a new instance of the test class
17.         iClass1 tx = new iClass1();
18.         // our constructor should have triggered and changed the _msg
19.         t = iClass1.Test();
20.         // lets see what that is in the console
21.         Debug.Log(t);
22.         // now lets destroy our test class object
23.         tx.__dtor();
24.         // our _msg should have changed based on the destructor
25.         t = iClass1.Test();
26.         // lets see what our message holds
27.         Debug.Log(t);
28.         // done with the test
29.         Debug.Log("Managed C++ VS 2003 Test complete.");
30.     }
31. }
32.  

This freaken PROVES that what I am trying to do should be working, period.
Oh my freaking (*(*^$(*% why in the *((*^(* does it not work on Windows without blowing up Mono and Mono p*(*(* all over my stuff with mono errors????

This is hell.
Just look at your console log, it should match the attached image, sometimes I think I should just take up either smoking or drinking or something, where as this would make sense, but it simply does NOT make any sense!

 

Interesting ... cause I see exactly that output on 2.6.1f3 on Windows 7 64Bit Ultimate, UAC on lowest level but still active, NOD32 Smart Security fully enabled, DEP set to protect only important services

Sure its not your security / DEP / UAC thats messing it up?

 

Would expect that from Vista or Windows 7, but XP Pro SP 2? XP Media Center Edition? I only tested it on my Windows 7, and my two different XP builds but those do not have Pro installed on them, they only have the public free version of Unity on them for testing, the only 2 installs I have right now with Unity Pro are my Mac Desktop and my Vista Ultimate Desktop, and of those the only machine that it works perfect on is my Mac Desktop with Pro, I haven't tried it yet on my Vista Ultimate because I know I have the security set high on it for other purposes, not sure well else to look, I figured with the XP Pro SP 2 box, that one would be the best to test with, but it blows up, maybe I should build it up and have the output to the screen and see what it looks like when built and ran.

 

well on non-pro non-managed code does not work so those have to fail ... would guess it kills the whole editor cause you try to bypass unitys direct "check mechanisms"

 

Well unmanaged requires dllimport, but managed does not and uses clr, it is my experience that managed libraries still work in regular Unity, it is the unmanaged assemblies that require the dllimport that is blocked by use for pro only, thus the plugins folder, this is managed and just can sit in the assets folder, so pro or not should not matter, I am doing all the heavy lifting inside the dll, not inside Unity, and Unity is only using the static objects or public ref objects as part of the managed side, so I guess I need more info on that side of things, as far as the 'check mechanisms' go, gar. Something to research though, good point.

 

Managed Libraries work with regular Unity right

But the problem is that managed C++ is only a managed library for the parts where you don't touch unmanaged pointers.
Unmanaged pointers are unsafe code independent on if you wrap them from Managed C++ or from C# through DLL import. Its always a call outside the secure environment. Especially in your case where you rely on OS Kernel components (any include of a h file from windows = unmanaged dll included)

Thats why I think that independent of how you do it: It will always be a Unity Pro, standalone distribution only DLL.

But the definite and final answer to the matter is something only MC++ experts and unity devs can give.

 

Right, but I don't know of any expert MC++ developers and I don't think I will get an answer here in the forums from internal Unity devs, they have way more to worry about than this thread If you look at my project, I am using header files, just not any #pragma unmanaged in the test, but that is going to be my next test, because the _msg object is a private variable that I get information from, but it is in the managed code section. I guess more tests and project updates to this thread by the weekend is in order.

 

I did just that and it works fine on a Win XP 32-bits, with Unity 2.6.1...
Tested in-editor, webplayer build, win standalone build. Don't have a Mac at hand atm but I bet it would work. If it really is managed, 1.0 IL (or 1.1 or 2.0 but anyway none of the new 3.5 IL ops, which is impossible with VS 2003), then it's compatible, period.

Also, I'm neither a Unity dev or a MC++ expert, but I'm a 100% confident that you will not be able to run any unsafe code in the webplayer, however cleverly you wrap it: that would totally defeat the purpose, obviously UT has stripped the webplayer VM of any unsafe capabilities, it's just trapped in a sandbox.

I'm glad I found your post because the same test (barebone managed CLR lib) is a big FAIL on VC++ 2008, and VC++ 2005 as well. Apparently the __gc is deprecated, or we have to use /clrldSyntax (seriously??) but it still fails in my experience. Too bad because it looks like the newSyntax (!) is much simpler.
I'm installing VC++ 2003 atm, will keep you posted...

 

I've yet to meet a MC++ proficient developer that does not hate to work with it

 

Well, VS2003 up and running. I've been able to rebuild your example, change the messages, and reimported in Unity: still works!

But... I've tried referencing the UnityEngine.dll (I need it for what I'm trying to do, although I understand you don't), and VS yells at me. I've tried with a C# project as well, same problem so it isn't C++-specific, I guess the DLLs are newer than what VS2003 support (.NET 2.0 vs 1.1 or something).

I'm downloading VS2003 SP1 atm, hoping this will fix it (2.0 support?)...

 

Let me know how you get along with it, I took a break from this for a bit and switched to playing with Simul Weather SDK, it is also unmanaged code, and the only managed C++ that I can get to work with the current build of Unity is using VS 2003 but I do have SP1 installed already. Now that I am finished with my first CCSI degree, I am going to work on the Masters so the first sets of classes also include programming in C++ dealing with Managed code and a few other things that I am not familiar with, so hopefully in the next 9 weeks I will have learned a trick or two that will make this work cleanly.

 

SP1 didn't do it for me, VS won't link to UnityEngine.dll

 

Today I was finally able to get all of the raknetworkfactory classes and methods to work in my managed c++ library with having a native c# project initialize and use the pointers then call all destructors withou a hitch. The new approach though, won't know until I get back to my home office if Unity will yield the same test results, I should
know in a few hours, these are all the key elements to make raknet work with native managed projects, it was a chore to get it this far.

 

looking forward to hear if you were able to find a way to invoke unmanaged code without p/invoke as p/invoke is not possible in the webplayer at all independent of the how

 

Getting ready to move the project to Pro from this regular box, but this machine has just the free version of Unity on it and here is the console log output: (as noted, not the pro machine yet)

 

http://www.gamedevonline.com/unity/mcpptest.zip

This zip contains just a test project as a windows stand alone build, the current method will not work for web, I didn't suspect it would since it uses 1 unmanaged DLL.

Some things to note, this just outputs to a text area some debug information to show that the hook worked. Also in this zip is a pre-build Raknet.dll release build. A few other things to note:

Files:
W32Standalone.exe <- test game (not really a game, just test)
RakNet.dll latest release of raknet prebuilt.
W32Standalone_Data <- contains the wrapper MCPP.dll and other junk
UnityMCPPTest <- unity package if you want to open Unity and import the package, it adds a demo scene, the MCPP.dll and code to run it, to run it in the Editor COPY the Raknet.dll into the C:\Program Files\Unity\Editor folder, this is a MUST, Unity has many bugs with regards to using unmanaged dll's and the Plugins folder -does not work- if you have a wrapper calling an unmanaged dll, and having the dll in the Editor folder is the main way to make it work.

Also, when building a game with a wrapper that talks to an unmanaged dll, you MUST make a copy of that dll into the ROOT of the game folder like I have Raknet.dll, this is also a MUST.

These are bugs I reported eons ago, which internal Q/A can't reproduce, this is how you fix managed wrappers to unmanaged libraries dll not found issue.

This package works for BOTH Pro and Free versions, this is an INCOMPLETE wrapper, it was a proof of concept and contains only what is necessary to ensure that Raknet can start up all main components and stop them.