Intel RealSense support in Unity 5.3.x WebGL? DLLNotFoundException

Hey everyone,

I built a RealSense game demo for Intel's RealSense App Challenge last year, and I'd like to host it on the web via Unity's WebGL exporter for anyone with a RealSense to enjoy. I can build to WebGL and the game runs, but the RealSense (F200 version) doesn't get activated and turn on in the WebGL build.

I saw that Webcams are now enabled in Unity 5.3, so is there a way for me to enable the RealSense as well? Hopefully it's just a matter of sending it the signal to activate? Pretty noob on this peripheral stuff, especially in WebGL, so any help would be appreciated!

Edit: Tried just running the RealSense as a webcam via WebcamTexture to put the RGB feed on a plane in-game, and while it worked in the Editor, again the RealSense camera did not even turn on from the WebGL build, let alone show the feed.

 

Hey Marco,

Not sure if you saw my edit to my original post above, but yes I did try that; it works in the editor (shows the RGB feed on a plane), but it's the same result in the WebGL build: the camera never activates.

Is there something special I have to do to activate a webcam via WebGL? I saw a thread where in the WebPlayer you had to ask the user for permission; do I have to do that too to activate the webcam?

As for the RealSense itself, it's a depth camera like a Kinect that's meant to be used up close. They have a Unity plugin which works on desktop.

 

Ok, I'm convinced the problem is that I'm not properly requesting authorization from the user to enable the webcam/microphone in the game window. Here's my code:

Code (CSharp):

1. IEnumerator AskPermission() {
2.         yield return Application.RequestUserAuthorization(UserAuthorization.WebCam | UserAuthorization.Microphone);
3.         if(Application.HasUserAuthorization(UserAuthorization.WebCam | UserAuthorization.Microphone))
4.         {
5.             WebCamDevice[] devices = WebCamTexture.devices;
6.             string deviceName = "none";
7.             for(int i = 0; i < devices.Length; i++)
8.             {
9.                 if(devices[i].name.Contains("RGB"))//RealSense has 3 inputs, so we want the RGB one for the webcamTexture
10.                     deviceName = devices[i].name;
11.             }
12.  
13.             print("deviceName = "+deviceName);
14.  
15.             WebCamTexture webcamTexture = new WebCamTexture(deviceName);
16.             Renderer renderer = GetComponent<Renderer>();
17.             renderer.material.mainTexture = webcamTexture;
18.             webcamTexture.Play();
19.         }
20.     }

I've tried triggering this both in Start() and when a button is pressed, but all it does is turn the webcamTexture black and I never get a prompt from my browser. Am I doing something wrong?

 

Got it. Oddly enough, the first thing I tried was to implement the latter part of that code on its own:

Code (CSharp):

1. WebCamDevice[] devices = WebCamTexture.devices;
2. string deviceName = "none";
3. for(int i = 0; i < devices.Length; i++)
4. {
5. if(devices[i].name.Contains("RGB"))//RealSense has 3 inputs, so we want the RGB one for the webcamTexture
6.       deviceName = devices[i].name;
7. }
8.  
9. print("deviceName = "+deviceName);
10.  
11. WebCamTexture webcamTexture = new WebCamTexture(deviceName);
12. Renderer renderer = GetComponent<Renderer>();
13. renderer.material.mainTexture = webcamTexture;
14. webcamTexture.Play();

But this yields the same result: I never get a pop-up asking to authorize the webcam in WebGL, and consequently the webcam never turns on. Am I missing code here that would generate that pop-up?

 

Quick update: It looks like the problem might be that Unity WebGL isn't seeing an important RealSense DLL file. I'm getting:

I can now actually call the webcam, get the user permission dialog, and get it to show my camera feed on the plane via WebCamTexture, but the RealSense won't do depth detection (even though that feed is active), presumably due to this DLL not being found.

So now my question is: how do I include this DLL into my WebGL build if it doesn't work now? it's currently in the Plugins folder and works fine on desktop builds.

Thanks for all the help!

Edit: DLL files in question are attached if it helps.

Tagging @jonas echterhoff as he seems to be a WebGL expert as well

 

Thanks for the note! How would I be able to tell if they're managed vs. native? I'm guessing they're native since it won't be found on the WebGL build, but their Unity Toolkit does come with a "Plugins.managed" folder (screenshot attached), so I'm hopeful that there's some way to still get this working. I'm also able to find Visual Studio .sln files with the DLL's name but not source code, so I'm opening the project to see what I can do.

Do you have any tips on what sources I'd need and what to do with them in Unity? Is it literally just drop them in?

Thanks again!

 

DllNotFoundException: Unable to load DLL 'MathFuncsDll': The specified module could not be found.

What, Marko? WebGL can't use DLLs - ? it's a joke? It's unacceptable !!!

 

@yuliyF as I understand it now, browsers don't allow native DLLs for security reasons. The only way around it is to load the source code for the DLL into Unity instead of the DLL itself, but I don't have the source for the RealSense DLL I've contacted my reps at Intel to try and obtain it, though. Will post here if I make progress.

 

The RealSense DLLs use hardware specific code that you can't be reach from the web, for security reasons...

But I know that there's a RealSense JavaScript SDK - so maybe a solution for you we'll be to implement it yourself to unity.

 

Thanks guys! I've reached out to Intel for some guidance, will report here if I hear anything.

 

try to look this: Tools and libraries to glue C/C++ APIs to .NET

 

Interesting...I'll give this a try, thanks!

 

hello
I use the webcamtexture in webgl by unity3d 5.3.2,but the camera texture is black,why?
my code:
WebCamTexture webcamTexture = new WebCamTexture();
Renderer renderer = GetComponent<Renderer>();
renderer.material.mainTexture = webcamTexture;
webcamTexture.Play();

 

Did you ever get this to work for you? I am starting with the realsense sr300...thanks.

 

Could you elaborate a little bit more about "dropping c++ code into unity and getting it compile it".
Is there any special folder to drop c++ code?
I have a quite complex c++ library with some dependencies and I have a working dll out of it.

Right now, the native plugin works well in android and desktop but I also need it in WebGL but the dll doesn't work there.