Core sample remains black on HoloLens

Hi,

I have some trouble getting the Core sample to work on the HoloLens. I tried to follow the getting started [1] section as good as possible, but some pages such as [2] seem to be a bit outdated and refer to older Unity versions, writing about options that are apparently gone in the current version. So I created a build including scenes Splash, Menu, Loading, and ImageTargets, generated a VS project out of that and built via VS (x86 Release). With my HoloLens connected, I hit the Play button in Visual Studio (set to "Device" target). I also tried to deploy and start out of the HoloLens menu directly. No matter what I did, what I got was a build on the HoloLens just showing a Unity splash screen at startup, then the display stayed black and nothing happened. I had a look into the log file, it says it fails to get spatial stage statics, though I'm not sure if that's related to the problem as I don't know what spatial stage statics are:

At [2] I saw scripting backend set to .NET, but when I try this, I get the following error during building in Unity:

When I run the same in the Unity Editor, I first see the Vuforia splash screen, then I am forwarded to the menu scene, showing the white scene selection buttons.

Well, I am completely new to Vuforia and I'm also not absolutely well versed in Unity development, especially for platforms other than PC, so I am most probably doing something wrong, but at this stage I'm a bit lost and don't know where to start troubleshooting. I would highly appreciated any thoughts and suggestions about what could be wrong and how I could debug the issue.

Thanks!

(Using Unity 2017.2.0f3 64bit)

[1] https://library.vuforia.com/articles/Training/getting-started-with-vuforia-in-unity-2017-2-beta.html
[2] https://library.vuforia.com/article...-with-Vuforia-for-Windows-10-Development.html

 

Hello rAigner,

I posted a short guide for getting started with Hololens in 2017.2 in this thread: https://forum.unity.com/threads/deploying-the-stereo-rendering-sample-to-hololens.500488/

A specific sample app for Hololens will also be available in the future.

Could you review the above guide and see if that works for you? Feel free to follow up here or there with any issues you encounter along the way.

Thanks!

 

Thanks for your reply. I tried the Stereo Rendering Sample and it does not work, unfortunately: I suppose, in VS I have to build Release x86, and set the target to "Device", right? When I deploy and run, the following exception is thrown:

Stack trace is:

Btw, what am I supposed to see in the sample at all? The Core Samples worked in the Unity Editor at least, anyways I got them to work by fiddling with the settings a bit. But here, when I run from 0-Splash, I first see the splash screen (0-Splash), then the loading animation (1-Loading), then the player window remains black and I get some warnings (complaining about missing scripts) and errors (e.g. "Could not start graph -> sounds DirectShow related?)

EDIT:
Okay, the exception is gone now since I upgraded to VS 2017. I had to use VS 2015 until recently since the HoloLens builds (without Vuforia) wouldn't work in VS 2017. The errors/warnings in Unity Editor Play Mode seem to occur due to the camera issues I mentioned earlier in my other post -- my webcam still isn't detected by Vuforia, no idea why.

The initial problem with the Core Sample build showing nothing however persists, so far, but at least I have a working reference project now.

 

Is there also an updated VuMark sample somewhere? I'm failing to get 3-VuMark to work... VuMark is maybe not detected, or something is misconfigured, or idk. I also tried to build one on my own, but I'm failing.

 

The only VuMark sample is in the core samples, which you can adapt for the stereo rendering.

What do you have setup in your scene and what image are you trying to detect? What do the logs say when you try this?

 

If I could get my USB camera to work with Vuforia that would really be a big deal since it is next to impossible to trace what's going on on the final build on the HoloLens. Having this run in the player would help observing what is going on.

About the VuMark I'm using: I try to get 3-VuMark to work, seems like this is a Mars marker. However, I noticed it doesn't match any of the MarsVuMark_X.pdf files that are contained in the package, so I printed the texture that's used in the scene itself. For testing I printed a mirrored version, like I mentioned the only camera Vuforia is able to run is the RGB sensor of my Kinect2, and Kinect color data is always flipped by design, but the according checkbox in the Vuforia Configuration doesn't work. I created the project from scratch and did a fresh import of the core samples package as I wasn't sure if maybe I messed up settings in the meantime. I enabled Virtual Reality Support in XR Settings, removed Oculus and OpenVR (for testing in the editor I use PC Standalone, but I also tried UWP, no difference there), and added Vuforia. In Vuforia Configuration I inserted my license key and enabled "Load VuforiaMars_VuMark" as well as the "Activate" checkbox beneath it. I switched "Camera Device" to "Kinect V2 Video Sensor" and checked "Flip Horizontally". When I hit play, I see the camera image in the background and a crosshair/frame above it. Holding my printouts (mirrored and non-mirrored) in front of the camera, nothing happens (does the marker have to be *inside* the crosshair edges? The camera resolution wouldn't suffice if so).

The log doesn't say anything unusual. How is this sample supposed to work, actually? What is supposed to happen? Is there any user interaction required?

Besides, given the nature of the Mars marker I would be very surprised if it worked well out of the box, as there is really very little contrast between the VuMark frame and the image content, especially at the upper regions, at the sun/atmosphere. Content and frame are really close in terms of hue, brightness, and saturation. I expect you need perfect lighting conditions and a totally diffuse material for the print, with absolute non-glossy surface, so there are no highlights at all, puls a very high-res camera. Is this really a good marker to start with?

EDIT:
I continued creating builds for the HoloLens, containing 3-VuMark scenes as follows:

In Player Settings, I switched platform to UWP, set Target Device to "HoloLens", Build Type "D3D", SDK "latest installed", Build and Run on "Local Machine". In Player Settings, UWP Tab, I checked "InternetClient", "InternetClientServer", "WebCam", "Microphone", and "SpatialPerception" in Capabilities, enabled "Virtual Reality Supported" in XR Settings, added Windows Mixed Reality and Vuforia (btw: which one do I need? Only WMR, only Vuforia, or both? I tried all combinations, see below). In Vuforia Configuration i set Eyewear Type to "Optical See-Through", See Through Config to "HoloLens. I click build to generate a VS project, then in VS 2017, I select Release, x86, switch from "Local Machine" to Device, build the solution, deploy, and try to run.

Here are my results:

WMR:
1st try: deploy fails:

2nd try, without any modification whatsoever, deploy succeeds, but exception is thrown:

3rd try, run without debugging: App starts, screen stays black after Unity splash screen, nothing happens, VuMark detection doesn't work.. Logfile has 0 bytes.

Vuforia:
1st try: deploy fails:

2nd try, without any modification whatsoever, deploy succeeds, but exception is thrown:

3rd try, run without debugging: App starts, then seems to have ended/crashed as I'm back in Windows menu. Log file shows no error:

Vuforia + WMR:
1st try: deploy fails:

2nd try, without any modification whatsoever, deploy succeeds, image stays black, nothing happens, VuMark detection doesn't work, logfile has 0 bytes.
3rd try, run without debugging:
Image stays black, nothing happens, VuMark detection doesn't work, logfile has 0 bytes.

 

The texture used for the actual VuMark game object in the scene is the template for that VuMark and won't be detected. The textures called MarsVuMark_X.pdf are the various instances generated for this VuMark and will be detected. I've attached a screenshot of what it looks like when one of these is detected in the core samples. I've also attached a screenshot of what your XR Settings should look like.

Does the Console in Visual Studio have anything in it when you run the app on the HoloLens?

 

Thanks.

I tried the VuMarks from the PDFs first and they didn't work, this was actually the reason I looked closer and found they differ from the texture.

Yes, when there is the Platform::COMException, there indeed are a few errors output to the console which I didn't notice earlier:

Here's the complete output:

 

Do you see this error:

when running in Playmode as well?

 

No, I don't.

Is it possible you guys provide an up-to-date sample for me to download somewhere?

In more general terms, I would suggest you provide complete pre-configured Unity projects as samples, thus including project settings, instead of just Unity packages and then running the users through lenghty how-to's in order to just get it to work. It would also be helpful to get to know the exact versions of Unity, Visual Studio, SDK and what not, thus the preconditions the package is guaranteed to work with, so we can just open, build, and run. Once we have a sample running, we can go from there, but when we have to struggle to get a sample to work, it gets a bit tedious, as we don't know where to start looking for the problem. This would be a lot easier for both your any us. Just a suggestion, because I'm trying for five days now to get a VuMark sample to work and I am starting to get a bit frustrated.

Edit:
I just tried to recreate a project with a previously working sample from scratch (3-ImageTarget), as I was trying to combine it with HoloLens gesture recognition. Now I get this:

I have no idea what is wrong again. Seems like several sensors cannot be opened but I have no clue why and what I did wrong.

 

There is a HoloLens specific sample that is coming in the future, the instructions for adapting the Stereo Rendering sample for use with the HoloLens are meant as a temporary sample to assist users in the mean time.

If you follow the steps in the thread with these instructions (linked above), you will end up with a project that can deploy to HoloLens and has a working Image Target. You can then create a VuMark within this project that will also work. You can use the Core Samples as a reference (or drag the files directly from the Core Samples) to get the VuMark working in the Stereo Rendering sample.

Can you confirm that this is what you are doing? It sounds like you might be trying to update the Core Samples to compile for HoloLens, which is not recommended.