AR Foundation on Hololens (Without MRTK?)

My perception from Unity's demos of AR Foundation, is that it is a single codebase that can handle basic AR across almost any platform. They demo AR Foundation on hololens, but have no tutorials that I can find on how to recreate it. Do you still need MRTK to use AR Foundation with hololens, if so how can I use that same project to build for iOS/Android without build errors when switching platforms?

If I don't need MRTK how do I handle the hololens cursor and gestures?

 

Where did you see that example? I know it's scheduled for 2019 but didn't see any announcements

 

I was just referring to the stage demos they have done from Unite

 

Any news on this? Like, a preview package to try out in 2019.3 beta?

I see direct mention of it here:
https://drive.google.com/file/d/1h4u_vHmm6ISxwn6-NAlcwOsuvVqb4iyK/view

(Slide # 54)

 

You can test it in preview form in 2019.3b5+ using the latest verified version of ARFoundation (2.1.4) and the preview version of the Windows MR package. Easiest way to get this is to edit your project's manifest.json to include this line:

"com.unity.xr.windowsmr": "2.0.3-preview.1",

 

DOWN WITH THE MIXED REALITY TOOLKIT TYRANNY!!!!

 

For anyone wanting a self-updating link to this package's docs:

https://docs.unity3d.com/Packages/com.unity.xr.windowsmr@latest

 

Are there any examples how to set up a very basic scene with AR Foundation image tracking on HoloLens?

I have set up the basic scene and it is working as expected on iOS + Android. I can see the content tied to image targets and all of the content that is just placed in the scene.

I have included the Windows MR 2.0.3-Preview package to the project. The project builds for HoloLens, but it doesn't ask for camera access etc. The content that is not tied to image targets is working as expected. It all stays in place when walking inside the room. The image targets are not working on the HoloLens build.

 

I’m familiar with these examples, but I’m looking for any HoloLens specific setting / tweak. The image target test scene working with Android + iOS is not working on HoloLens. I’m simply placing a simple cube on top of the image target by using the cube as the instantiated prefab. The problem seems to be that the HoloLens is not even asking for the camera access permission although I have enabled webcam access in the capabilities. So it seems I’m missing something else in the setup.

Or should the preview version even support image targets on HoloLens at this stage?

 

@tdmowrer, thanks for the preview link.

One problem though: When I install any of the "new" XR packages (Windows MR, ARCore, ARKit) it prevents any "legacy" XR (e.g. Google Cardboard) from working. (Unity 2019.3.0b6) Are there plans to promote Google Cardboard to non-legacy XR status? My app has to support both AR and VR in the same build.



Thanks,
Adam

 

The Hololens package uses a new stereoscopic rendering path which is incompatible with the legacy one. That means that currently, you cannot mix Cardboard (which uses the legacy path) and the new Hololens package (which uses the new one).

It's fine to mix the ARCore XR Plugin or ARKit XR Plugin and Cardboard, but once you add the new Hololens package to the mix it disables the legacy stereoscopic pipeline.

There was a roadmap talk at Unite that might be useful.

 

I tried Unity 2019.2 + ARFoundation 3.0.1 + ARSubsystem 3.1.0 (preview2) with the HoloLens2 and the
ARSessionState.CheckingAvailability function to know if it's supported. It is not.

In my experience ARFoundation 3.0.1 does not support HoloLens2.

Maybe there is a way to make it work in Unity 2019.3 but MRTK does not support 2019.3.

 

Hi,
the Unity doc says, that there is a 3.0.0 version of this plugin. But it doesn´t appeare in the package manager (2019.3.0f5)

Can anybody help me?

 

I was able to add it with an entry in the json.
But now my app shows nothing. There should be a cube on my HoloLens 1.
Is there a general "how to" for scene and project setup on HoloLens?

edit: on Android it is running like a charm.

 

The roadmap on the video says there will be a preview of ARFoundation on HoloLens in Q4 2019 but I can't find anything about this preview

 

Is ARFoundation supposed to run with the HoloLens 1? I am trying all kinds of settings but only get a black screen with the cursor visible. There is also no splash screen.

My setup:

• Unity 2019.3.0f3
• "com.unity.xr.arfoundation": "3.0.1"
• "com.unity.xr.windowsmr": "3.0.0"
• "com.unity.xr.legacyinputhelpers": "2.0.6",
• "com.unity.modules.xr": "1.0.0"
• "com.unity.modules.vr": "1.0.0",

 

@BenjaminBachman can you try the "ARSession.CheckAvailability()" function to get the result ? Like in the documentation ? (see image attached)

If it return "ARSessionState.Unsupported" it will be a clear indication that the HL does not support ARFoundation.

 

I had some insight from the support.
Hololens for ARFoundation is not supported : https://support.unity3d.com/hc/en-us/requests/771497?page=1

 

Thank you for investigating. I could get some features working by switching to the following versions:

• Unity: 2019:3.0f3
  
• "com.unity.xr.arfoundation": "2.1.4",
• "com.unity.xr.arsubsystems": "2.1.1",
• "com.unity.xr.management": "3.0.5",
• "com.unity.xr.windowsmr": "2.0.3",
• "com.unity.modules.xr": "1.0.0"

I had to use the "tracked pose driver" and the ARCameraManager doesn't work but at least the app is running and objects are showing up.

It seems like I have no access to this link. Do you know anything about HoloLens 2?
Anyway, I will probably get one this week and try it out.

 

You need neither ARFoundaiton nor MRTK to work on HoloLens in Unity. You can just use the native Unity APIs: gaze, SpatialMapping, WorldAnchors, GestureRecognizer, DictationRecognizer, PhraseRecognizer etc.

MRTK in fact uses those as well. Plus some UWP native APIs too. MRTK is higher level kit on top of that, which turns combination of low level inputs into re-usable components. Let's say you have camera.forward, Collider and GestureRecognizer in native APIs. MRTK turns that into a cursor, that snaps onto interactive objects, glides over them, changes its shape based on Ready gesture, smoothens your head jitter with some kind of gravity well formula etc. Or let's say on HoloLens 2 you have API for retrieving hand bones. MRTK finds the index finger bone, sees that it moved there and back in the proximity of a collider and considers it a finger tap. Or extrapolates your index finger, turns it into a spider-man dashed line, finds its intersection with a collider and upon airtap considers it a button tapped from afar. All these things you can implement yourself with native APIs only. If you have time? And things like image markers - good luck re-implementing that from camera rgb feed.

We have an industrial HoloLens navigation system built on top native APIs only. And consider actually going to MRTK instead. It's simply to much to re-create. In fact, we have a home-made TK, if you wish. Last time I checked MRTK (which was then HoloToolkit) it had atrocious architecture (if any), was just a pile of throw-in solutions unrelated to each other. Word on the street is MRTK is re-designed and is good. We're planning to use MRTK's input system, their hover and such handlers, but will probably stick to native APIs for spatial tracking, since there we need low level precision.

The biggest problem of MRTK is not MRTK in itself (the idea is brilliant), it's that it's advertised and promoted (by Microsoft) as if there are no native APIs at all.

As for MRTK vs. ARFoundation (one the latter support HoloLens) the way I speculate is they're going to be "competitors" of some sort, if you speak about their supported devices overlap. MRTK wanting to be a go-to solution for all Windows-based headsets (immersive and holographic) and ARFoundation wanting to be a go-to solution to build cross-platform AR, phone or headset. In that sense, if you only work on the overlapping set of supported devices, you will have a choice. Although, with coming of Azure Spatial Anchors, HoloLens anchors will be able to be decoded on iOS and Android too. So MRTK, if they incorporate Azure anchors, will go mobile too. So go figure.

Why exactly do you want to avoid MRTK?

PS2: I actually recommend learning everything: native APIs, MRTK and ARFoundation. And especially learn what becomes from what.

 

@BenjaminBachman oh yeah sorry. Here is the conversation I had with Unity3D Support team as a PDF.

And one more proof : https://github.com/microsoft/MixedRealityToolkit-Unity/issues/5527

And if you look at my post I said that the function returns "ARSessionState.Unsupported"

 

I'm also unable to get a simple scene with a single cube running on the Hololens 2 using Unity 2019.3.0f6, ARFoundation 2.1.4 and Windows XR Plugin 2.0.3.

I can deploy and start the app, but the cube does not track properly (even though I've set the Tracked Pose Driver to track the HMD reference) and the cube is barely visible because of tons of artifacts. It seems like the screen is rendered to a render target that is corrupted before it has a chance to render to the screen.

Can we get an answer from Unity whether or not this should actually work?

 

I had a similar poor tracking result when I tried a very simple scene with a cube right in front of the camera, using the same setup as for smartphones. You can guess by this table that the support of Hololens is at a very early stage, as most of the basic features are still, by the time I am writing this, not available. I hope we can get more updates from Unity about the future of the support of AR Foundation for these platforms.

 

It seems like in the newest preview version (4.0.0-preview.1) ARFoundation now supports HoloLens and I could successfully test it on a HoloLens2!
The feature set is still very limited though:

See documentation here: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.0/manual/index.html

 

ARFoundation has supported HoloLens since 2.1. Each release of Unity includes a set of "verified" packages. When a package is verified for a version of Unity, it refers to a set of packages that are known to work together for that version of Unity. I strongly recommend using the verified packages that go with your version of Unity.

ARFoundation 2.1 and Windows XR Plugin 2.0 are verified for Unity 2019.3
ARFoundation 3.1 and Windows XR Plugin 3.0 are verified for Unity 2020.1

ARFoundation 4.0.0-preview.1 is in, well, preview and hasn't yet been verified with any version of Windows XR Plugin.

 

@tdmowrer thank you for your explanation.
So how do you explain this ?
Can you provide us a screenshot in the HoloLens 2 where the function return "ARSessionState.Supported" ?
Thank you.

 

I have just tried it again on HL2 without MRTK and the verified packages (for unity 2020.1) and it still doesn't work properly. Some tracking seems to be there but there is a bug/problem with the camera clear and I see horrible rendering artifacts.

Edit:
I have set up a very simple Unity 2020.1 project with AR Foundation for HoloLens 2 which exhibits the rendering problem. See here: https://github.com/brookman/hl2_ar_foundation
@tdmowrer Can you tell me what I am doing wrong?

 

Is there an update regarding ARFoundation on the HL2?

 

We still face the same problem. Is there any update?

 

Any update ?