View Mixed Reality APIs?

Hello together!
I am researching for a project and am interested in something that microsoft calls "View Mixed Reality". It is basically a trackerless AR technology that can be used on any windows 10 device together with any webcam. In our case it is a Surface Pro tablet which has a camera build in.
Microsoft uses this technology in their app: "Mixed Reality Viewer" https://liliputing.com/2017/05/windows-10-pc-webcam-will-support-mixed-reality-fall.html

On my searches I only find Hololens related information.

My question: Is there a way to access "View Mixed Reality" APIs? Are they maybe hidden somewhere in the Mixed Realty toolkit?

Thanks in advance!

 

This appears to be just a feature (gimmick) built into the 3D Viewer app and doesn't seem to have specific APIs.

However, I looked through the Microsoft Mixed Reality docs which cover a "Locatable Camera" (https://docs.microsoft.com/en-us/windows/mixed-reality/locatable-camera) which provides a coordinate system and project matrix for the camera frame. Using this data you can project 3D content into the camera frame for HoloLens.

It's possible the camera in the Surface Pro may also provide this data (or simulate it) which 3D Viewer app is utilizing (just a guess it not work this way at all). If so, according to the docs you can access the data directly from MediaFoundation by reading extended Attributes on the MFSample.

You won't be able to get these Attributes through Unity or (probably) MR Toolkit, but instead you'll have to use Windows MediaFoundation APIs directly; this is not trivial.

I'd recommend asking this question on the Microsoft developer forums to get better guidance.

 

Thank you timke, for your elaborate answer!
I reckon the Windows MediaFoundation APIs are a bit over my head : )
But yeah the Microsoft developer forums are the next best place to research.