Question ULSee dependency for FaceTracking?

Congrats to the MARS team for the launch!

Couple quick questions re: facial tracking:

On the MARS FaceTracking doc here it mentions ULSee is a dependency for face tracking but also states "you can still create facemasks in the Editor without ULSee".

AFAIK a "Facemask" is a new concept (at least in terms of ARFoundation) -- is MARS compatible with the established ARFoundation Face Tracking pipeline?

My hope is that MARS supports existing ARFoundation facial tracking but offers ULSee as an option to utilize a standard (non depth-sensing) webcam while in the editor, not as a hard dependency.

If so, couple follow-on questions:

- Is ULSee interpreting facial landmarks to drive ARKit-compatible blendshapes that can be used once deployed to an ARKit-compatible device?

- Are Facemasks themselves usable at runtime w/o ULSee? Ex: via ARKit facial blendshape data

- Does the MARS companion app itself contribute any value to developers working w/ facial tracking? Ex: Streaming depth sensing data to the editor, remoting the scene (similar to the old Unity pre-ARFoundation remote), etc?

Thanks,

Chad

 

Hey Chad,

Yes it is roughly as you had hoped, we have some recordings of faces that come included with MARS, for use in the editor. Or with ULSee you can work on a live stream of your own face in the editor.

When you actually deploy to a device that supports AR Foundation face tracking we default to that.

Note that there may be some features that AR Foundation face tracking supports that aren’t in MARS. You are welcome and encouraged to use both MARS and AR Foundation APIs in the same scene if needed.

Also note that face landmarks are a little different than morph targets, and hopefully easier for some.

We will be launching the MARS companion app later in the year and will describe more of its use then.
-Lewey

 

Thanks for the clarifications Lewey, might suggest adding them to the documentation - your response really clears it up.