Search Unity

Editing and using mixed reality capture in Unity

Discussion in 'AR/VR (XR) Discussion' started by yusefkerr, Mar 1, 2019.

  1. yusefkerr

    yusefkerr

    Joined:
    Apr 5, 2011
    Posts:
    179
    Hello, I have a chance to work on an experimental project for someone who wants to take a mixed-reality capture "hologram" and add effects to it to make it stand out. Here is the file type, "High-Quality Streamable Free-Viewpoint Video":

    https://docs.microsoft.com/en-us/windows/mixed-reality/how-it-works-mixed-reality-capture-studios

    I have seen these used before in VR apps, but haven't used them myself, and I wondered whether there are any sources of information for how people are working with them in unity. A lot of techniques I would usually use to add real-time effects to models (such as referencing animated bone position, or animating the UV's) are not going to work - I don't think there's a bone structure there, and the UV's and underlying mesh change every few frames so editing either is going to be complicated with normal techniques.

    I wondered whether there are other solutions people have tried or can suggest, like post-processing effects that track face/body motion, or the best external program to re-sample these kind of animations. Even good resources on how to manipulate/import/export this file type, or sample project files, would be appreciated.

    The target is a simple AR app with the model and added effects sizzle.

    Please say if this is better in another forum, e.g. External Tools, General Graphics...:)