Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Is it possible to build a packaged version receiving arkit blendshape information?

Discussion in 'Virtual Production' started by fengkan, Aug 5, 2021.

  1. fengkan

    fengkan

    Joined:
    Jul 10, 2018
    Posts:
    82
    Without the connections panel in the editor to create server and so on.

    There are so much code in the Editor folder, so I am wondering if it's even possible to do face capture in a packaged application.
     
  2. ScottSewellUnity

    ScottSewellUnity

    Unity Technologies

    Joined:
    Jan 29, 2020
    Posts:
    20
    This is something we want to be able to support in the future, so we've architected the package with this in mind, but there are a few key challenges to implementing this:
    • There are a lot of Editor only functionalities that we would need to replace (AssetDatabase, animation curve utilities, etc)
    • We'd need to add a runtime UI for the various components.
    Those are very significant time investments, so for now we we have focused on the Editor case, given that for most production working in the Editor is preferred to reduce iteration time and increase flexibility.

    For now, if you really want a runtime build that is able to communicate with the Face Capture app, the best approach would be to import the package source code and use the
    Unity.LiveCapture.ARKitFaceCapture.
    FaceClient class directly. You'll still need to implement the take database, UI, etc. yourself, but at least you can talk to the app the same way our current packages does.

    Out of curiosity, what is your use case for a Runtime build?
     
  3. fengkan

    fengkan

    Joined:
    Jul 10, 2018
    Posts:
    82
    Hi, Scott, thanks for the reply, it's very helpful.

    We build vtuber applications for our customers' characters so that they can make some short videos or live shows. Currently, we have our own face capture app based on the Facial AR Remote repository, but I think it will be great to use an official build, thus we don't need to worry about the app anymore, just like the ‎Live Link Face app of Unreal.

    We will take a look at the Unity.LiveCapture.ARKitFaceCapture package while waiting for the future release :)