Search Unity

Question Face Tracking (Facemask) Roadmap goes nearby to Spark (Instagram)

Discussion in 'Unity MARS' started by makaka-org, Jun 2, 2020.

  1. makaka-org

    makaka-org

    Joined:
    Dec 1, 2013
    Posts:
    1,026
    As I understand at this moment Unity wants to develop Facemasks features in MARS in way of Spark by Facebook, but with power of Unity. Am I right?

    Tell about plans, please.
     
    makaka-org, Jun 2, 2020
    #1

  2. timoni

    timoni

    Joined:
    Mar 23, 2015
    Posts:
    12
    Hello! You can currently make facemasks in Unity either using a model in the sim view, or via live or recorded video. If you want live face tracking on top of your video, you'll need a third-party provider like ULSEE. Unity does not provide our own face recognition technology.

    As far as I'm aware, Facebook doesn't allow for their Spark AR ML tech to be exposed in external tools outside of Spark AR.
     
    timoni, Jun 2, 2020
    #2

  3. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    jmunozarUTech, Jun 2, 2020
    #3

  4. makaka-org

    makaka-org

    Joined:
    Dec 1, 2013
    Posts:
    1,026
    I don't mean export mask to Instagram. I mean node based environment like Spark.
     
    makaka-org, Jun 2, 2020
    #4

  5. Jono_Unity

    Jono_Unity

    Unity Technologies

    Joined:
    Apr 5, 2016
    Posts:
    18
    Hey @atorisa, are you talking about Spark's 'Patch Editor'? MARS doesn't provide this style of Visual Scripting / graph based behavior authoring, but we do provide non-code ways to react to face expressions. Visual Scripting as a whole is a separate project at Unity.

    In MARS, you can hook into face events from the Inspector, which will let you define what happens for different expressions (Mouth Open/Close, Eyes Open/Close, etc). If you make a Face Mask Proxy, check out the `Face Expression Action` on it, and under `Add Event` you'll see those expressions:
    upload_2020-6-2_16-4-59.png

    DOTS Visual Scripting is a separate project, and the latest on that is at this forum thread: https://forum.unity.com/threads/dots-visual-scripting-experimental-drop-8.857707/
    Alternatively, Unity recently brought Bolt Visual Scripting into the first-party fold, so I'd encourage you to check that out if you haven't already: https://docs.unity3d.com/Manual/VisualScripting.html

    We aren't planning to integrate MARS with Visual Scripting in the near-term, but would love to hear if this is an important feature for you!
     

    Attached Files:

    Jono_Unity, Jun 2, 2020
    #5

  6. makaka-org

    makaka-org

    Joined:
    Dec 1, 2013
    Posts:
    1,026
    So since you position MARS as Paid Tool for Professionals, yes it is desired feature.
     
    makaka-org, Jun 2, 2020
    #6

  7. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    @atorisa, For the time being, BOLT visual scripting should work with MARS if needed.
     
    jmunozarUTech, Jun 2, 2020
    #7

  8. wenhan9604

    wenhan9604

    Joined:
    Oct 10, 2019
    Posts:
    1
    Hi timoni, thanks for the information using live face tracking. How do i set up the third-party provider like ULSEE on Unity?
     
    wenhan9604, Jun 11, 2020
    #8

  9. Jackiedg

    Jackiedg

    Joined:
    Jul 10, 2018
    Posts:
    3
    Hey Timoni, Im also looking to integrate ULSEE with Mars, is there an easy solution for this?
     
    Jackiedg, Jun 12, 2020
    #9

  10. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    > How do i set up the third-party provider like ULSEE on Unity

    Hi there! Thanks for your interest in MARS and ULSee. We are currently coordinating with ULSee to update their Asset Store package to include MARS integration code. When this is ready, you will be able to purchase the ULSee Face Tracker Lite asset, import it in a project with MARS, and have access to face tracking features in the Editor.

    I will update this thread when the asset is ready.
     
    mtschoen, Jun 12, 2020
    #10