Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

AR face tracking for both iOS and Android

Discussion in 'AR' started by dustin_red, Nov 28, 2018.

  1. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    I'm looking for good solutions or ideas for AR face tracking in Unity for both iOS and Android. My ideal solution would be one API for both platforms, but could have separate solutions if needed.

    I need to be able to place 3D models (like hats, sunglasses, necklace, etc..) on a user's head, face, and neck, with masking, while tracking position/rotation, and running at a decent framerate. My client is basically asking for Snapchat AR, at least the model placement part, dont really need the face lens mesh part, except for maybe masking. I believe Snapchat's solution is a custom low level version of OpenCV.

    From my quick research, and correct me if I'm wrong or missing something, here are my options:

    iOS:
    Android
    • Vuforia does not currently support face tracking.
    • ARCore does not currently support face tracking.
    • Use OpenCV plugin

    Anyone else seen solutions for this in Unity? What is the best way to achieve face tracking in Unity?
     
    Last edited: Nov 28, 2018
    drawcode likes this.
  2. ferretnt

    ferretnt

    Joined:
    Apr 10, 2012
    Posts:
    309
    Obvious open-source alternatives include:

    - DLib (C++ lib, or you can buy a unity plugin that bridges to it.)
    - Firebase MLKit face recognition. Has ObjC/Java API which would be relatively trivial to bridge to Unity.

    The performance both of these options (both in terms of framerate and detection robustness to e.g. head rotation) is nowhere near parity with ARKit, Snapchat lenses, etc.
     
  3. rob_james

    rob_james

    Joined:
    Oct 6, 2018
    Posts:
    1
  4. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    193
  5. unity_WRbSonr_togDDw

    unity_WRbSonr_togDDw

    Joined:
    Apr 16, 2019
    Posts:
    1
    I would recommend you check this step-by-step guide to building an AR app for iOS. The author in the simple terms explains how to get started with the ARKiT SDK and guides you through the whole process of creating an AR app with all necessary code snippets.
     
  6. martinamenegon

    martinamenegon

    Joined:
    Aug 28, 2014
    Posts:
    28
    I am trying to use the xzimg one as well, but with no success.. would you mind sharing some tips with me? basically not even by just opening their project sample I can manage to open the device webcam :-(
     
  7. dimib

    dimib

    Joined:
    Apr 16, 2017
    Posts:
    16
    Since ARFoundation 3.0 is out, is it now possible to use face tracking on Android with ARF3.0 ?