Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Unity wants to learn about your experiences working on a Unity project today. We'd like to hear from you via this survey.
    Dismiss Notice
  3. Unity 2020 LTS & Unity 2021.1 have been released.
    Dismiss Notice
  4. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  5. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

AR face tracking for both iOS and Android

Discussion in 'AR' started by dustin_red, Nov 28, 2018.

  1. dustin_red

    dustin_red

    Joined:
    Feb 7, 2018
    Posts:
    46
    I'm looking for good solutions or ideas for AR face tracking in Unity for both iOS and Android. My ideal solution would be one API for both platforms, but could have separate solutions if needed.

    I need to be able to place 3D models (like hats, sunglasses, necklace, etc..) on a user's head, face, and neck, with masking, while tracking position/rotation, and running at a decent framerate. My client is basically asking for Snapchat AR, at least the model placement part, dont really need the face lens mesh part, except for maybe masking. I believe Snapchat's solution is a custom low level version of OpenCV.

    From my quick research, and correct me if I'm wrong or missing something, here are my options:

    iOS:
    Android
    • Vuforia does not currently support face tracking.
    • ARCore does not currently support face tracking.
    • Use OpenCV plugin

    Anyone else seen solutions for this in Unity? What is the best way to achieve face tracking in Unity?
     
    Last edited: Nov 28, 2018
  2. ferretnt

    ferretnt

    Joined:
    Apr 10, 2012
    Posts:
    355
    Obvious open-source alternatives include:

    - DLib (C++ lib, or you can buy a unity plugin that bridges to it.)
    - Firebase MLKit face recognition. Has ObjC/Java API which would be relatively trivial to bridge to Unity.

    The performance both of these options (both in terms of framerate and detection robustness to e.g. head rotation) is nowhere near parity with ARKit, Snapchat lenses, etc.
     
    Appwebstudios and andreiagmu like this.
  3. rob_james

    rob_james

    Joined:
    Oct 6, 2018
    Posts:
    1
  4. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    227
  5. unity_WRbSonr_togDDw

    unity_WRbSonr_togDDw

    Joined:
    Apr 16, 2019
    Posts:
    1
    I would recommend you check this step-by-step guide to building an AR app for iOS. The author in the simple terms explains how to get started with the ARKiT SDK and guides you through the whole process of creating an AR app with all necessary code snippets.
     
  6. martinamenegon

    martinamenegon

    Joined:
    Aug 28, 2014
    Posts:
    29
    I am trying to use the xzimg one as well, but with no success.. would you mind sharing some tips with me? basically not even by just opening their project sample I can manage to open the device webcam :-(
     
  7. dimib

    dimib

    Joined:
    Apr 16, 2017
    Posts:
    47
    Since ARFoundation 3.0 is out, is it now possible to use face tracking on Android with ARF3.0 ?
     
  8. lucycummings120

    lucycummings120

    Joined:
    Jan 31, 2020
    Posts:
    1
    A few of my colleagues were having a similar discussion and they mentioned using AR Foundation and point clouds or planar surface detection to configure similar functions on a 3d image. I think that's something you can look into, especially since AR in mobile apps is trending in the industry. I think specifically for Android, that could work. Good luck!
     
    andreiagmu likes this.
  9. alexeikovalev

    alexeikovalev

    Joined:
    Jun 3, 2020
    Posts:
    7
    Hello dustin_red .

    If you are still interested in such a solution, take a look at Banuba's Unity AR SDK which works for both iOS and Android operating systems, supports the face tracking functionality on the Unity engine apps and allows you to implement virtual try-ons of the accessories you've mentioned. In addition to the latter, there is also a possibility to use their face filters as well as avatars and emojis if you're ever going to need one.

    Best,

    Alexei.
     
  10. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    542
    I would recommend using AR Foundation.
    It wraps native ARKit and ARCore solutions so you'll get native performance and tracking quality.
     
  11. michellescully

    michellescully

    Joined:
    Jul 5, 2020
    Posts:
    1
    All the above answers are indeed useful and good to take into the consideration for developing AR face tracking feature of android and iOS application, I have been into the tech market for 5+ years and I have worked with numerous mobile/web and software application design and development companies who taught me the tech vulnerabilities of various techs, such as AR/VR, IoT, AI, and Chatbot.

    Although, an ideal case would be hiring leading iOS app developers for iOS mobile application AR tech integration and for android, finding an authentic android app development firm that could leverage AR tech sufficiently in the android application would be the right decision.

    Furthermore, after researching a bit I found some resources for your development approach, and that can play an imperative role in your AR face tracking feature development for android and iOS.

    1). https://developers.google.com/ar/develop/unity/augmented-faces
    2). https://developer.apple.com/documentation/arkit/tracking_and_visualizing_faces
    3). https://www.banuba.com/blog/how-to-choose-face-tracking-sdk-for-android-ios
    4). https://www.raywenderlich.com/5491-ar-face-tracking-tutorial-for-ios-getting-started

    Let me know how these blogs you use in your application development, and if you any further questions similar to the above one, I would love to help.
     
  12. Appwebstudios

    Appwebstudios

    Joined:
    Sep 8, 2020
    Posts:
    1
unityunity