Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Custom mesh for AR Face Visualizer

Discussion in 'Handheld AR' started by Cenda, Jul 14, 2020.

  1. Cenda

    Cenda

    Joined:
    Jun 3, 2010
    Posts:
    59
    Hello, I want to make AR Face tracking for Android with AR Foundation. AR Face Mesh Visualizer works great, but I can not find the way to use my own custom mesh.
     
  2. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    177
    Unfortunately, you can't use a custom face mesh with ARCore/ARKit. This mesh is generated by ARCore/ARKit at runtime.

    What you can do - is to place your mesh on top of a tracked face. You can use ARCore face regions to place different parts of your mesh on top of left/right forehead and nose tip.
     
  3. Cenda

    Cenda

    Joined:
    Jun 3, 2010
    Posts:
    59
    Thanks for the answer. I tried "ARCoreFaceRegions" sample from AR Foundation Sample. But it detect just 3 points on my face: 2 eyebrow and nose. It looks that mouth tracking is not detected.
     
    Last edited: Jul 14, 2020
  4. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    177
    @Cenda yes, the mouse position is not included to face regions. I made a test and found ARFace indices that correspond to the main mouse points.
    1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner

    Here is a quick example, you can find other indices with it. Add this script to your face prefab. Also, add ParticleSystem to visualize mesh points. This is a dirty hack and there is no guarantee that face indices will not be changed in future versions of ARCore.

    Code (CSharp):
    1. using System.Linq;
    2. using UnityEngine;
    3. using UnityEngine.XR.ARFoundation;
    4.  
    5.  
    6. public class FaceMeshTest : MonoBehaviour {
    7.     [SerializeField] new ParticleSystem particleSystem = null;
    8.     [SerializeField] ARFace face = null;
    9.     [SerializeField] int skipNumOfIndices = 1; // 1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner
    10.     [SerializeField] int visualizeNumOfIndices = 1;
    11.  
    12.  
    13.     void Awake() {
    14.         face.updated += delegate {
    15.             var mainModule = particleSystem.main;
    16.             var particles = face.vertices
    17.                 .Skip(skipNumOfIndices)
    18.                 .Take(visualizeNumOfIndices)
    19.                 .Select(pos => new ParticleSystem.Particle {
    20.                     startColor = mainModule.startColor.color,
    21.                     startSize = mainModule.startSize.constant,
    22.                     position = pos,
    23.                     remainingLifetime = 1
    24.                 })
    25.                 .ToArray();
    26.              
    27.             particleSystem.SetParticles(particles, particles.Length);
    28.         };
    29.     }
    30. }
    31.  
     
  5. Cenda

    Cenda

    Joined:
    Jun 3, 2010
    Posts:
    59
    What a great tip! Thank you :) I can get position of these components and move with my bones on the face :)
     
  6. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    41
    How to use this? this display nothing !! i thought this will display particle on the detected face but nothing happens!!
     
  7. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    177
    Please make sure that you've populated properties in Inspector and set up ParticleSystem correctly.
     
  8. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    41
    I had followed all steps :

    >>AR Foundation setup :
    >> Add face Manager
    >> Create new AR Default face and add your script and assign the same object as Face and debug particle
    >> create a prefab of Face and assign in face manager.
    >> Run in device


    Face prefab displayed but nothing else !! even I tried transparent material on face object


    If possible can you provide a demo GitHub project for the test ?
     
  9. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    177
  10. jalajshah

    jalajshah

    Joined:
    Mar 5, 2018
    Posts:
    41
    Great. It works, Thanks dude for helping.
     
unityunity