Custom mesh for AR Face Visualizer

Hello, I want to make AR Face tracking for Android with AR Foundation. AR Face Mesh Visualizer works great, but I can not find the way to use my own custom mesh.

 

Unfortunately, you can't use a custom face mesh with ARCore/ARKit. This mesh is generated by ARCore/ARKit at runtime.

What you can do - is to place your mesh on top of a tracked face. You can use ARCore face regions to place different parts of your mesh on top of left/right forehead and nose tip.

 

Thanks for the answer. I tried "ARCoreFaceRegions" sample from AR Foundation Sample. But it detect just 3 points on my face: 2 eyebrow and nose. It looks that mouth tracking is not detected.

 

@Cenda yes, the mouse position is not included to face regions. I made a test and found ARFace indices that correspond to the main mouse points.
1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner

Here is a quick example, you can find other indices with it. Add this script to your face prefab. Also, add ParticleSystem to visualize mesh points. This is a dirty hack and there is no guarantee that face indices will not be changed in future versions of ARCore.

Code (CSharp):

1. using System.Linq;
2. using UnityEngine;
3. using UnityEngine.XR.ARFoundation;
4.  
5.  
6. public class FaceMeshTest : MonoBehaviour {
7.     [SerializeField] new ParticleSystem particleSystem = null;
8.     [SerializeField] ARFace face = null;
9.     [SerializeField] int skipNumOfIndices = 1; // 1: top lip, 14: bottom lip, 78: left mouse corner, 292: right mouse corner
10.     [SerializeField] int visualizeNumOfIndices = 1;
11.  
12.  
13.     void Awake() {
14.         face.updated += delegate {
15.             var mainModule = particleSystem.main;
16.             var particles = face.vertices
17.                 .Skip(skipNumOfIndices)
18.                 .Take(visualizeNumOfIndices)
19.                 .Select(pos => new ParticleSystem.Particle {
20.                     startColor = mainModule.startColor.color,
21.                     startSize = mainModule.startSize.constant,
22.                     position = pos,
23.                     remainingLifetime = 1
24.                 })
25.                 .ToArray();
26.              
27.             particleSystem.SetParticles(particles, particles.Length);
28.         };
29.     }
30. }
31.  

 

What a great tip! Thank you I can get position of these components and move with my bones on the face

 

How to use this? this display nothing !! i thought this will display particle on the detected face but nothing happens!!

 

Please make sure that you've populated properties in Inspector and set up ParticleSystem correctly.

 

I had followed all steps :

>>AR Foundation setup :
>> Add face Manager
>> Create new AR Default face and add your script and assign the same object as Face and debug particle
>> create a prefab of Face and assign in face manager.
>> Run in device


Face prefab displayed but nothing else !! even I tried transparent material on face object


If possible can you provide a demo GitHub project for the test ?

 

I replaced ParticleSystem with spheres and uploaded an example to github:
https://github.com/KirillKuzyk/ARFo...-for-arcore/blob/master/Assets/MousePoints.cs

 

Great. It works, Thanks dude for helping.