Search Unity

ARKit face mesh stops updating when I access the mesh.

Discussion in 'AR/VR (XR) Discussion' started by itsDrew, Jan 20, 2019.

  1. itsDrew

    itsDrew

    Joined:
    Aug 16, 2017
    Posts:
    32
    Unity 2018.2.11f1
    Iphone XR
    ARKit Plugin https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/src/default/

    I created a game object recorder... It records relevant transform data and mesh data.
    Here is a test scene with it in action on a cube. (I'm controlling the transform with inputs and randomly deforming the mesh).




    I've applied the recording tool I made to the AR face mesh sample scene. As soon as I click the 'Record' button, the face mesh stops deforming. The face object still moves around and gets recorded, but if I do something like open my mouth, the mesh doesn't follow. In fact, if I start recording with my mouth (and the face objects mouth) open, it just sticks open when I click the record button. My recorder does not modify the mesh at all, it just reads the mesh data. Why would reading the mesh data stop ARKit from being able to deform the mesh? No errors or warnings in the logs... Assuming async stuff is going on in the background. Maybe some kind of race condition?

    Any thoughts as to how to workaround?
     
  2. itsDrew

    itsDrew

    Joined:
    Aug 16, 2017
    Posts:
    32
    For anyone wondering, I just solved this... I'm assuming that a reference to meshFilter.mesh that is normally cached ends up breaking when I access the mesh from a different script. I know that same behavior occurs when directly accessing fields from the physics engine.


    Code (CSharp):
    1.         void FaceUpdated(ARFaceAnchor anchorData) {
    2.             if (faceMesh != null) {
    3.                 gameObject.transform.localPosition = UnityARMatrixOps.GetPosition(anchorData.transform);
    4.                 gameObject.transform.localRotation = UnityARMatrixOps.GetRotation(anchorData.transform);
    5.                 faceMesh.vertices = anchorData.faceGeometry.vertices;
    6.                 faceMesh.uv = anchorData.faceGeometry.textureCoordinates;
    7.                 faceMesh.triangles = anchorData.faceGeometry.triangleIndices;
    8.                 faceMesh.RecalculateBounds();
    9.                 faceMesh.RecalculateNormals();
    10.  
    11.  
    12.                 meshFilter.mesh = faceMesh; //Need to put this line in if you are accessing the data elsewhere
    13.  
    14.  
    15.             }
    16.         }