Search Unity

Has the order of blendshapes for face tracking changed since iOS 13?

Discussion in 'AR' started by dnoparker, Oct 4, 2019.

  1. dnoparker

    dnoparker

    Joined:
    Aug 28, 2013
    Posts:
    63
    We created an app with ARKit face tracking and the blend shapes would come consistently in this order -

    "eyeBlink_R",
    "eyeWide_R",
    "mouthLowerDown_L",
    "eyeLookDown_R",
    "cheekSquint_L",
    "mouthDimple_R",
    "browInnerUp",
    "eyeLookIn_L",
    "mouthPress_L",
    "mouthStretch_R",
    "browDown_L",
    "mouthFunnel",
    "noseSneer_L",
    "eyeLookOut_L",
    "eyeLookIn_R",
    "mouthLowerDown_R",
    "browOuterUp_R",
    "mouthLeft",
    "cheekSquint_R",
    "jawOpen",
    "eyeBlink_L",
    "jawForward",
    "mouthPress_R",
    "noseSneer_R",
    "jawRight",
    "mouthShrugLower",
    "eyeSquint_L",
    "eyeLookOut_R",
    "mouthFrown_L",
    "cheekPuff",
    "mouthStretch_L",
    "tongueOut",
    "mouthRollLower",
    "mouthUpperUp_R",
    "mouthShrugUpper",
    "eyeSquint_R",
    "mouthSmile_L",
    "eyeLookDown_L",
    "eyeWide_L",
    "mouthClose",
    "jawLeft",
    "mouthDimple_L",
    "mouthFrown_R",
    "mouthPucker",
    "mouthRight",
    "browDown_R",
    "eyeLookUp_L",
    "mouthSmile_R",
    "mouthUpperUp_L",
    "browOuterUp_L",
    "mouthRollUpper",
    "eyeLookUp_R"

    If you update the iOS to 13 and then run our app it appears to be in a slightly different order?

    Or at least the mouthsmile_L blendshape seems to be triggered by something else and we can't work out what it is.
     
  2. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    Not that I know of. In ARFoundation, we don't rely on any specific order, we just use the string names. The specific order of the blend shapes sounds like an implementation detail that shouldn't be relied upon. Can you provide more context or a bit of code for how you're accessing the blend shape values?
     
  3. dnoparker

    dnoparker

    Joined:
    Aug 28, 2013
    Posts:
    63
    We have since figured out that the order of blendshapes coming from the device has changed since the iOS 13 update. And you're right, we shouldn't really be relying on it. So we have moved to a more generic approach!

    Thanks!