Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question LIveCapture Head Position seems completely inaccurate

Discussion in 'Virtual Production' started by dgoyette, Jul 7, 2023.

  1. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    4,212
    I'm happy to be corrected here, but it seems like Live Capture Head Position doesn't work. When I activate it, it always places the head behind the body a significant amount. You can see the two inspectors in this screenshot, one showing the settings for the Face Mapper, and the other showing the Transform for the Head bone that the mapper is moving:

    upload_2023-7-7_12-2-54.png

    I don't understand where the -0.34~ Z-position amount is coming from. This is with a character coming out of Character Creator 4. But I also tried it with a default Mixamo model as well, with the same issue:

    upload_2023-7-7_12-7-57.png

    The head is always about -0.34 back on the z-axis.

    Is this a bug in Face Capture? Or is there a way to configure this to account for it?

    I've tried moving the camera very close to my face, and I think the -0.34 is actually how far the camera is from my face. So, to get the head to sit on the neck properly, I'd need to put my phone inside my actual physical head... Is there a z-offset we can adjust for this?
     
  2. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    4,212
    FWIW, if I edit the Default Face Mapper, and offset the head position it's receiving, the animation looks pretty good:

    Code (CSharp):
    1.         public override void ApplyHeadPositionToRig(
    2.             FaceActor actor,
    3.             FaceMapperCache cache,
    4.             ref Vector3 headPosition,
    5.             bool continuous
    6.         )
    7.         {
    8.             if (actor == null)
    9.                 return;
    10.  
    11.             var c = cache as Cache;
    12.  
    13.             if (!string.IsNullOrEmpty(m_HeadPosition) && c.TryGetBone(actor, m_HeadPosition, out var head))
    14.             {
    15.                 // DGG: This should be configurable, based on individual phone placement.
    16.                 var hp = headPosition + new Vector3(0, 0, 0.35f);
    17.                 head.localPosition = continuous ? Vector3.Lerp(hp, head.localPosition, m_HeadSmoothing) : hp;
    18.             }
    19.         }
    However, it feels like the head position is completely dependent on the relative position of the phone and the head, and depending on where a user chooses to place their phone (ideally on a stand, or something static), this would is something you might expect to set on the mapper.

    Am I missing something here? Is there a way to use the Head Position on a model without doing this offset?
     
  3. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    4,212
    I think I've concluded that the Head Position is probably only intended for floating heads that aren't attached to a body, like what you see in a lot of demos. Or rather, that using the Default Face Mapper along with Head Position isn't a good combination for fully-rigged humanoid characters. But on the other hand, in those cases, you probably only want Head Rotation anyway.
     
  4. alfance

    alfance

    Joined:
    Jan 28, 2018
    Posts:
    3
    Hi Dgoyette, you are right. The head movement field can be finicky for fully rigged characters.
    It is recommended that you do not drive head movement directly.
    • If you are driving movement for an entire character using a full mocap setup, use the data from your body tracking solution to get head movement.
    • For best results, use a Constraint component from the Animation Rigging package to more realistically drive head movement accounting for neck bone rotation.