Search Unity

  1. Get all the Unite Berlin 2018 news on the blog.
    Dismiss Notice
  2. Unity 2018.2 has arrived! Read about it here.
    Dismiss Notice
  3. We're looking for your feedback on the platforms you use and how you use them. Let us know!
    Dismiss Notice
  4. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  5. Improve your Unity skills with a certified instructor in a private, interactive classroom. Learn more.
    Dismiss Notice
  6. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  7. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  8. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Fuse motion capture and eye tracking into joint HumanPose

Discussion in 'Animation' started by Peter-K, May 9, 2018.

  1. Peter-K

    Peter-K

    Joined:
    Apr 12, 2017
    Posts:
    2
    I want to retarget both tracked eye movements and body motion to one humanoid avatar.

    The motion capture system's plugin provides a stream of updates to a HumanPose which I merely copy to the Player Prefab via HumanPoseHandler.SetHumanPose within my retargeting script's Update method.

    The eye tracking device provides the current fixation point within the scene. To use this position, I decided to add an empty AnimatorController with IK pass enabled. In OnAnimatorIK I use the eye tracker's fixation position as the Animator LookAtPosition with the LookAtWeights set to only move the eyes. This works like a charm, but unfortunately completely overwrites the HumanPose set previously in the Update method resulting in the character being in the fetus-like pose just like when setting up a humanoid character in the Editor.

    Is there any way to use Mecanim's built-in IK system without overwriting the motion capture input?
     
    Last edited: Jul 23, 2018
  2. theANMATOR2b

    theANMATOR2b

    Joined:
    Jul 12, 2014
    Posts:
    7,773
    Does adding this to an animation layer - set to additive - solve the overwrite problem?
    Base layer mocap, new additive layer eye tracking data.
     
  3. Peter-K

    Peter-K

    Joined:
    Apr 12, 2017
    Posts:
    2
    Whereas body motion is set in the Update() method by copying muscle values from an invisible character that receives the motion capture data stream, the Animator Controller is only used to set the avatar LookAtPosition in OnAnimatorIK(). Neither using AnimationBlendMode to "Additve" nor using an AvatarMask prevents overriding the animation.

    Previously, I manually converted the gaze vectors to muscle values which felt like an overly bloated solution. I've started experimenting with the AimConstraint recently added to Unity and might get away with that.
    I'll keep you updated, thanks for the suggestion!
     
    theANMATOR2b likes this.