Search Unity

  1. Full schedule for #UniteBerlin is now available! Featuring talks on our roadmap, hands-on labs and much more! Check it out!
    Dismiss Notice
  2. Unity 2018.1 has arrived! Read about it here
    Dismiss Notice
  3. Scriptable Render Pipeline improvements, Texture Mipmap Streaming, and more! Check out what we have in store for you in the 2018.2 Beta.
    Dismiss Notice
  4. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  5. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  6. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Fuse motion capture and eye tracking into joint HumanPose

Discussion in 'Animation' started by PKullmann, May 9, 2018.

  1. PKullmann

    PKullmann

    Joined:
    Apr 12, 2017
    Posts:
    2
    I want to retarget both tracked eye movements and body motion to one humanoid avatar.

    The motion capture system's plugin provides a stream of updates to a HumanPose which I merely copy to the Player Prefab via HumanPoseHandler.SetHumanPose within my retargeting script's Update method.

    The eye tracking device provides the current fixation point within the scene. To use this position, I decided to add an empty AnimatorController with IK pass enabled. In OnAnimatorIK I use the eye tracker's fixation position as the Animator LookAtPosition with the LookAtWeights set to only move the eyes. This works like a charm, but unfortunately completely overwrites the HumanPose set previously in the Update method resulting in a character.

    Is there any way to use Mecanim's built-in IK system without overwriting the motion capture input?
     
    Last edited: May 9, 2018
  2. theANMATOR2b

    theANMATOR2b

    Joined:
    Jul 12, 2014
    Posts:
    7,679
    Does adding this to an animation layer - set to additive - solve the overwrite problem?
    Base layer mocap, new additive layer eye tracking data.
     
  3. PKullmann

    PKullmann

    Joined:
    Apr 12, 2017
    Posts:
    2
    Whereas body motion is set in the Update() method by copying muscle values from an invisible character that receives the motion capture data stream, the Animator Controller is only used to set the avatar LookAtPosition in OnAnimatorIK(). Neither using AnimationBlendMode to "Additve" nor using an AvatarMask prevents overriding the animation.

    Previously, I manually converted the gaze vectors to muscle values which felt like an overly bloated solution. I've started experimenting with the AimConstraint recently added to Unity and might get away with that.
    I'll keep you updated, thanks for the suggestion!
     
    theANMATOR2b likes this.