Recording Humanoid Animations with Foot IK

I am attempting to set up a humanoid animation recorder to save animations from a motion capture system within Unity. I have successfully set up the recorder so that it can save animation clips with all the muscle space data, but am running into problems with foot IK. Every time I have tried recording the feet position/orientation in muscle space the results are not what I expect (wrong position, wrong orientation, etc...). Disabling foot-ik on the animation fixes the issue, but means it cannot be used with timeline (since timeline appears to force foot-ik on).

The only information I've been able to find regarding encoding the feet positions and orientations is from this Unity blog post: https://blogs.unity3d.com/2014/05/26/mecanim-humanoids/
In that post it says "The hands and feet position and orientation are normalized relative to Humanoid Root (center of mass, average body rotation and humanoid scale) in the Muscle Space."

I've tried recording the feet positions and orientations relative to the Animator bodyPosition and bodyRotation without success. I've also attempted recording relative to the root transform of the body. So I'm reaching out to see if anyone has found any further documentation regarding how the feet position and orientation is transformed to make it normalized relative to Humanoid Root. Ideally I’d really appreciate a short example script that shows a transform for the position/orientation of a foot being normalized relative to humanoid root in the muscle space, but any help is much appreciated.

 

Hi @jacques_f

take a look at this script, it should give you all the info to compute the hands and feet position relative to animator body position and rotation.

Code (CSharp):

1.  
2. using System.Collections;
3. using System.Collections.Generic;
4. using UnityEngine;
5. using System;
6. using System.Reflection;
7. public class TQ
8. {
9.     public TQ(Vector3 translation, Quaternion rotation)
10.     {
11.         t = translation;
12.         q = rotation;
13.     }
14.     public Vector3 t;
15.     public Quaternion q;
16.     // Scale should always be 1,1,1
17. }
18. public class AvatarUtility {
19.     static public TQ GetIKGoalTQ(Avatar avatar, float humanScale, AvatarIKGoal avatarIKGoal, TQ animatorBodyPositionRotation, TQ skeletonTQ)
20.     {
21.         int humanId = (int)HumanIDFromAvatarIKGoal(avatarIKGoal);
22.         if(humanId == (int)HumanBodyBones.LastBone)
23.             throw new InvalidOperationException("Invalid human id.");
24.         MethodInfo methodGetAxisLength = typeof(Avatar).GetMethod("GetAxisLength", BindingFlags.Instance | BindingFlags.NonPublic);
25.         if(methodGetAxisLength == null)
26.             throw new InvalidOperationException("Cannot find GetAxisLength method.");
27.         MethodInfo methodGetPostRotation = typeof(Avatar).GetMethod("GetPostRotation", BindingFlags.Instance | BindingFlags.NonPublic);
28.         if(methodGetPostRotation == null)
29.             throw new InvalidOperationException("Cannot find GetPostRotation method.");
30.         Quaternion postRotation = (Quaternion)methodGetPostRotation.Invoke(avatar, new object[]{humanId});
31.         var goalTQ = new TQ(skeletonTQ.t, skeletonTQ.q * postRotation);
32.         if(avatarIKGoal == AvatarIKGoal.LeftFoot || avatarIKGoal == AvatarIKGoal.RightFoot)
33.         {
34.             // Here you could use animator.leftFeetBottomHeight or animator.rightFeetBottomHeight rather than GetAxisLenght
35.             // Both are equivalent but GetAxisLength is the generic way and work for all human bone
36.             float axislength = (float) methodGetAxisLength.Invoke(avatar, new object[]{humanId});
37.             Vector3 footBottom = new Vector3(axislength, 0, 0);
38.             goalTQ.t += (goalTQ.q * footBottom);
39.         }
40.         // IK goal are in avatar body local space
41.         Quaternion invRootQ = Quaternion.Inverse(animatorBodyPositionRotation.q);
42.         goalTQ.t = invRootQ * (goalTQ.t - animatorBodyPositionRotation.t);
43.         goalTQ.q = invRootQ * goalTQ.q;
44.         goalTQ.t /= humanScale;
45.      
46.         return goalTQ;
47.     }
48.     static public HumanBodyBones HumanIDFromAvatarIKGoal(AvatarIKGoal avatarIKGoal)
49.     {
50.         HumanBodyBones humanId = HumanBodyBones.LastBone;
51.         switch(avatarIKGoal)
52.         {
53.             case AvatarIKGoal.LeftFoot: humanId = HumanBodyBones.LeftFoot; break;
54.             case AvatarIKGoal.RightFoot: humanId = HumanBodyBones.RightFoot; break;
55.             case AvatarIKGoal.LeftHand: humanId = HumanBodyBones.LeftHand; break;
56.             case AvatarIKGoal.RightHand: humanId = HumanBodyBones.RightHand; break;
57.         }
58.         return humanId;
59.     }
60. }

 

Hi @Mecanim-Dev
we're attempting to use this script for the same purpose, as above, but our resultant positions are off.
The closest result we have is using HumanPose.bodyPosition and Rotation, and world-space bone positions of the feet.
It seems when our character is closer to the origin the error is smaller.
We'd appreciate a code example or some help related to the snippet you posted.

The reason we are also in need of this is the same too, because Timeline forces foot IK to be enabled, so our workaround is to record the feet positions into the IK. It seems like that should not be the default for Timeline animation tracks perhaps?

 

If you're on 2018.2 and later you can also use the new animation jobs to extract ik goal position/rotation.

https://docs.unity3d.com/ScriptRefe...ationHumanStream.GetGoalPositionFromPose.html
https://docs.unity3d.com/ScriptRefe...ationHumanStream.GetGoalRotationFromPose.html

those function will return the IK goal global position and rotation.
IK goal in the animation clip are stored into body's local space.

So you will also need to retrieve those values
https://docs.unity3d.com/ScriptRefe...ations.AnimationHumanStream-bodyPosition.html
https://docs.unity3d.com/ScriptRefe...ations.AnimationHumanStream-bodyRotation.html

here what it could look like

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.Playables;
5. using UnityEngine.Animations;
6. using UnityEngine.Experimental.Animations;
7.  
8. public class FootIK : MonoBehaviour
9. {
10.     public struct GetGoalPositionAndRotationJob : IAnimationJob
11.     {
12.         public Vector3 leftFootGoalFromPosePosition;
13.         public Quaternion leftFootGoalFromPoseRotation;
14.         public Vector3 rightFootGoalFromPosePosition;
15.         public Quaternion rightFootGoalFromPoseRotation;
16.         public Vector3 leftHandGoalFromPosePosition;
17.         public Quaternion leftHandGoalFromPoseRotation;
18.         public Vector3 rightHandGoalFromPosePosition;
19.         public Quaternion rightHandGoalFromPoseRotation;
20.  
21.         public Vector3 bodyPosition;
22.         public Quaternion bodyRotation;
23.  
24.         public void ProcessRootMotion(AnimationStream stream) {}
25.         public void ProcessAnimation(AnimationStream stream)
26.         {
27.             var humanStream = stream.AsHuman();
28.  
29.             leftFootGoalFromPosePosition = humanStream.GetGoalPositionFromPose(AvatarIKGoal.LeftFoot);
30.             leftFootGoalFromPoseRotation = humanStream.GetGoalRotationFromPose(AvatarIKGoal.LeftFoot);
31.             rightFootGoalFromPosePosition = humanStream.GetGoalPositionFromPose(AvatarIKGoal.RightFoot);
32.             rightFootGoalFromPoseRotation = humanStream.GetGoalRotationFromPose(AvatarIKGoal.RightFoot);
33.             leftHandGoalFromPosePosition = humanStream.GetGoalPositionFromPose(AvatarIKGoal.LeftHand);
34.             leftHandGoalFromPoseRotation = humanStream.GetGoalRotationFromPose(AvatarIKGoal.LeftHand);
35.             rightHandGoalFromPosePosition = humanStream.GetGoalPositionFromPose(AvatarIKGoal.RightHand);
36.             rightHandGoalFromPoseRotation = humanStream.GetGoalRotationFromPose(AvatarIKGoal.RightHand);
37.  
38.             bodyPosition = humanStream.bodyPosition;
39.             bodyRotation = humanStream.bodyRotation;
40.         }
41.     }
42.  
43.  
44.     public AnimationClip animationClip;
45.    
46.     void BakeIKGoal()
47.     {
48.         var animator = GetComponent<Animator>();
49.  
50.         var graph = PlayableGraph.Create();
51.         graph.SetTimeUpdateMode(DirectorUpdateMode.Manual);
52.         var clipPlayable = AnimationClipPlayable.Create(graph, animationClip);
53.  
54.         var job = new GetGoalPositionAndRotationJob();
55.         var scriptPlayable = AnimationScriptPlayable.Create(graph, job, 1);
56.  
57.         scriptPlayable.ConnectInput(0, clipPlayable, 0, 1.0f);
58.  
59.         var output = AnimationPlayableOutput.Create(graph, "output", animator);
60.         output.SetSourcePlayable(scriptPlayable);
61.  
62.         var frameRate = animationClip.frameRate;
63.         var step = 1.0f / frameRate;
64.         var length = animationClip.length;
65.  
66.         for (float time = 0.0f; time < length; time += step)
67.         {
68.             graph.Evaluate(time == 0.0f ? 0.0f : step);
69.  
70.             job = scriptPlayable.GetJobData<GetGoalPositionAndRotationJob>();
71.  
72.              // IK goal are in avatar body local space
73.             Quaternion invRootQ = Quaternion.Inverse(job.bodyRotation);
74.             Vector3 goalPosition = (invRootQ * (job.leftFootGoalFromPosePosition - job.bodyPosition)) / animator.humanScale;
75.             Quaternion goalRotation = invRootQ * job.leftFootGoalFromPoseRotation;
76.         }
77.  
78.         graph.Destroy();
79.  
80.     }
81. }

 

This may not be your wheelhouse, but we're finding GameObjectRecorder is now recording with less accuracy (more keyframe compression) on these new curves. It'd be nice if in addition to controlling FootIK on Animation TimelineClips to have more control over the GameObjectRecorder.

 

The footQ is absolutely not relative to the body rotation. Say you have an "idle" animation with foot pointing forward as the body rotation, then the footQ would be something near Quaternion.Identiy, but it's not!!. So what's the hack behind it?