Read bones transforms before IK corrections with Animation Rigging

How can I read the bones transform values before the ik correction is applied with the new Animation Rigging Package?
The standard method, using only the animator, consists to read the transforms within the OnAnimatorIK callback and then apply any ik corrections, with Animation Rigging this doesn't seems to work, it returns the transform (position and rotation) of the ik target instead of the bone transform before ik corrections.

 

Oh thank you!

 

Sorry for necroing this thread, but I've been working on a problem related to this and I've come up with a solution I wanted to share.

I'm trying to make a foot-align IK system that automatically places the feet based on the distance left to the floor. Using the OnAnimatorIK() method is pretty straightforward, since you can get to read the bone position *before* any IK process, so you can easily adjust the position and rotation of each foot.

However, as @simonbz mentioned, this is not possible with the Animation Rigging package. I've created a new "Rig Constraint" called ExtractTransformConstraint that basically extracts the position and rotation of a bone before any other constraints are applied.

Here's the 4 code pieces to make it work.

ExtractTransformConstraint.cs: Define a component that can be attached to any child of a Rig component:

Code (CSharp):

1. [DisallowMultipleComponent]
2. [AddComponentMenu("Animation Rigging/Extract Transform Constraint")]
3.  
4. public class ExtractTransformConstraint : RigConstraint<
5.     ExtractTransformConstraintJob,
6.     ExtractTransformConstraintData,
7.     ExtractTransformConstraintJobBinder>
8. {
9.    
10. }

ExtractTransformConstraintData: This is the data struct where you basically query the position and rotation outputted by the animation job.

Code (CSharp):

1. [Serializable]
2. public struct ExtractTransformConstraintData : IAnimationJobData
3. {
4.     [SyncSceneToStream] public Transform bone;
5.  
6.     public Vector3 position;
7.     public Quaternion rotation;
8.  
9.     public bool IsValid()
10.     {
11.         return bone != null;
12.     }
13.  
14.     public void SetDefaultValues()
15.     {
16.         this.bone = null;
17.        
18.         this.position = Vector3.zero;
19.         this.rotation = Quaternion.identity;
20.     }
21. }

ExtractTransformConstraintJob: The animation job. All this does is to let through any animation stream, but captures the position and rotation of the chosen bone. It's important to note that the weight property has no effect.

Code (CSharp):

1. public struct ExtractTransformConstraintJob : IWeightedAnimationJob
2. {
3.     public ReadWriteTransformHandle bone;
4.  
5.     public FloatProperty jobWeight { get; set; }
6.    
7.     public Vector3Property position;
8.     public Vector4Property rotation;
9.    
10.     public void ProcessRootMotion(AnimationStream stream)
11.     { }
12.    
13.     public void ProcessAnimation(AnimationStream stream)
14.     {
15.         AnimationRuntimeUtils.PassThrough(stream, this.bone);
16.  
17.         Vector3 pos = this.bone.GetPosition(stream);
18.         Quaternion rot = this.bone.GetRotation(stream);
19.        
20.         this.position.Set(stream, pos);
21.         this.rotation.Set(stream, new Vector4(rot.x, rot.y, rot.z, rot.w));
22.     }
23. }

ExtractTransformConstraintJobBinder: And finally the job binder t̶o̶ ̶r̶u̶l̶e̶ ̶t̶h̶e̶m̶ ̶o̶f̶ ̶a̶n̶d̶ ̶i̶n̶ ̶t̶h̶e̶ ̶d̶a̶r̶k̶n̶e̶s̶s̶ ̶b̶i̶n̶d̶ ̶t̶h̶e̶m̶ to link the data and the job.

Code (CSharp):

1. public class ExtractTransformConstraintJobBinder : AnimationJobBinder<
2.     ExtractTransformConstraintJob,
3.     ExtractTransformConstraintData>
4. {
5.     public override ExtractTransformConstraintJob Create(Animator animator,
6.         ref ExtractTransformConstraintData data, Component component)
7.     {
8.         return new ExtractTransformConstraintJob
9.         {
10.             bone = ReadWriteTransformHandle.Bind(animator, data.bone),
11.             position = Vector3Property.Bind(animator, component, "m_Data." + nameof(data.position)),
12.             rotation = Vector4Property.Bind(animator, component, "m_Data." + nameof(data.rotation))
13.         };
14.     }
15.  
16.     public override void Destroy(ExtractTransformConstraintJob job)
17.     { }
18. }

How to use:

Similar to adding a Two Bone Constraint (or any other), you'll see there's a new "Extract Transform Constraint" component under the Animation Rigging component menu.

Simply add this before any other constraint. In order to get the position and rotation extracted by the constraint, you can do:

Code (CSharp):

1. // reference to the constraint:
2. public ExtractTransformConstraint constraint;
3.  
4. void Update()
5. {
6.     Debug.Log("Position: " + this.constraint.data.position);
7.     Debug.Log("Rotation: " + this.constraint.data.rotation);
8. }

PS: Not sure if this is the best way to tackle this problem. Any suggestions or ideas are welcome.

PSS: One thing I'm missing from the Animation Rigging package is that there's no QuaternionProperty class. There's the Vector4Property, which is basically the same, but makes it a bit cumbersome to convert all Quaternion (x,y,z,w) fields to Vector4 (x,y,z,w). Could a QuaternionProperty be added in the future?

 

I have spent some time implementing some custom Constraints and as a coder it is really interesting what you can do with the Animation Rigging Package. Very nice indeed.
Catsoft-Studios, your code is perfect to start playing for a newcomer. I suggest you add the initial includes and maybe also the burst reference.
I agree with you that Vector4 vs Quaternion is somehow strange, as the subtle difference is only in normalization and in general most people have no understanding of Hamilton's math theory and complex numbers in general.
I am curios about your usage of AnimationRuntimeUtils.PassThrough, something never used in Unity examples.
As you are not doing any modifications settings in bone but only in local environment why did you feel the need to use PassThrough. Just out of curiosity to understand it. Thanks.

As you seem pretty knowledgeable on the matter, have you also been using the Playable Graph Visualizer, now pretty difficult to find (used github to import it). I do not completely understand its behaviour in some cases.

Last: I am now using a Malberts Animal Controller for my cat and other quads. I also have tested most of the character controllers around (KCC, Opsive, the new Unity Standard Character Controller, and others) but in general they still mostly rely on humanoids for feet IK.
Could you elaborate a little on your experience on this issue?
thanks for your contributions.

sergio from Italy

 

As you are the author of Game Creator I was wandering if your interest in non humanoid (generic) feet IK means there will be good support for them in GC.

Frankly I never got GC as visual programming is not very appealing for coders like me, but as you are one of the few around working in a serious way on Animation Rigging I will check it out immediately.
Let me know about your further developments. I am personally focused on realtime facial expression - lip sync of my characters via Animation Rigging.

 

I'm just experimenting and getting the hang of the Animation Rigging package, really. Basically getting used to the rigging package, which I believe it's super duper awesome.

The only things I'm missing is accessing the unprocessed position of each bone before the IK pass. Working with non-humanoids should add little complexity, unless you want to tackle more complex things like, shifting the center of mass of the non-humanoid based on how steep and the orientation of the terrain vs the character.

I think I took the idea from this post. PassThrough basically relays all the motion information from that rig layer to the next one, without modifying it (I'm pretty sure, but not 100%, so make sure to check the documentation!).

Hope this helps!

 

I assume you are referring to humanoid-Unity-handled-IK. I believe unity is not going to stick with this traditional humanoid monolith as you can understand pretty clearly from last GDC speech by unity's Dave Hunt (

) on evolution of animation rigging workflows.

 

Nope, I was talking about my proposed solution for getting the unprocessed IK animation information (for the Animation Rigging package). The built-in humanoid IK system might be monolithic, but it works out of the box.

With the Animation Rigging package, there's no way to get the unprocessed (unless you use my proposed solution from above, but it's a bit of a hack) bone location information. When you query a bone position to adjust its position using IK, there should be a way to get the raw bone's position and rotation. Instead, it returns the last frame's position, with IK applied.

Regardless, I'm still experimenting with this package, so I may be mistaken and haven't been able to put more than a handful of hours into this. Once this issue is resolved (https://issuetracker.unity3d.com/is...by-several-units-when-using-animation-rigging) I'll continue testing this package.

 

Well, I am using your solution just for that, and it solves brilliantly the problem, for now. Of course this should be a basic feature, as you are asking Unity.
Thanks

 

I have been studying the Rigging portion of Unity IK. And I think I have a missing step problem. I have the Speed Ball player asset. It is a free Unity Store Asset. I would like to IK rig it. It is an FBX, no doubt done in Autodesk which I don't use. I import in Blender and see the model with what looks to be IK handles. I read where IK rigs dont transfer to Unity. I also have the Blender Rigify plugin to correct the transforms from Blender to Unity. If I have the GameObject parent child body structure in Unity, is this enough to forego Blender FBX(except for modeling and sculpture changes) to rig this model in Unity to get to IK rigging? Most of what efforts I have seen jump straight to animation which is final step after static posing. At this point I just want to pose/position by lets say pulling a hand and the rest of the model hierarchy responds accordingly.
Thanks in advance.

 

Glad to hear this from an authotitative source like you. Sorry to misinterpreter your thoughts.
Still, your point on modular rigs versus monolithic has a deep rationale and I believe Unity is smart enough to think about it. Of course I did not mean that they will drop support for humanoid, just that the future in not as restricted as before to humanois for many features it has been offering.
Thanks for your wise views, great work and also for this clarification.

 

I have been trying to make this all day,with no luck.I am using a two bone IK constraint with it and I set the IK targets position and rotation to your constraints calculated position and rotation. The code has no errors but for some reason the target's rotation is going crazy. It keeps changing its rotation and it doesn't stop. Also the Ik target's position is not in the hand's animated position but in the default t-pose position.
I hope you understand and are able to help me, thanks

 

Hello! Could you please help me how to use your code? I tried to put them all together but i get 2 errors:

"The type or namespace name 'SerializableAttribute' could not be found (are you missing a using directive or an assembly reference?) "

"The type or namespace name 'Serializable' could not be found (are you missing a using directive or an assembly reference?)"

The code:

Code (CSharp):

1.  
2. using System.Collections;
3. using System.Collections.Generic;
4. using UnityEngine;
5. using UnityEngine.Animations;
6. using UnityEngine.Animations.Rigging;
7. [DisallowMultipleComponent]
8. [AddComponentMenu("Animation Rigging/Extract Transform Constraint")]
9. public class ExtractTransformConstraint : RigConstraint<ExtractTransformConstraintJob,ExtractTransformConstraintData,ExtractTransformConstraintJobBinder>{
10.  
11. }
12.  
13. public class ExtractTransformConstraintJobBinder : AnimationJobBinder<ExtractTransformConstraintJob,ExtractTransformConstraintData>{
14.     public override ExtractTransformConstraintJob Create(Animator animator,
15.         ref ExtractTransformConstraintData data, Component component)
16.     {
17.         return new ExtractTransformConstraintJob
18.         {
19.             bone = ReadWriteTransformHandle.Bind(animator, data.bone),
20.             position = Vector3Property.Bind(animator, component, "m_Data." + nameof(data.position)),
21.             rotation = Vector4Property.Bind(animator, component, "m_Data." + nameof(data.rotation))
22.         };
23.     }
24.  
25.     public override void Destroy(ExtractTransformConstraintJob job)
26.     { }
27. }
28.  
29. [Serializable]
30. public struct ExtractTransformConstraintData : IAnimationJobData{
31.     [SyncSceneToStream] public Transform bone;
32.  
33.     public Vector3 position;
34.     public Quaternion rotation;
35.  
36.     public bool IsValid()
37.     {
38.         return bone != null;
39.     }
40.  
41.     public void SetDefaultValues()
42.     {
43.         this.bone = null;
44.  
45.         this.position = Vector3.zero;
46.         this.rotation = Quaternion.identity;
47.     }
48. }
49.  
50. public struct ExtractTransformConstraintJob : IWeightedAnimationJob{
51.     public ReadWriteTransformHandle bone;
52.  
53.     public FloatProperty jobWeight { get; set; }
54.  
55.     public Vector3Property position;
56.     public Vector4Property rotation;
57.  
58.     public void ProcessRootMotion(AnimationStream stream)
59.     { }
60.  
61.     public void ProcessAnimation(AnimationStream stream)
62.     {
63.         AnimationRuntimeUtils.PassThrough(stream, this.bone);
64.  
65.         Vector3 pos = this.bone.GetPosition(stream);
66.         Quaternion rot = this.bone.GetRotation(stream);
67.  
68.         this.position.Set(stream, pos);
69.         this.rotation.Set(stream, new Vector4(rot.x, rot.y, rot.z, rot.w));
70.     }
71. }