Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Read bones transforms before IK corrections with Animation Rigging

Discussion in 'Animation Rigging' started by reaven_code, Jan 22, 2020.

  1. reaven_code

    reaven_code

    Joined:
    Dec 10, 2015
    Posts:
    2
    How can I read the bones transform values before the ik correction is applied with the new Animation Rigging Package?
    The standard method, using only the animator, consists to read the transforms within the OnAnimatorIK callback and then apply any ik corrections, with Animation Rigging this doesn't seems to work, it returns the transform (position and rotation) of the ik target instead of the bone transform before ik corrections.
     
    Last edited: Jan 22, 2020
  2. simonbz

    simonbz

    Unity Technologies

    Joined:
    Sep 28, 2015
    Posts:
    295
    Hi!

    The OnAnimatorIK callback is used to handle IK for characters using humanoid. This is not compatible with AnimationRigging constraints.

    To do some pre-process work before TwoBoneIK or to just extract a position at any point during contraints evaluation, I suggest writing a custom constraint.

    You can have a look at our SIGGRAPH workshop if you need some help getting started: https://github.com/Unity-Technologies/animation-rigging-workshop-siggraph2019. There are several examples of custom constraints in the project (e.g. HelloWorld.cs) and the workshop PDF explains how constraints are built.
     
  3. reaven_code

    reaven_code

    Joined:
    Dec 10, 2015
    Posts:
    2
    Oh thank you!
     
  4. Catsoft-Studios

    Catsoft-Studios

    Joined:
    Jan 15, 2011
    Posts:
    708
    Sorry for necroing this thread, but I've been working on a problem related to this and I've come up with a solution I wanted to share.

    I'm trying to make a foot-align IK system that automatically places the feet based on the distance left to the floor. Using the OnAnimatorIK() method is pretty straightforward, since you can get to read the bone position *before* any IK process, so you can easily adjust the position and rotation of each foot.

    However, as @simonbz mentioned, this is not possible with the Animation Rigging package. I've created a new "Rig Constraint" called ExtractTransformConstraint that basically extracts the position and rotation of a bone before any other constraints are applied.

    Here's the 4 code pieces to make it work.

    ExtractTransformConstraint.cs: Define a component that can be attached to any child of a Rig component:

    Code (CSharp):
    1. [DisallowMultipleComponent]
    2. [AddComponentMenu("Animation Rigging/Extract Transform Constraint")]
    3.  
    4. public class ExtractTransformConstraint : RigConstraint<
    5.     ExtractTransformConstraintJob,
    6.     ExtractTransformConstraintData,
    7.     ExtractTransformConstraintJobBinder>
    8. {
    9.    
    10. }
    ExtractTransformConstraintData: This is the data struct where you basically query the position and rotation outputted by the animation job.

    Code (CSharp):
    1. [Serializable]
    2. public struct ExtractTransformConstraintData : IAnimationJobData
    3. {
    4.     [SyncSceneToStream] public Transform bone;
    5.  
    6.     public Vector3 position;
    7.     public Quaternion rotation;
    8.  
    9.     public bool IsValid()
    10.     {
    11.         return bone != null;
    12.     }
    13.  
    14.     public void SetDefaultValues()
    15.     {
    16.         this.bone = null;
    17.        
    18.         this.position = Vector3.zero;
    19.         this.rotation = Quaternion.identity;
    20.     }
    21. }
    ExtractTransformConstraintJob: The animation job. All this does is to let through any animation stream, but captures the position and rotation of the chosen bone. It's important to note that the weight property has no effect.

    Code (CSharp):
    1. public struct ExtractTransformConstraintJob : IWeightedAnimationJob
    2. {
    3.     public ReadWriteTransformHandle bone;
    4.  
    5.     public FloatProperty jobWeight { get; set; }
    6.    
    7.     public Vector3Property position;
    8.     public Vector4Property rotation;
    9.    
    10.     public void ProcessRootMotion(AnimationStream stream)
    11.     { }
    12.    
    13.     public void ProcessAnimation(AnimationStream stream)
    14.     {
    15.         AnimationRuntimeUtils.PassThrough(stream, this.bone);
    16.  
    17.         Vector3 pos = this.bone.GetPosition(stream);
    18.         Quaternion rot = this.bone.GetRotation(stream);
    19.        
    20.         this.position.Set(stream, pos);
    21.         this.rotation.Set(stream, new Vector4(rot.x, rot.y, rot.z, rot.w));
    22.     }
    23. }
    ExtractTransformConstraintJobBinder: And finally the job binder t̶o̶ ̶r̶u̶l̶e̶ ̶t̶h̶e̶m̶ ̶o̶f̶ ̶a̶n̶d̶ ̶i̶n̶ ̶t̶h̶e̶ ̶d̶a̶r̶k̶n̶e̶s̶s̶ ̶b̶i̶n̶d̶ ̶t̶h̶e̶m̶ to link the data and the job.

    Code (CSharp):
    1. public class ExtractTransformConstraintJobBinder : AnimationJobBinder<
    2.     ExtractTransformConstraintJob,
    3.     ExtractTransformConstraintData>
    4. {
    5.     public override ExtractTransformConstraintJob Create(Animator animator,
    6.         ref ExtractTransformConstraintData data, Component component)
    7.     {
    8.         return new ExtractTransformConstraintJob
    9.         {
    10.             bone = ReadWriteTransformHandle.Bind(animator, data.bone),
    11.             position = Vector3Property.Bind(animator, component, "m_Data." + nameof(data.position)),
    12.             rotation = Vector4Property.Bind(animator, component, "m_Data." + nameof(data.rotation))
    13.         };
    14.     }
    15.  
    16.     public override void Destroy(ExtractTransformConstraintJob job)
    17.     { }
    18. }
    How to use:

    Similar to adding a Two Bone Constraint (or any other), you'll see there's a new "Extract Transform Constraint" component under the Animation Rigging component menu.

    Simply add this before any other constraint. In order to get the position and rotation extracted by the constraint, you can do:

    Code (CSharp):
    1. // reference to the constraint:
    2. public ExtractTransformConstraint constraint;
    3.  
    4. void Update()
    5. {
    6.     Debug.Log("Position: " + this.constraint.data.position);
    7.     Debug.Log("Rotation: " + this.constraint.data.rotation);
    8. }
    PS: Not sure if this is the best way to tackle this problem. Any suggestions or ideas are welcome.

    PSS: One thing I'm missing from the Animation Rigging package is that there's no QuaternionProperty class. There's the Vector4Property, which is basically the same, but makes it a bit cumbersome to convert all Quaternion (x,y,z,w) fields to Vector4 (x,y,z,w). Could a QuaternionProperty be added in the future?
     
  5. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
     
    Catsoft-Studios likes this.
  6. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
    I have spent some time implementing some custom Constraints and as a coder it is really interesting what you can do with the Animation Rigging Package. Very nice indeed.
    Catsoft-Studios, your code is perfect to start playing for a newcomer. I suggest you add the initial includes and maybe also the burst reference.
    I agree with you that Vector4 vs Quaternion is somehow strange, as the subtle difference is only in normalization and in general most people have no understanding of Hamilton's math theory and complex numbers in general.
    I am curios about your usage of AnimationRuntimeUtils.PassThrough, something never used in Unity examples.
    As you are not doing any modifications settings in bone but only in local environment why did you feel the need to use PassThrough. Just out of curiosity to understand it. Thanks.

    As you seem pretty knowledgeable on the matter, have you also been using the Playable Graph Visualizer, now pretty difficult to find (used github to import it). I do not completely understand its behaviour in some cases.

    Last: I am now using a Malberts Animal Controller for my cat and other quads. I also have tested most of the character controllers around (KCC, Opsive, the new Unity Standard Character Controller, and others) but in general they still mostly rely on humanoids for feet IK.
    Could you elaborate a little on your experience on this issue?
    thanks for your contributions.

    sergio from Italy
     
  7. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
    As you are the author of Game Creator I was wandering if your interest in non humanoid (generic) feet IK means there will be good support for them in GC.

    Frankly I never got GC as visual programming is not very appealing for coders like me, but as you are one of the few around working in a serious way on Animation Rigging I will check it out immediately.
    Let me know about your further developments. I am personally focused on realtime facial expression - lip sync of my characters via Animation Rigging.
     
    Last edited: Sep 28, 2020
  8. Catsoft-Studios

    Catsoft-Studios

    Joined:
    Jan 15, 2011
    Posts:
    708
    I'm just experimenting and getting the hang of the Animation Rigging package, really. Basically getting used to the rigging package, which I believe it's super duper awesome.

    The only things I'm missing is accessing the unprocessed position of each bone before the IK pass. Working with non-humanoids should add little complexity, unless you want to tackle more complex things like, shifting the center of mass of the non-humanoid based on how steep and the orientation of the terrain vs the character.

    I think I took the idea from this post. PassThrough basically relays all the motion information from that rig layer to the next one, without modifying it (I'm pretty sure, but not 100%, so make sure to check the documentation!).

    Hope this helps!
     
  9. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
    I assume you are referring to humanoid-Unity-handled-IK. I believe unity is not going to stick with this traditional humanoid monolith as you can understand pretty clearly from last GDC speech by unity's Dave Hunt (
    ) on evolution of animation rigging workflows.
    upload_2020-9-28_19-41-20.png
     
  10. Catsoft-Studios

    Catsoft-Studios

    Joined:
    Jan 15, 2011
    Posts:
    708
    Nope, I was talking about my proposed solution for getting the unprocessed IK animation information (for the Animation Rigging package). The built-in humanoid IK system might be monolithic, but it works out of the box.

    With the Animation Rigging package, there's no way to get the unprocessed (unless you use my proposed solution from above, but it's a bit of a hack) bone location information. When you query a bone position to adjust its position using IK, there should be a way to get the raw bone's position and rotation. Instead, it returns the last frame's position, with IK applied.

    Regardless, I'm still experimenting with this package, so I may be mistaken and haven't been able to put more than a handful of hours into this. Once this issue is resolved (https://issuetracker.unity3d.com/is...by-several-units-when-using-animation-rigging) I'll continue testing this package.
     
  11. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
    Well, I am using your solution just for that, and it solves brilliantly the problem, for now. Of course this should be a basic feature, as you are asking Unity.
    Thanks
     
    Last edited: Sep 29, 2020
  12. giantkilleroverunity3d

    giantkilleroverunity3d

    Joined:
    Feb 28, 2014
    Posts:
    385
    I have been studying the Rigging portion of Unity IK. And I think I have a missing step problem. I have the Speed Ball player asset. It is a free Unity Store Asset. I would like to IK rig it. It is an FBX, no doubt done in Autodesk which I don't use. I import in Blender and see the model with what looks to be IK handles. I read where IK rigs dont transfer to Unity. I also have the Blender Rigify plugin to correct the transforms from Blender to Unity. If I have the GameObject parent child body structure in Unity, is this enough to forego Blender FBX(except for modeling and sculpture changes) to rig this model in Unity to get to IK rigging? Most of what efforts I have seen jump straight to animation which is final step after static posing. At this point I just want to pose/position by lets say pulling a hand and the rest of the model hierarchy responds accordingly.
    Thanks in advance.
     
  13. davehunt_unity

    davehunt_unity

    Unity Technologies

    Joined:
    Nov 13, 2017
    Posts:
    32
    Hello,

    I wanted to mention that this assumption is untrue. That section of my GDC presentation was talking about authoring rigs for animators to create new animation content. This says nothing about the future of Unity's runtime animation retargeting, that is a different thing entirely. I just wanted to add this clarification.
     
    Catsoft-Studios likes this.
  14. ProceduralCatMaker

    ProceduralCatMaker

    Joined:
    Mar 9, 2020
    Posts:
    108
    Glad to hear this from an authotitative source like you. Sorry to misinterpreter your thoughts.
    Still, your point on modular rigs versus monolithic has a deep rationale and I believe Unity is smart enough to think about it. Of course I did not mean that they will drop support for humanoid, just that the future in not as restricted as before to humanois for many features it has been offering.
    Thanks for your wise views, great work and also for this clarification.
     
  15. ClipzVFX

    ClipzVFX

    Joined:
    Aug 3, 2022
    Posts:
    2
    I have been trying to make this all day,with no luck.I am using a two bone IK constraint with it and I set the IK targets position and rotation to your constraints calculated position and rotation. The code has no errors but for some reason the target's rotation is going crazy. It keeps changing its rotation and it doesn't stop. Also the Ik target's position is not in the hand's animated position but in the default t-pose position.
    I hope you understand and are able to help me, thanks RandomHandRot.PNG Animation.PNG
     
  16. Xwad

    Xwad

    Joined:
    May 14, 2020
    Posts:
    41

    Hello! Could you please help me how to use your code? I tried to put them all together but i get 2 errors:

    "The type or namespace name 'SerializableAttribute' could not be found (are you missing a using directive or an assembly reference?) "

    "The type or namespace name 'Serializable' could not be found (are you missing a using directive or an assembly reference?)"

    The code:
    Code (CSharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEngine.Animations;
    6. using UnityEngine.Animations.Rigging;
    7. [DisallowMultipleComponent]
    8. [AddComponentMenu("Animation Rigging/Extract Transform Constraint")]
    9. public class ExtractTransformConstraint : RigConstraint<ExtractTransformConstraintJob,ExtractTransformConstraintData,ExtractTransformConstraintJobBinder>{
    10.  
    11. }
    12.  
    13. public class ExtractTransformConstraintJobBinder : AnimationJobBinder<ExtractTransformConstraintJob,ExtractTransformConstraintData>{
    14.     public override ExtractTransformConstraintJob Create(Animator animator,
    15.         ref ExtractTransformConstraintData data, Component component)
    16.     {
    17.         return new ExtractTransformConstraintJob
    18.         {
    19.             bone = ReadWriteTransformHandle.Bind(animator, data.bone),
    20.             position = Vector3Property.Bind(animator, component, "m_Data." + nameof(data.position)),
    21.             rotation = Vector4Property.Bind(animator, component, "m_Data." + nameof(data.rotation))
    22.         };
    23.     }
    24.  
    25.     public override void Destroy(ExtractTransformConstraintJob job)
    26.     { }
    27. }
    28.  
    29. [Serializable]
    30. public struct ExtractTransformConstraintData : IAnimationJobData{
    31.     [SyncSceneToStream] public Transform bone;
    32.  
    33.     public Vector3 position;
    34.     public Quaternion rotation;
    35.  
    36.     public bool IsValid()
    37.     {
    38.         return bone != null;
    39.     }
    40.  
    41.     public void SetDefaultValues()
    42.     {
    43.         this.bone = null;
    44.  
    45.         this.position = Vector3.zero;
    46.         this.rotation = Quaternion.identity;
    47.     }
    48. }
    49.  
    50. public struct ExtractTransformConstraintJob : IWeightedAnimationJob{
    51.     public ReadWriteTransformHandle bone;
    52.  
    53.     public FloatProperty jobWeight { get; set; }
    54.  
    55.     public Vector3Property position;
    56.     public Vector4Property rotation;
    57.  
    58.     public void ProcessRootMotion(AnimationStream stream)
    59.     { }
    60.  
    61.     public void ProcessAnimation(AnimationStream stream)
    62.     {
    63.         AnimationRuntimeUtils.PassThrough(stream, this.bone);
    64.  
    65.         Vector3 pos = this.bone.GetPosition(stream);
    66.         Quaternion rot = this.bone.GetRotation(stream);
    67.  
    68.         this.position.Set(stream, pos);
    69.         this.rotation.Set(stream, new Vector4(rot.x, rot.y, rot.z, rot.w));
    70.     }
    71. }
     
  17. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    1,012
    Thank you for this it worked perfectly!

    I needed this to basically make it so if the torso moved the hands wouldn't move from their already animated position. And I wanted to avoid having to copy animations to a whole secondary rig.
     
  18. Xwad

    Xwad

    Joined:
    May 14, 2020
    Posts:
    41

    Hello! How did you do it? Did you put all code snippets into the ExtractTransformConstraint.cs?
    It gives a lot of errors for me.
     
  19. sacb0y

    sacb0y

    Joined:
    May 9, 2016
    Posts:
    1,012
    No, i put them in separate individual files.

    You can resolve the name space issues in VS or Jetbrains by right clicking them and seeing suggestions.
     
  20. Xwad

    Xwad

    Joined:
    May 14, 2020
    Posts:
    41
    Thanks! What confused me was that only ExtractTransformConstraint has .cs at the end.
     
  21. HitsuSan

    HitsuSan

    Joined:
    Sep 28, 2014
    Posts:
    158

    This just saved me so much headache and time... Thank you ^^