How to change muscles values from code? (animate character with code)

Hi,
1.I would like to animate my character with random footsteps, so I would like to move the bones myself. There is the muscle setup page, and some code like this :

Code (CSharp):

1. string[] muscleName = HumanTrait.MuscleName;
2.         int i = 0;
3.         while (i < HumanTrait.MuscleCount) {
4.             Debug.Log(muscleName[i] + " min: " + HumanTrait.GetMuscleDefaultMin(i) + " max: " + HumanTrait.GetMuscleDefaultMax(i));
5.             i++;
6.         }

How can I link "HumanTrait" with my character? Something like myCharacter.GetComponent<Animator>().GetComponent<HumanTrait>()...

2.Also, my character animation plays well in Legacy mode, but not in Humanoid mode, the muscles are correctly set up though, anyone would know how to fix this?
Thanks

 

Thanks, so I have got this import message :



I also don't have 2 of the bones, maybe it is part of the problem (chest and the one above chest).

____

My goal is to get something like this, the legs are following the position of the body dynamically :

I was looking for a way to move a target bone as it is in Blender (the inverse kinematic constraint) where you move one target bone and the 2 bones of the leg are moving accordingly. (so that I can simulate the footsteps only by moving one target bone, instead of the 2 bones, but if I don't find a way, I will just manually change each bone for each leg).

 

Thanks, could you help me making this work? I sent my character on pasteall here ( I put the translation of the chest to 0, but I don't see any scale translation, so the import message is only saying something about the scale). The preview in Unity (the square at the bottom right) shows the animation, but the scene does not.
I found this tutorial on youtube (I put the exact second on the link), the guy can move the arrow of an object to test the IK, how is that possible? When I click Play, no object in my scene can be selected and moved with the arrows.

 

I don't have the "arrows" to move the ik targets in Play Mode, maybe there is an option to select them in Play Mode? I tried to move the ik target with the code, but the leg does not follow. Maybe because the rig does not animate at all in the scene.

 

Doesn't HumanPoseHandler give you access to tweak muscles directly?

 

I'm not sure I follow here.

I've been playing with this for some time now and have a character I'm trying to set his muscle values directly by first calling GetHumanPose to grab his initial T pose in the Start function, then in Update I try to set one of the muscles (the neck muscle, for example) to something different and use SetHumanPose to update the new muscle settings in the Update method, but this doesn't work right since the farther away from the scene origin he is, the weirder things get with his origin (for example, moving his transform doesn't change his position, yet even though he's way away from his bounding box and transform in the scene view, his neck still rotates correctly. Also not sure why this muscle change doesn't work with optimize hierarchy checked).

Care to explain what's going on here?

I'm trying to find a way to overshoot a target pose (using custom interpolation) but I'm having a heck of a time making this possible with Mecanim. I thought muscles would be the way to go, but they're proving to be unpredictable and essentially useless. Any ideas on what else I can try?

 

What kind of code would a custom mixer like that require with the new Jobs system?

I wasn't really looking to have to rewrite the entire Animation / humanoid system from the ground up just to do simple interpolation curves. Since I currently have Mecanim's "humanoid" component / retargeting and its basic muscle limits already there for me to work with, I was planning to simply leverage those systems and use its existing bone-mappings, retargeting, and muscle constraints for the majority of my pose-changing / blending. Is this still possible?



Also is there any way planned to be able to easily (and directly) grab poses? I simply wanted to grab a "humanoid pose" with it, then tweak that pose a little, and write it back. This is actually what led me to "GetHumanPose" in the first place.

Right now I'm just not seeing any easy way to just "grab a pose and change it" without jumping through some major hoops with sampling animations. GetHumanPose seemed to be based on mocap stuff but clearly doesn't seem usable to work as a Mecanim "pose-grabber" that properly preserves both world position and humanoid muscles like I would expect.

 

@Mecanim-Dev
Is there still no way to access the muscle bend rotations directly from code?

I was looking through the humanoid avatar scripting and saw that you expose various limits to code. The docs on this stuff is very vague.

Human trait: https://docs.unity3d.com/ScriptReference/HumanTrait.html
this gives you access to muscle mapping, kinda. It gives access to muscle name string index.

Then you gotta look that up against the HumanDescription's name map? https://docs.unity3d.com/ScriptReference/HumanDescription-human.html

Then you gotta cross reference that against the limits: https://docs.unity3d.com/ScriptReference/HumanLimit.html

I'm still somewhat confused though because most muscles have two rotation axis, there's usually a bend and a twist, so I need two muscle definitions per bone in most cases.

Can you give me a clearer idea on how to interpret all of this? I'm trying to replicate the muscle bends outside of mecanim.

 

Thanks for the response, so to clarify, I'm working on a custom system with physics and ik, I could just hardcode each of the bone rotation limits, but I thought it would be nice to use the definitions from humanoid avatar. It would be even better if I could directly manipulate the joint rotations using the same sliders unity does.

Ideally, I am not looking to do this in job system (I have deadlines and I wanna get this finished asap), but I can set up tool to copy values if needed.

So to use this, I would need to set up a custom job as in example, record the min and max values of each muscle rotation, then use that output as reference?

I just started to write some prototype code to generate the various limits from avatar using

Animator.avatar.humanDescription.human

- I should be able to lerp between the muscle limits here, correct? I would need to hardcode the twist vs rotate from the separate min/max axis here manually, since I don't think there's a reference.

And use the

Animator.avatar.humanDescription.skeleton

tpose positions as a reference point for each of those rotations (they are treated as center, i assume?)

Which approach would you recommend, or is there a better approach elsewhere?

 

So just for reference, this seems to work in most cases.

Code in this screencap:

Code (csharp):

1.  
2.  
3. public HumanBodyBones Bone;
4. [Range(-1,1)]
5. public float LeftArmAmt = 1;
6.  
7. [Range(0,2)]
8. public int TargetAxis;
9.  
10. private void OnValidate(){ SetFromTpose(); }
11.  
12. public void SetFromTpose()
13. {
14.   EnforceTPose();
15.   var bone = Animator.GetBoneTransform(Bone);
16.  
17.   var limit = GetLimit(Bone);
18.  
19.   var angleMin = limit.min[TargetAxis];
20.   var angleMax = limit.max[TargetAxis];
21.  
22.   var ang = Mathf.LerpAngle(angleMin, angleMax, (LeftArmAmt+1)/2f);
23.   Vector3 f = new Vector3();
24.   f[TargetAxis] = ang;
25.  
26.   bone.localRotation = f.EulerToRotation() * bone.localRotation;
27. }
28.  

This seems to work almost perfectly in most cases - except the upper arm twist as shown here - which seems upsidedown. Assuming my avatar looks pretty good during animation, I don't think this is a problem with the avatar definition (?). Any ideas? Should I not be starting from tpose rotations?

EDIT: problem was Mathf.LerpAngle, should have been using vanilla lerp, also I don't think an exact lerp is what I want. Zero from fetal is neutral local rotation, not the half point between the upper and lower limit. So it's like

if( amount < 0 ) Mathf.Lerp( 0, limit.min, -amount);
if( amount > 0 ) Mathf.Lerp( 0, limit.max, amount);

Also, 'origin' rotations start from the fetal pose, not tpose.
For people who might read this later - you can get the fetal pose reference using the sample code found here: https://docs.unity3d.com/2019.3/Documentation/ScriptReference/Animations.MuscleHandle.html

EDIT2: Sadly, this doesn't actually work right. There's another step to converting the limits into rotations that I'm missing. Doesn't seem like enough data is being exposed to actually do a proper mapping. @Mecanim-Dev would love any insight you could provide (I just spent 10 hours on this and I've gotten nowhere - I just want the rotation ranges)...

Just to clarify the problem, the limits

Animator.avatar.humanDescription.human[x].limit

presented in left upper arm are:

LeftUpperArm :: X:-90,90, y:-100,100 z:-60,100

The (-60, 100) matches the muscle settings in arm up down, but to actually match this rotation you need to rotate on the x axis not the z axis, and to fully extend up you need to be at -100 not 100. So this looks like a dead end unless there is a matrix or a mapping I can apply.

 

I'm working with Animation Rigging package, and have the same requirement.
There should be a "Muscle Space", that limits can work properly.
I think Muscle Space transform matrix is very important for Animation Rigging package. Or we can not handle rotation part well...

 

I spent a lot of time trying to duplicate the Mecanim calculations and still don't have it 100% perfect. As long as you have a regular character with an avatar you should be able to get it to pre-calculate the limits for you using a human pose handler.

 

I eventually gave up trying to duplicate.

The human pose handler results don't currently match the output you get from https://docs.unity3d.com/2019.3/Documentation/ScriptReference/Animations.MuscleHandle.html

and I believe that one is more correct. I ended up scrapping the effort and taking a different approach. I believe there are simply additional outputs mecanim adds in that aren't exposed to the managed side.

 

It seems as though you have to recreate the job, script, and graph every time you want to change a source value - If I just want to rotate a mechanim joint, it looks like I have to do this:

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Animations;
3. using UnityEngine.Playables;
4.  
5. public struct MuscleHandleExampleJob : IAnimationJob
6. {
7.     public MuscleHandle muscleHandle;
8.     public float newMuscleValue { get; set; }
9.  
10.     public void ProcessRootMotion(AnimationStream stream) { }
11.     public void ProcessAnimation(AnimationStream stream)
12.     {
13.         AnimationHumanStream humanStream = stream.AsHuman();
14.  
15.         // Get a muscle value.
16.         float muscleValue = humanStream.GetMuscle(muscleHandle);
17.  
18.         // Set a muscle value.
19.         humanStream.SetMuscle(muscleHandle, newMuscleValue);
20.     }
21. }
22.  
23. [RequireComponent(typeof(Animator))]
24. public class Avatar : MonoBehaviour
25. {
26.     PlayableGraph graph;
27.     AnimationPlayableOutput output;
28.     MuscleHandleExampleJob job;
29.     AnimationScriptPlayable script;
30.     private float muscleValue = 0f;
31.  
32.     void Start()
33.     {
34.         job = new MuscleHandleExampleJob();
35.         job.muscleHandle = new MuscleHandle(HumanPartDof.RightArm, ArmDof.ArmDownUp);
36.         job.newMuscleValue = 0f;
37.  
38.         graph = PlayableGraph.Create();
39.  
40.         script = AnimationScriptPlayable.Create(graph, job);
41.      
42.         output = AnimationPlayableOutput.Create(graph, "output", GetComponent<Animator>());
43.         output.SetSourcePlayable(script);
44.      
45.     }
46.  
47.     private void Update()
48.     {
49.      
50.         if (Input.GetKeyDown(KeyCode.E))
51.         {
52.             graph.Destroy();
53.  
54.             job = new MuscleHandleExampleJob();
55.             job.muscleHandle = new MuscleHandle(HumanPartDof.RightArm, ArmDof.ArmDownUp);
56.             muscleValue += 0.1f;
57.             job.newMuscleValue = muscleValue;
58.  
59.             graph = PlayableGraph.Create();
60.  
61.             script = AnimationScriptPlayable.Create(graph, job);
62.  
63.             output = AnimationPlayableOutput.Create(graph, "output", GetComponent<Animator>());
64.             output.SetSourcePlayable(script);
65.  
66.             graph.Evaluate(1.0f);
67.             Debug.LogFormat("Job.newMuscleValue: {0}", job.newMuscleValue.ToString());
68.         }
69.  
70.         if (Input.GetKeyDown(KeyCode.P))
71.         {
72.             graph.Play();
73.         }
74.  
75.         if (Input.GetKeyDown(KeyCode.S))
76.         {
77.             graph.Stop();
78.         }
79.     }
80.  
81.     void OnDisable()
82.  
83.     {
84.         graph.Destroy();
85.     }
86. }

If I try to just set the job.newMuscleValue from Update(), the graph just seems to ignore the changes.

If I try to do something within the job definition itself (for example, setting the value based on Time.realtimeSinceStartup), it of course doesn't allow this because it doesn't allow access to Time except on the main thread.

If I try to set an animationClip as a source.... well, that just brings me all the way back to the original issue that you can't use SetCurve() on an animationClip on non-legacy (mechanim) animator.

? Are we really going to have to create a new graph, new input, and new output every frame just to slowly rotate a mechanim joint?

? This doesn't seem to solve the main issue at all. I'm just trying to dynamically create an animation clip for an Animator (mechanim) at runtime and SetCurve()... which works in the Editor (so, I know it's possible), but it won't work in the build. The reason why I would like to use a curve is so that I can SmoothTangents, and get a nice smooth animation from just a few keyframes, like the colde below -- but SetCurve() fails in a build:

Code (CSharp):

1. // STEP 4: Create a animation curves from the curveSource dictionary and add the curves to the animation clip
2.         foreach (KeyValuePair<string, List<Keyframe>> curve in curveSource)
3.         {
4.             AnimationCurve m_curve = new AnimationCurve(curve.Value.ToArray());
5.             if (smoothTangents)
6.             {
7.                 for (int i = 0; i < m_curve.length; i++)
8.                 {
9.                     m_curve.SmoothTangents(i, 0.5f);
10.                 }
11.             }
12.             animClip.SetCurve(curve.Key, typeof(Animator), curve.Key, m_curve);
13.         }

 

I'm scripting an editor tool to control muscle handle and ik with playablegraph,
after fixed the function about recording animation clip with this tool(got some trouble to get playablegraph of unity animation window),
I will update the post.

Here is the workflow

1.MuscleHandle.GetMuscleHandles(handles);
2.Inject handles to some class which you could analyze and edit

Code (CSharp):

1. public class MuscleGroup
2.     {
3.         public event Action UpdateMuscle;
4.         public MuscleGroup(HumanPartDof dof, MuscleData[] datas)
5.         {
6.             Datas = datas;
7.             GroupName = dof;
8.         }
9.         [HideInInspector]
10.         public readonly HumanPartDof GroupName;
11.         [HideInInspector]
12.         public readonly MuscleData[] Datas;
13.  
14.         [HideInInspector]
15.         public int Max=int.MinValue;
16.         [HideInInspector]
17.         public int Min=int.MaxValue;
18.         public MuscleData this[int index]
19.         {
20.             get
21.             {
22.                 return Datas[index + Min];
23.             }
24.             set
25.             {
26.                 Datas[index + Min] = value;
27.             }
28.         }
29.  
30.         public void KeyVolume(int sn, MuscleData muscleData)
31.         {
32.             Max = Mathf.Max(sn,Max);
33.             Min = Mathf.Min(sn, Min);
34.             m_Datas.Add(muscleData);
35.         }
36.  
37.         [HideLabel]
38.         [ShowInInspector][OnValueChanged(nameof(Update),IncludeChildren =true,InvokeOnInitialize =false)]
39.         protected List<MuscleData> m_Datas=new List<MuscleData>();
40.  
41.         void Update()
42.         {
43.             var length = m_Datas.Count;
44.  
45.             for (int i = 0; i < length; i++)
46.             {
47.                 Debug.Log(Datas[Min + i]);
48.                 Datas[Min + i].m_Value = m_Datas[i].m_Value;
49.             }
50.  
51.             UpdateMuscle?.Invoke();
52.         }
53.     }

3.Add these data to scriptplayable like

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.Jobs;
5. using UnityEngine.Animations;
6. using Return.Humanoid.Animation;
7.  
8. public partial struct Job_Humanoid_MuscleRigging : IAnimationJob
9. {
10.     public Vector3 BodyPosition;
11.     public Quaternion BodyRotation;
12.     public TransformStreamHandle Root;
13.  
14.     public MuscleData[] Datas;
15.  
16.     public void ProcessAnimation(AnimationStream stream)
17.     {
18.         var humanStream = stream.AsHuman();
19.  
20.         if (Datas == null)
21.             return;
22.  
23.         foreach (var data in Datas)
24.         {
25.             humanStream.SetMuscle(data.m_Handle, data.m_Value);
26.         }
27.  
28.  
29.         Root.SetLocalTRS(stream, BodyPosition, BodyRotation, Vector3.one, false);
30.     }
31.  
32.     public void ProcessRootMotion(AnimationStream stream)
33.     {
34.         var humanStream = stream.AsHuman();
35.  
36.         if(Root.IsValid(stream))
37.         Root.SetLocalTRS(stream, BodyPosition, BodyRotation, Vector3.one,false);
38.  
39.     }
40. }
41.  

 

It works fine.

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Animations;
3. using UnityEngine.Playables;
4. public struct MuscleHandleExampleJob : IAnimationJob
5. {
6.     HumanPoseHandler humanPoseHandler;
7.     public MuscleHandle muscleHandle;
8.     public float newMuscleValue { get; set; }
9.     public void ProcessRootMotion(AnimationStream stream) {
10.         // AnimationHumanStream humanStream = stream.AsHuman();
11.         // float muscleValue = humanStream.GetMuscle(muscleHandle);
12.     }
13.     public void ProcessAnimation(AnimationStream stream)
14.     {
15.         AnimationHumanStream humanStream = stream.AsHuman();
16.         humanStream.SetMuscle(muscleHandle, newMuscleValue);
17.     }
18. }
19. [RequireComponent(typeof(Animator))]
20. public class T : MonoBehaviour
21. {
22.     PlayableGraph graph;
23.     AnimationPlayableOutput output;
24.     MuscleHandleExampleJob job;
25.     AnimationScriptPlayable script;
26.     private float muscleValue = 0f;
27.     void Start()
28.     {
29.         job = new MuscleHandleExampleJob();
30.         graph = PlayableGraph.Create("lzx395");
31.         script = AnimationScriptPlayable.Create(graph, job);
32.         output = AnimationPlayableOutput.Create(graph, "output", GetComponent<Animator>());
33.         output.SetSourcePlayable(script);
34.         graph.Play();
35.     }
36.     private void Update()
37.     {
38.         if (Input.GetKeyDown(KeyCode.E))
39.         {
40.             job.muscleHandle = new MuscleHandle(BodyDof.ChestFrontBack);
41.             muscleValue += 0.1f;
42.             job.newMuscleValue = muscleValue;
43.             script.SetJobData(job);
44.         }
45.     }
46.     void OnDisable()
47.     {
48.         graph.Destroy();
49.     }
50. }