Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

HumanPose-issue (continued)

Discussion in 'Animation' started by roumenf, Jul 19, 2017.

  1. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    Dear @JamesB,

    Here is the same question, again:
    What code should I add between GetHumanPose() and SetHumanPose() in the the script below, to keep the model's global position and orientation unchanged between the updates, and if possible the same as in the edit mode. Thank you for the hint at Unite, and thank you in advance for any further help or suggestion!

    "
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. public class HumanPoseTest : MonoBehaviour
    6. {
    7.     private Animator animatorComponent = null;
    8.     private HumanPoseHandler humanPoseHandler = null;
    9.     private HumanPose humanPose = new HumanPose();
    10.  
    11.     void Start ()
    12.     {
    13.         animatorComponent = GetComponent<Animator>();
    14.         humanPoseHandler = new HumanPoseHandler(animatorComponent.avatar, transform);
    15.     }
    16.  
    17.     void Update ()
    18.     {
    19.         humanPoseHandler.GetHumanPose(ref humanPose);
    20.         Debug.Log("bodyPosition: " + GetPosAsString (humanPose.bodyPosition) + ", bodyRotation: " + GetRotAsString (humanPose.bodyRotation));
    21.         humanPoseHandler.SetHumanPose(ref humanPose);
    22.     }
    23.  
    24.  
    25.     private string GetPosAsString(Vector3 vPos)
    26.     {
    27.         return string.Format ("({0:F6}, {1:F6}, {2:F6})", vPos.x, vPos.y, vPos.z);
    28.     }
    29.  
    30.  
    31.     private string GetRotAsString(Quaternion qRot)
    32.     {
    33.         Vector3 vRot = qRot.eulerAngles;
    34.         return string.Format ("({0:F0}, {1:F0}, {2:F0})", vRot.x, vRot.y, vRot.z);
    35.     }
    36.  
    37. }
    "
     
  2. JamesB

    JamesB

    Unity Technologies

    Joined:
    Feb 21, 2012
    Posts:
    133
    As we discussed before the Get function returns a human pose in local space. But it is not the position of a transform and getting no movement at all is going to be very difficult. I've spoken to the devs about how to counter the movement and their advice was not to do the Getting and Setting all the time and find another way to achieve what you want. When you get the HumanPose you are getting the result of retargetting a skeleton and when you Set it it's retargetting again. The process of retargetting is subject to error so doing this every frame is just going to compound the error.

    What is it that you want to do here? We'll try and think of another way of doing it.
     
  3. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    I try to fit the muscle values in the human pose to their configured limits in 'Muscles & Settings'. This has to be done at run-time. So, I process humanPose.muscles[], and if anything is changed in there, I try to apply the pose back. In this regard, re-retargeting the skeleton is what I need, hence not a problem. But then, there are these two other properties - bodyPosition and bodyRotation that I don't need at all, but they ruin the effect of applying the pose back.

    I looked for other ways in the Mecanim API, but could not find anything else. So, I would appreciate it, if you suggest another way of doing it.

    By the way, I'm not sure if this was by-design so, but the local position of skeleton's root node (in Inspector, and scripts) is in many cases different than the local position of the same root node, shown in Humanoid's configuration window. This may be a hint to the devs.
     
  4. JamesB

    JamesB

    Unity Technologies

    Joined:
    Feb 21, 2012
    Posts:
    133
    So if you aren't looking to do this every frame then just subtracting the world position from the body position and doing the inverse rotation of the body rotation should be all you need. Also, if you are copying from one avatar to another then you don't have compound retargetting problems. It's only if you are getting and setting on top of one another again and again. I had it explained to me with this helpful gif: http://imgur.com/Z60Q0Xk
    There's no other way that I know of to set the pose. It's just a matter of when you set it and from what.
     
  5. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    May I ask you to check the scene in my test project: http://ratemt.com/downloads/HumanPoseTest.zip

    I did as you suggested, but the model now is swapping its position up and down alternatively, after each invocation of SetHumanPose(). It is invoked once per second in the script, just not to be on each Update().
    What may be the reason for this behavior, and what should I do to fix it?
     
  6. JamesB

    JamesB

    Unity Technologies

    Joined:
    Feb 21, 2012
    Posts:
    133
    I'm very sorry but I don't have time to look at your project. Also sorry for my lack of clarity before: it is not the frequency that you call it with that causes the error. It is that you call get and then set it back. Each call to Get or Set runs the retargertting algorithm which is subject to error. Imagine changing an image from bitmap to a jpeg then back to bitmap and to jpeg again. The error only gets worse as the process is lossy. It seems like the thing you are trying to do is not what those functions were designed for. Sorry I don't have better news.
     
  7. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    OK, I don't want to press. Please check it, as soon as you have a free minute. FYI: The scene is just an implementation of your suggestion above: "subtracting the world position from the body position and doing the inverse rotation of the body rotation should be all you need". The scene contains one humanoid model with the HumanPoseTest-script attached to it. When you run it, the model starts jumping up and down each second, when the SetHumanPose()-method gets invoked. If you move the model in the Scene-view at runtime, additional effects pop up.

    I mean: something is still not right, and it is not just a gradual loss of quality. The project above is only a practical demo. Feel free to forward it to Mecanim devs as well, if they could provide any insight. I would appreciate a code sample in Unity documentation or here, of what these functions were designed for.

    Anyway, thank you for the responsiveness and the great support!
     
  8. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    @JamesB, according to your suggestion, here is the code I added between GetHumanPose() and SetHumanPose():
    "
    Code (CSharp):
    1. humanPose.bodyPosition = transform.position - humanPose.bodyPosition;
    2. humanPose.bodyRotation = Quaternion.Inverse(humanPose.bodyRotation);
    3.  
    "
    And here is the log, including body position & rotation, as well as root-node position before and after the call:
    "
    Code (CSharp):
    1. before SetHumanPose() - bodyPos: (-0.000060, 0.466750, -0.000394), bodyRot: (0, 180, 0), rootNodePos: (0.000000, 1.000000, 0.000000)
    2. after SetHumanPose() - bodyPos: (0.000060, 0.533250, 0.000394), bodyRot: (0, 180, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    3. before SetHumanPose() - bodyPos: (-0.000060, 0.996487, -0.000394), bodyRot: (0, 0, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    4. after SetHumanPose() - bodyPos: (0.000060, 0.003513, 0.000394), bodyRot: (0, 0, 0), rootNodePos: (0.004962, 0.709042, 0.032610)
    5. before SetHumanPose() - bodyPos: (-0.000060, 0.466750, -0.000394), bodyRot: (0, 180, 0), rootNodePos: (0.004962, 0.709042, 0.032610)
    6. after SetHumanPose() - bodyPos: (0.000060, 0.533250, 0.000394), bodyRot: (0, 180, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    7. before SetHumanPose() - bodyPos: (-0.000060, 0.996487, -0.000394), bodyRot: (0, 0, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    8. after SetHumanPose() - bodyPos: (0.000060, 0.003513, 0.000394), bodyRot: (0, 0, 0), rootNodePos: (0.004962, 0.709042, 0.032609)
    9. before SetHumanPose() - bodyPos: (-0.000060, 0.466750, -0.000394), bodyRot: (0, 180, 0), rootNodePos: (0.004962, 0.709042, 0.032609)
    10. after SetHumanPose() - bodyPos: (0.000060, 0.533250, 0.000394), bodyRot: (0, 180, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    11. before SetHumanPose() - bodyPos: (-0.000060, 0.996487, -0.000394), bodyRot: (0, 0, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    12. after SetHumanPose() - bodyPos: (0.000060, 0.003513, 0.000394), bodyRot: (0, 0, 0), rootNodePos: (0.004962, 0.709042, 0.032609)
    13. before SetHumanPose() - bodyPos: (-0.000060, 0.466750, -0.000394), bodyRot: (0, 180, 0), rootNodePos: (0.004962, 0.709042, 0.032609)
    14. after SetHumanPose() - bodyPos: (0.000060, 0.533250, 0.000394), bodyRot: (0, 180, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    15. before SetHumanPose() - bodyPos: (-0.000060, 0.996487, -0.000394), bodyRot: (0, 0, 0), rootNodePos: (-0.005221, 1.852597, -0.034309)
    16. after SetHumanPose() - bodyPos: (0.000060, 0.003513, 0.000394), bodyRot: (0, 0, 0), rootNodePos: (0.004962, 0.709042, 0.032609)
    "

    Is there an error in my code? If not, why do body position and rotation change their values alternatively (see the before-values), and why the root-node position is changed too (maybe it is so by design), but it's never the same or close to the initial position, as one would expect?
     
    Last edited: Jul 27, 2017
  9. JamesB

    JamesB

    Unity Technologies

    Joined:
    Feb 21, 2012
    Posts:
    133
    Sorry I just don't know. Maybe someone else here will be able to help you.
     
  10. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    Hm, I'm surprised this function is so mysterious in the end. Someone should have written, tested or used it for something, after it was added to the API about 2 years ago.
     
    CloudyVR likes this.
  11. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    @roumenf and @Mecanim-Dev

    I really would like it if there was some sort of clarification about these two functions. The "brushing-us-off" thing by the other unity representative doesn't really inspire confidence... D:

    In my particular case, I have a bone transform I'm trying to set via muscles so that I can retarget it to another avatar after applying manual animation of, say, the y value.

    This seems to go nuts and offset my character from its initial position, or make my character circle around its origin position, depending on how I try to do it with the get/set humanpose. If I offer any other transform besides the topmost gameobject transform (containing the bone transform heirarchy), the thing doesn't do anything at all and not even the offset works.


    Here's my code, but please note, my avatar is not put in the proper spot upon hitting "Play" to test the game:

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. public class HeadRotatorMuscle : MonoBehaviour
    4. {
    5.  
    6.     //   Human muscle stuff
    7.     HumanPose humanPose;
    8.     public float moves = -20f;
    9.  
    10.  
    11.     public GameObject source;
    12.     public Avatar destAvatar;
    13.  
    14.     private HumanPoseHandler sourceHandler;
    15.     private HumanPoseHandler destHandler;
    16.  
    17.  
    18.     // Indexes to muscles
    19.     // 9: Neck Nod Down-Up min: -40 max: 40
    20.     // 10: Neck Tilt Left-Right min: -40 max: 40
    21.     // 11: Neck Turn Left-Right min: -40 max: 40
    22.     // 12: Head Nod Down-Up min: -40 max: 40
    23.     // 13: Head Tilt Left-Right min: -40 max: 40
    24.     // 14: Head Turn Left-Right min: -40 max: 40
    25.  
    26.  
    27.     //   Bone stuff
    28.     Transform head;
    29.  
    30.     // Muscle name and index lookup (See in Debug Log)
    31.     void LookUpMuscleIndex ()
    32.     {
    33.         string[] muscleName = HumanTrait.MuscleName;
    34.         int i = 0;
    35.         while (i < HumanTrait.MuscleCount)
    36.         {
    37.             Debug.Log(i + ": " + muscleName[i] +
    38.                 " min: " + HumanTrait.GetMuscleDefaultMin(i) + " max: " + HumanTrait.GetMuscleDefaultMax(i));
    39.             i++;
    40.         }
    41.     }
    42.  
    43.     // Set character in fetus position
    44.     void ResetMuscles ()
    45.     {
    46.         // reset all muscles to 0
    47.         for (int i = 0; i < humanPose.muscles.Length; i++)
    48.         {
    49.             //Debug.Log (humanPose.muscles [i]);
    50.             humanPose.muscles[i] = 0;
    51.         }
    52.     }
    53.  
    54.    
    55.  
    56.     //   !!! Human Pose approach !!!
    57.     void Start ()
    58.     {
    59.         // https://forum.unity.com/threads/humanposehandler.430354/
    60.  
    61.         humanPose = new HumanPose();
    62.         sourceHandler = new HumanPoseHandler( source.GetComponent<Animator>().avatar , source.transform);
    63.         destHandler = new HumanPoseHandler(destAvatar , transform);
    64.  
    65.         sourceHandler.GetHumanPose(ref humanPose);
    66.         destHandler.SetHumanPose(ref humanPose);
    67.        
    68.  
    69.         // run this if you want the indexes to muscles on your character
    70.         LookUpMuscleIndex();
    71.  
    72.     }
    73.  
    74.  
    75.     // Update is called once per frame
    76.     void Update ()
    77.     {
    78.        
    79.         humanPose.muscles[9] = moves;
    80.         destHandler.SetHumanPose(ref humanPose);
    81.        
    82.     }
    83. }

    It seems like sethumanpose with any value changed in the humanpose.muscles[] array makes this thing lose its mind. D:

    Any clarification on how we're supposed to even USE this thing? -- I simply want to retarget to the same gameobject (which either doesn't work -- or is totally broken in 2017.)
     
    Last edited: May 18, 2018
  12. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    My solution was as follows:
    1. In the Start()-method:
    Code (CSharp):
    1. initialHipsPosition = (humanPose.bodyPosition - transform.position);
    2. initialHipsRotation = humanPose.bodyRotation;
    2. In Update():
    Code (CSharp):
    1. Quaternion localBodyRot = Quaternion.Inverse(initialHipsRotation) * humanPose.bodyRotation;
    2.  
    3. humanPose.bodyPosition = initialHipsPosition;
    4. humanPose.bodyRotation = localBodyRot;
    5.  
    6. humanPoseHandler.SetHumanPose(ref humanPose);
    This solution still has some issues, but it was the best I could do so far. As you found out by yourself, the documentation and samples on the topic are missing, and the Unity staff is not very experienced and supportive.
     
  13. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    Thanks for sharing that -- it definitely helped me. -- At least the character starts in the proper location now! D:

    ...and on another note:

    Yeah, sadly, judging from this experience, I can easily see why you'd say that. D: -- At the very least, he could have poked one of his buddies around the office (or maybe his boss?) to chime in...? At least then he could have learned something too...

    As far as I know, no job I've ever had dealing with customers has ever condoned the "Sorry I just don't know. Maybe someone else here will be able to help you." attitude..
     
  14. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    The guy above was one of the supportive ones. He tried to debug the issue at Unite last year. The question is, where are the authors of this wonderful API, to comment or provide some insight at least.
     
  15. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    Yeah, that's a good point.

    The Mecanim devs are clearly out of the picture (and the one guy we all know as a "mecanim dev" (@Mecanim-Dev, lol) is clearly overworked and absent from the posts/forums a majority of the time (hopefully working on refactoring the Mecanim system or its API? -- one can only hope), so, sadly, it's pretty hard to get enough info about this system's black box nature to be able to work with it smoothly on our own.

    To be honest, I may talk smack about the design of Mecanim sometimes, but it is a really great technical feat. The problem I have is that it's so difficult to understand the API for scripting when you don't have access to what's going on behind the scenes. On top of that, its design (and documentation) isn't intuitive or clear at all. Out of all of the systems in Unity, sadly, sometimes Mecanim seems the least Unity-like to me for the reasons mentioned above.

    That being said -- I'm going to make a topic on that soon and offer an alternative way to look at Mecanim (and possibly help them refactor its API design a bit better). I love the fact that Mecanim can animate almost anything by simply hitting record and changing it -- however, it's kind of sad that such a cool feature gets overshadowed by the poor support for basic 3d model pose animation thanks to the convoluted Humanoid integration. :(
     
    Last edited: May 25, 2018
    roumenf likes this.
  16. BernieRoehl

    BernieRoehl

    Joined:
    Jun 24, 2010
    Posts:
    80
    Sadly, I have to agree. For the Unity tech support people to say "sorry, we don't know how our code works" is disappointing.

    Even if the people who originally wrote the code aren't there anymore, it should be easy enough for someone at Unity to simply look at the source for GetHumanPose() and SetHumanPose() and tell us what's happening with bodyPosition and bodyRotation.

    I'm having the exact same problem that other people are, and it's really frustrating to not have any way to solve or debug it.
     
    Last edited: Jun 10, 2018
  17. BernieRoehl

    BernieRoehl

    Joined:
    Jun 24, 2010
    Posts:
    80
    By trial and error, I've found the following works in my specific case:
    Code (csharp):
    1.  
    2.         void LateUpdate() {
    3.             handler.GetHumanPose(ref humanPose);
    4.             humanPose.bodyPosition = humanPose.bodyPosition.y * Vector3.up;
    5.             humanPose.bodyRotation = Quaternion.identity;
    6.             for (int i = 0; i < muscleIndices.Length; ++i)
    7.                 humanPose.muscles[muscleIndices[i]] = values[i];
    8.             handler.SetHumanPose(ref humanPose);
    9.         }
    10.  
    I have no idea why it works, or if it will work for you, because there's no documentation on how GetHumanPose() and SetHumanPose() handle the bodyPosition and bodyRotation values.
     
    Chao0707 and roumenf like this.
  18. roumenf

    roumenf

    Joined:
    Dec 12, 2012
    Posts:
    635
    I think it would be fine, if there is an option for us (paid or not), to look at the source code by ourselves, in order to find a proper solution. Now the only way to look at the engine sources is to have a Pro-license, as far as I remember. But, for an indie to buy a pro-license to fix Unity's own bugs would be a bit too much.

    And I'm amazed for years, how Unity manages to keep a top product, and fail almost always in its own native code support. I even bought premium support package once, to try it out, and it was not much better than the free one.
     
  19. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    I had wondered about this -- but after seeing some of the actual devs from the Unity team (with the source-code in hand) actually say "I don't know -- ask somebody else" -- I pretty much gave up all hope on "pro" support being any better.

    I had recently read that Unity was one of the "best places to work" in silicon valley, but I bet a bit of the tradeoff for that is to let their programmers "do whatever they want to make a good tool" -- except, the problems with that approach is lack of team coordination (a specific goal for the day/week), lack of devotion to the final "design" specs (if they exist at all), lack of any personal investment in the tool after its "completion" (because "ooh, more [different!] fun projects and so little time!", and, clearly, the lack of documentation (and ultimately knowledge about the tool) so that other people may know (or learn) about the tool even after the original "team of programmers" are out of the picture and working on other things.

    It's clearly a culture issue -- and it's one that needs solving.

    I really enjoy the heck out of Unity -- As a whole, it's an amazing technical accomplishment! -- but it's stuff like this (the little "design" quirks and unorthodox / undocumented features / quirks / interrelations you can't do anything about) that end up making a project 10x harder than it "should" be, which tests many people's patience day-to-day.

    I'm glad they're moving to ECS though -- I think it's a good move to let people have much lower-level control over the stuff they use in Unity (such as meshes and transforms). I'm still awaiting plans for what the Animation Jobs system will be able to accomplish with ECS, but as long as they give us low-level control over the bone transforms, we might be good to go on reimplementing stuff like HumanPose in a better way.
     
    roumenf likes this.