Search Unity

Optmize hierarchy of a generated skinned mesh

Discussion in 'Animation' started by drolak, Jul 14, 2015.

  1. drolak

    drolak

    Joined:
    Jan 21, 2014
    Posts:
    49
    Hey all!

    I'm working on a character customization system for our game. Don't want to bore you with specifics, but at some point I need to merge some meshes on a animated (by mecanim) skinned mesh renderer (we have different meshes for different cloths, hairstyles etc.).

    It's quite simple - it's a matter of combining meshes and coping the boneWeights.

    It works BUT.... only when the transform hierarchy is deoptimized.
    If the heirarchy is optimized (or when I optimize it after generating the mesh) I get this error:

    itParent != transformToMark.end ()
    Bones do not match bindpose.

    Funny thing - I downloaded the UMA package (which does a similar thing to my system but is much bigger than my needs) and I noticed that their avatars also use deoptimized hierarchy. When I try to optimze it (by hitting the gear icon on Animator component and using optimize transform hierarchy option) I get the same error.

    So the question is - am I doing something wrong or did I hit a bug in unity (saw it on 5.1.0 and 5.1.1p4)? Anyone stumbled upon this?

    This can be easily reproduced by downloading UMA, opening the Crowd scene, hitting play and doing optimize transform hierarchy on any animator.
     
    burningmime likes this.
  2. theANMATOR2b

    theANMATOR2b

    Joined:
    Jul 12, 2014
    Posts:
    7,790
    Did you find a solution - beyond not optimizing?
     
  3. drolak

    drolak

    Joined:
    Jan 21, 2014
    Posts:
    49
    Unfortunately not. I did some performance tests and for my setup there was no visible difference, so I left the hierarchy in the "not optimzed" state.
     
    theANMATOR2b likes this.
  4. Trigve

    Trigve

    Joined:
    Mar 17, 2013
    Posts:
    139
    I was hit also by this, I'm on 5.3.3p1. The error message is really unhelpful. Could some unity developer chime in?
     
  5. KB73

    KB73

    Joined:
    Feb 7, 2013
    Posts:
    234
    Getting something similar - if i manually optimize the mesh and run the game, it kinda works....root motion etc behaves. If we run it at runtime, we get the error above / hierarchy looks correct but the root motion fails.

    *Root motion was fixed by doing it offline, not runtime, now i have the issue of the odd bone disappearing and being optimized even though it being told not.

    *Odd bone disappearing can be worked around by adding a stub component
     
    Last edited: Apr 10, 2016
  6. watsonsong

    watsonsong

    Joined:
    May 13, 2015
    Posts:
    555
  7. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    Would you look at that. Been looking at why OptimizeTransformHierarchy broke combined combined skinned meshes this week. And then this gets popped up!

    @DavidGeoffroy, since you've been around here a bit, is this something you guys have on your list? The animators are wrecking our frame rate on WiiU, and optimizing the transform hierarchy fixes that. But we can't do that on combined skinned meshes, which we've got a bunch of.

    Asking to know if we can just wait for a fix or need to do some serious re-thinking of how we do things.
     
  8. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    The reason why it's not working is the Optimize Transform Hierarchy feature relies on a translation map between animation curves and vertices, and by combining meshes, the bones and vertices don't match anymore, and we can't optimize.

    We haven't forgotten about it, but we have been swamped with fixing higher priority bugs (crashes, performance regressions and feature regressions) for the whole 5.2 and 5.3 cycle. Since this has never actually been supported, its priority is low.

    I understand fully how optimize hierarchy and combine mesh would be useful together, but it's not a trivial fix, so I'm not sure when it will be fixed.
     
    theANMATOR2b likes this.
  9. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    I found a workaround, at least for https://issuetracker.unity3d.com/is...nimatorutility-dot-optimizetransformhierarchy

    Here's the conditions in which this will work:
    1- If you're starting from a mesh/skeleton combo from an FBX file (where the bone mappings are defined)
    2- If you're not adding new bones/bindposes

    All you have to do is start from a copy of the original mesh instead of creating a new one, this will copy the bone mappings necessary for Optimize Transform hierarchy to work.

    It should (almost) be as simple as changing this:
    //pseudocode, will not compile
    newMesh = new Mesh()
    to
    newMesh = Instantiate(originalMesh)

    then change whatever values you need to change.

    Also, when you change the properties of your mesh, make sure to change triangles before vertices, otherwise, you will get some errors (I'm not exacly sure why this doesn't happen with a new mesh, and it's somewhat outside my purview).

    Also, as far as I could see, if you're using Mesh.CombineMeshes, this "should" already be handled correctly, but I have no example to test against. If someone has a simple-ish example that uses CombineMeshes, and doesn't work with Optimize Transform Hierarchy, please file a bug and paste the number here, I'll check it out.
     
  10. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    Doesn't work for me. My case is different from the one reported in the bug - I'm not just changing the properties of an existing mesh, I'm creating a brand new mesh based on about 20ish old meshes.

    It works somewhat like:
    - Gather data about all of the old meshes
    - Calculate bindposes and bone weights and such for the new mesh
    - Generate new mesh, add it to a brand new skinned mesh renderer.

    That works great, but if I use the Optimize Transform Hierarchy functionality, the entire thing breaks, and Unity spams "Bones do not match bindpose." to the console - no stack trace, just the error. Probably an assert somewhere?

    Basing the new mesh on one of the old meshes instead of creating a new one made no difference.

    I can make a bug report where I include a model and the scripts I use, if you're interested. As I've said, not being able to optimize the hierarchy on WiiU is a major dealbreaker. With 10 somewhat complex animators on the screen, the profiler shows that Animator.Update alone eats most of our ms budget per frame.


    You sure you don't mean "change vertices first"?

    If I increase the number of vertices and triangles, setting the triangles triggers the "Failed setting triangles. Some indices are referencing out of bounds vertices" error, as some of the vertices referenced in the new triangles are out of bounds in the old vertices.
     
  11. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Yes please. Worst case, I can probably figure out a workaround for you, best case, I can use your use case to create an API that makes sense in real world conditions (and test it out in real world conditions).

    Frankly, this is the first time I play around with the Mesh. So I'm not sure.

    In this bug: https://issuetracker.unity3d.com/is...nimatorutility-dot-optimizetransformhierarchy, I had to write triangles first, otherwise there was a mismatch. YMMV.
     
  12. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    It's Case 819713. I copied over the scripts and models that were necessary from the real project. The combination script is based on one from the wiki and hacked until it worked, so there might be an error.

    As I wrote in the bug report, there's buttons both for optimizing and then combining meshes, and for first combining meshes and then baking. The combination process needs to gather information about all of the bones, so optimizing the hierarchy from outside of the script before combining will just cause a bunch of nullrefs.

    If there's any problems, or follow-up questions, please ask!
     
  13. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
  14. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    @Baste Sorry about the long delay, I was quite tied up with high priority regressions.

    Yes, I looked at your project, and you definitely need a method to set the hashes of the bones associated to the bindposes in order to make this work.

    2-3 weeks ago, I tried making workaround methods using Serialized properties, so that you could try it out without a new build, but I hit a dead end because of some issues with the way the mesh is shared. Then I got pulled back on bugs.

    Now, I have a new API, and it works with a simple use case (https://issuetracker.unity3d.com/is...nimatorutility-dot-optimizetransformhierarchy).

    And it solves the issue with "Bones do not match bindpose.", but it uncovers a new error:

    "The input bones do not match the skeleton of the Avatar(somename).
    Please check if the Avatar is generated in optimized mode, or if the Avatar is valid for the attached SkinnedMeshRenderer."

    The new skeleton you make has bones that are not part of the avatar, and partial skeleton matches are not supported, because then your new bones could not be interacted with in any way.

    Figuring out if we can support partial skeleton matches is quite another can of worms, and one we won't address for the moment.

    Still.. the new API will be in 5.6.

    If you want to try it out, I can make a 5.4 build with the API.
     
    Last edited: Sep 7, 2016
  15. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    Hi!

    Thanks for the effort! Sadly, we're in such a late stage of production that 5.6 won't be in time. If you need feedback on it, I can take a look at the 5.4 build and see where it gets us. But it's nice to see progress.


    We were using this to make an in-editor NPC designer for our level designers. They simply picked the head, torso, arms, etc. they wanted. We needed to bake the SkinnedMeshRenderers to a single renderer since we're using projectors, which each multiply the number of draw per renderer by 2. We needed to optimize the animator since all of the bones were causing the WiiU to grind to a halt.

    We've ended up just doing this the hard way - baking out each version of what we want in blender, and importing them. Oh well.
     
  16. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    How underpowered is the Wii U? I do all of my mobile tests on a crappy mobile phone from several years ago with a 1ghz single core A8 cpu and optimizing the hierarchy didn't make any difference at all. Combining meshes did as the animator is quite expensive, but calculating transforms isn't.
     
  17. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    I don't have numbers on the Wii U, but updating transforms, for characters or objects with a deep hierarchy, generally represents 60-80% of the animation cost, and it can't be multithreaded.

    Of course, if your GPU is what's holding your framerate back, saving on CPU possibly won't change much
     
  18. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    I need to rebenchmark the thing then, cause in my test there was no difference and I was using 3 bones per finger too, so not exactly a lightweight bone setup. I must have made some mistake.
     
  19. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,338
    Note that in our use case, the lag shows up with 10+ NPCs on screen, each with ~180 bones in their rig.

    That means that optimizing the hierarchy saves us ~1800 transform updates per frame. It would probably be a problem on all platforms, we just noticed it first on the WiiU, since we're running on quite beefy development computers. Or it might be a specific thing with that platform that makes it worse, idk.
     
    theANMATOR2b and 00christian00 like this.
  20. BTables

    BTables

    Joined:
    Jan 18, 2014
    Posts:
    61
    We have a similar workflow, multiple mesh sources combined together (ours are stiched from out own intermediate format so are essentially procedural), we would love to be able to optimise the hierarchy .

    Did this new API make it into 5.6? I couldn't see anything in the changelog.
     
  21. brettj

    brettj

    Joined:
    Feb 9, 2015
    Posts:
    43
  22. Sebioff

    Sebioff

    Joined:
    Dec 22, 2013
    Posts:
    218
    Doesn't look like it's been released yet as of Unity 2017.3 and the problem described in the original post still exists - any idea what the current state is?
     
  23. wanban42

    wanban42

    Joined:
    Mar 18, 2013
    Posts:
    2
    Hi !
    According to ...

    It should (almost) be as simple as changing this:
    //pseudocode, will not compile
    newMesh = new Mesh()
    to
    newMesh = Instantiate(originalMesh)

    I saved the combined mesh as asset. It's YAML file content is as follow ...

    .....................
    m_BoneNameHashes: 82d9c0f4a6a1a2f1d1331a48751932a0dc026901fe6dc5e7c5c604062f86d171bf5ce0fda5e7259f2a4bcc5914d3a06220dc00f0
    m_RootBoneNameHash: 4053967270
    ......................

    The mesh is OK when optimize game object option is on.

    I think Unity may add API functions to fix the problem.

    static int Mesh.BoneNameToHash(string sname);
    int Mesh.RootBoneNameHash { get; set;};
    int[] Mesh.BoneNameHashes {get; set;};
     
  24. Dorodo

    Dorodo

    Joined:
    Mar 8, 2015
    Posts:
    44
    @DavidGeoffroy I'm interested in this as well. It's been a while, but it looks like the problem is still around in 2017.3?
     
  25. jhughes2112

    jhughes2112

    Joined:
    Nov 20, 2014
    Posts:
    178
    Curious. Can ANYBODY confirm that OptimizeTransformHierarchy even works when passing in a list of bones to protect from removal? Everything I have attempted to pass in has resulted in this call stack:

    Assertion failed: Assertion failed on expression: 'it != transformToMark.end()'
    UnityEngine.AnimatorUtility:OptimizeTransformHierarchy(GameObject, String[])

    I've tried the transform.name, I've tried absolute full names of the object with dots between the names, slashes, etc. I even tried full paths relative from the animator object. Little help? The docs don't really help. As far as I can tell, it's broken. The only way I can figure I can keep a bone from being removed is by adding a (nearly) empty script component to the bone.
     
  26. KB73

    KB73

    Joined:
    Feb 7, 2013
    Posts:
    234
    The empty script is the approach we use and have done for a while.
     
    jhughes2112 likes this.
  27. FrozenEmpire

    FrozenEmpire

    Joined:
    Sep 10, 2016
    Posts:
    96
    Hi,

    Just encountered this limitation with optimising combined or simplified skinned meshes and resulting in "Bones do not match bindpose".

    Using 2020.3

    Did it ever get solved as it seems like an incredibly useful way to gain additional performance...
     
    burningmime likes this.