Feedback Animation wishlist

I hear a few people badmouthing Mecanim, but I don't think it's as bad as they make it out to be - it's pretty nice for some things. That said, I do think it has some shortcomings that need to be addressed. These are the things I ran into in my latest project.

1. Mecanim has no built-in support for facial animation. Facial animation is becoming more and more common in games that involve NPC dialogue. During this project, I was tasked with making Faceware integrate with Mecanim, along with the body mocap. Live Client wasn't an option, because, well, it isn't a live performance. Making it work was an exercise in frustration. I eventually got it to work, but the solution felt very hacky. (I'll post that solution later once it's fully fleshed out.)
It would be nice to have an out-of-the-box retargeting system for facial animation. I know there aren't any established "official" standards, but someone has to start, and I think Unity is influential enough to get the ball rolling. Faceware has excellent documentation on facial rigging (you can find it here). That might be a good starting point. Also check that out if you just want to learn more about facial rigging.

2. There is no way to alter an animation event graph (Animator component) at runtime. Sure, you can change floats, bools, ints, and triggers, but if you want to play an arbitrary AnimationClip, you're out of luck. It won't play it unless it's already in the animation graph somewhere. One of my solutions has been to just dump all animations on an override layer and call Animator.Crossfade() as needed. This is also hacky.
It would be nice to be able to either add Motions at runtime, or just simply play an arbitrary one. The programmer would need to specify a layer for it, but that's not hard. Or just default to layer 0.

3. This one feels like a no-brainer. The NavMeshAgent doesn't play nice with root motion. I always have to make my movement animations "in-place" and then match the NavMeshAgent's speed so that the character doesn't roller-skate. It would be nice if root motion would work with NavMeshAgents out of the box.

4. The idea of an event graph is great for locomotion and fighting moves, but once you get into anything more sophisticated, such as dialogue and interaction, it breaks down rather quickly. We had a lot of conversation and various actions that NPCs needed to play back. Throw in random idles and random variations on actions and dialogue, and it got ugly. One solution I came up with, which also helped with #2, was to swap out the RuntimeAnimatorController as needed.That way, I could get the control of the event graph, but also the flexibility of handing it an arbitrary animation, and it kept the graph relatively simple. The problem is that I had other components trying to set variables on the graph, and naturally, not all AnimatorControllers have the same list of variables. That generated a lot of console noise, because it would log a warning every time there was a missing variable.
It would be nice to have a "stack" of AnimatorControllers, where you just "push" a new controller on top of the existing one, and then "pop" it off when you're done with it. The existing one would save its state, and transitioning between them would just simply crossfade between the results of the two different graphs. Any variables being set would be directed at the one on top, and if it doesn't exist, it gets propagated down the stack until it either finds one or gets to the bottom. Only then would it generate a log message. That would also make it mesh nicely with AI state machines.

5. Sometimes, you have something you need to do that's so outlandish or uncommon that it would be unreasonable to expect the animation system to support it. For times like those, you just have to roll your own. However, the animation data is in a sealed data structure with no way (at least none that I found) to get to the underlying keys. If the animation system doesn't do something that I need it to do, I can't roll my own.
It would be nice if I had a way to grab that clip animation data on the fly and make use of it in a custom component. That was one of the things I ran into in this past project. Fortunately, we found a workaround, but it did expose this weakness.

 

1. It may have changed since I used it, but the Faceware plugin was clearly written by someone very unfamiliar with Unity and even C# in general. I remember having to rewrite one of the core methods running every update because it was concatenating dozens of strings in a loop using the + operator so it constantly produced truckloads of garbage.

2. I'm one of the people frequently badmouthing Mecanim so you may already be aware, but Animancer (link in my signature) is entirely built around the ability to play whatever AnimationClip you want from any script without needing to mess around in a centralised god object first. The thing that baffles me is that Animancer and Mecanim are both built on the Playables API which allows me to do that, so I can't actually see anything preventing them from implementing it like that too. The only thing stopping me from adding that functionality directly to Animator Controllers myself is the fact that they lock the structure of their playable graph so anything I've tried to get around it either gives an error or crashes the editor (I haven't actually figured out why the Animator.playableGraph property even exists since you can't seem to actually do anything with it).

3. I always just use the agent to calculate a path which my scripts then use to apply the actual movement so I can implement whatever physics acceleration, moving platform handling, and other interactions I need.

4. This is another thing Animancer would make really easy. You could simply reference the desired AnimationClips in your dialogue data or structure it however you think will best suit a given situation. Practically any structure would be better than the "all animations go here" approach forced upon people by Animator Controllers.

5. Being able to inherit from AnimationClip to add your own data would be extremely useful. For example, a fighting game could add data about hitboxes and stuff directly into the animations instead of always needing to store that data separately.

 

I just saw this on Reddit: https://www.reddit.com/r/gamedev/comments/fofdmc/i_created_an_animation_retargeting_utility_its/

 

Have you looked into Playables? They're pretty nifty when you inevitably hit a brick wall where the AnimatorController doesn't work anymore.

You can mix it with AnimatorControllers as well, and I think you can get the underlying PlayableGraph of the AnimatorController, so that should solve most things you're asking about.

But, yeah, not very fond of how Mecanim works. It's got an impressive combination of over-engineering, poor design, poor performance, black boxyness and poor documentation to really just not be that worthwhile. I rolled my own replacement on top of playables and never looked back.

 

I'd start with the Apple Blendshapes (or a superset of those) since that's something that nearly everyone will have a (relatively) easy way to produce. https://developer.apple.com/documentation/arkit/arfaceanchor/blendshapelocation