Still no way to get fine-grained masking with humanoid animations?

I'm experimenting with combining animations with masking for the first time today. I'm trying to use some imported animations that don't apparently include jaw or eye motion. So when I apply this to my model, she deals cards as she should, but also stops blinking and holds her mouth open — she looks like a gaping fish!

Also, I want her to deal the cards sitting down, while the dealing animation I have is standing up.

It should be "animation masking to the rescue" as I let the sit/idle animation that came with the model continue to play on her face and lower body, but override this with the dealing animation on her upper body, arms/hands, and neck.

But I can't get it to work properly, and I think it's because of the problem described here — the "Transform" part of an animation mask is ignored for humanoid avatars, and only the crude "Humanoid" portion of the mask is used. So I can't allow the dealing animation to control her neck and head, but not her face. Either I turn off the whole head/neck portion of the body map, in which case she stares idle at the ceiling while her hands deal cards; or I turn it on, and get the gaping fish.

Is there really no way to get Transform-level control over animation masks when using humanoid animations?

(Note: I'm comfortable with coding, and so if there is a solution that involves doing it from code, please give me a shove in that direction!)

 

I've seen some suggestions that AddMixingTransform can be used to get finer control over animation mixing. But it looks like that would mean throwing Animator and AnimationController out the window, and using the legacy animation system — is that correct?

I'm fine with that if it works, but my main concern in that case is retargeting of animations from one model to another (for example, using mixamo animations, or other mecanim animations from humanoid models from the Asset Store). Would I still be able to do that with this approach?

Also, it seems so heavy-handed, I feel I must be missing something... the whole point of Mecanim is to allow retargeting of animations, but if doing so causes your model to gape like a fish, and there is no way to adjust the animation mask to avoid it, that seems like a major limitation. What am I missing?

 

I remember testing this and got it working, but can't remember how.
Did you try disabling the whole humanoid mask and selecting each bone manually in the transform section?
I seem to remember the humanoid was overriding the transform.

EDIT
I found the test scene, here is how I have it setup.
1-On base layer I have the regular animation.
2-On top of that I have an override layer with a mask with all the humanoid parts red and then I selected only the head and tail bones in the transform section( I had to test tail animation).
3-The override animation is made on the same skeleton of the humanoid rig, so I don't know if it's working when using a different skeleton.

 

Hmm, I can't get this to work. In my test, I have a base layer with the regular animation; and then an override layer, using a mask pretty much as you describe — all the humanoid parts are red, and I selected just the spine joints, arms, and head (but not the face).

But when I run it, I don't see any of the override animation at all — I'm seeing just the animation from the base layer. But when I use the crude humanoid mask, I do see the override animation in whatever parts I select.

So it's working with the humanoid mask; the only problem is that the resolution is much too crude. I want to animate the face independent of the head, and there appears to be no way to do that, at least in my hands.


I also tried a different approach: masking the animation itself, in the animation import settings. I thought perhaps I could make a series of facial animations for various expressions, and then just apply those as an override layer. But no — when I do this, all the joints which I masked out go to the "default" (relaxed fetal) position, and stay stuck there. It appears that the animator controller applies all joints, even those not included in the animation! Argh!

This is really frustrating. I'm having serious thoughts of throwing Unity's whole animation pipeline out the window and rolling my own. Or, having several invisible avatars playing the different animations, and then mixing these myself onto a visible model in LateUpdate. But either one seems stupid. Or I could switch from Humanoid to Generic, but then I lose the ability to use IK (unless perhaps I use a 3rd-party IK package), and will probably have a hard time making use of animations from different sources (such as Mixamo). Surely I'm missing something? Is there really no way to override just the face transforms (for example) when using humanoid animations?

 

Huge mistake, as mecanim is one of the absolutely brilliant spot on AAA parts of Unity, it just needs to be handled right. There's 2 kinds of animation: Generic and Humanoid. If you want to blend an animation on top you need another "layer" and this layer receives a special avatar mask, that will mask out the bones you want to effect.

 

I think the transform mask doesn't work with normal retargeting, so you must have the exact same bone hierarchy for the masked bones. Probably yours is different.

 

Thanks for jumping in, but please read the rest of the thread. The whole point is that, with Humanoid animation, the masking available is woefully inadequate.

In this test I used the very same avatar to create the mask as for the model itself, and it still didn't work. It only worked when I switched from Humanoid to Generic — could that be what you did in your test, too?

 

No, I am using humanoid.
Anyway the point is not what you use to create the mask, but what is the skeleton of the animation you are playing in the override layer.
You should check the skeleton of that animation and see if it match with the skeleton of your character. It must match 1:1 at least until the point where the bone you need is.
For example if you need to override the spine you can have :
hips-spine-chest in your character
and
hips-spine-spine2 in your animation and you will have the hips and spine overridden (if selected in the mask) but not the chest.
if in the animation you have hips-spine1-chest, only the hips are matching, because chest is child of spine1 and not spine as your character(not 100% sure of this second case,I never verified, but should be like this).

 

I'm virtually certain that both animations (and the mask, for that matter) were all made from the same model. But if it's working for you, then it's worth triple-checking. I'll make an even smaller test project that only has one model in it, and see if I can get it to work there. I would love to find some conditions under which what you're describing actually works, because so far, everything I've seen indicates that the Transform mask is simply ignored when using humanoid animation.

 

OK. Here's what appears to be happening:

Any bone which is mapped to a Humanoid slot can be controlled only by the "Humanoid" section of an animation mask. The Transform part of the mask is irrelevant to these bones.

Any bone which is not mapped to a Humanoid slot is controlled by the Transform part of the animation mask.

I took great care in this test; I actually dragged an avatar out from the model, to another folder, and named it TestAvatar (it was written out as a model, which contained TestAvatarAvatar, but whatever). Then I made sure that the model itself, the animation I'm testing with, and the mask were all using TestAvatarAvatar, and all set to Humanoid.

The result: when I turn off everything under Humanoid in the mask, the model is frozen in the relaxed-fetal position... except for the eyelids. The eyelids continue to blink, as long as those transforms are checked in the Transform part of the mask. I believe this is because the eyelid joints are not mapped to Humanoid slots. When I uncheck the eyelid bones in the Transform section of the mask, then the blinks stop too.

Conversely, for any bone that is slotted into the Humanoid part of the avatar config, checking or unchecking it in Transform makes no difference whatsoever. These only respond (in groups I have no control over) to the Humanoid portion of the mask.

I've been thinking of "Humanoid" and "Generic" animations as two separate things, but I guess really they're not — every Mecanim animation has 0 or more bones slotted into the Humanoid framework, and 0 or more bones which are not. "Generic" is just the extreme where there are no Humanoid bones at all, but in a Humanoid animation, there may nonetheless be some bones (like the eyelids in my test) which are not slotted, and so don't suffer from all the limitations (or benefits) of Humanoid animation.

So I guess this could work fine for something like a humanoid character with a tail or wings... these joints would simply be ignored by the Humanoid system, and so you could mask them as you please at the Transform level. The problem only arises when you are trying to have finer control over bones which are also part of the Humanoid system, e.g., layering control of the jaw. But I may be able to just unslot the jaw and other face joints, and deal with those in a Generic way, while continuing to use Humanoid animation for the body and limbs (for which Humanoid's crude masking is probably good enough).

Still not what I would call an AAA solution... but possibly an adequate work-around, now that I understand what's going on.

 

You are right, I just tested and it's working only for non humanoid bones. I was also testing head scaling for some cartoon effect and I thought it was working on humanoid bones, but that's probably because humanoid doesn't do scale animation so it wasn't overriding that.

I noticed you talk about the character stopping moving, that shouldn't happen. It is better if you apply the mask only on the upper layer, and not on the base layer, it's easier to work this way.

Few more tips:
1-You can control the base pose when defining the avatar, if you close the mouth when setting the t-pose on the avatar it will default to mouth closed(no fish effect).

2-Since all you need is some simple blinking, why don't you do it by script?
It will be even more realistic. On lateupdate start a rotation every tot seconds on the eyes bones.

3-For the sitting, On base layer put the dealing animation, on the override layer put the sit animation with a mask with green chest and legs only. It should do the trick. You can also try using additive and experiment with layer weights.

 

I remember @TonyLi detailing a proper workflow for this setup - humanoid rigs (with bone controlled expressions)
So a complete character can be retargeted and still have facial rigs. He has also explained in several posts what is needed to go about 'retargeting' on generic rigs.
I looked for a while but couldn't find the exact thread. Might help to search his posts.
An alternate process could be to use morphs/blend shapes instead of bone controlled facial expressions. This would remove the need to mask the facial bones.

 

I use FaceFX. The Unity - FaceFX integration scripts use legacy animation. I use Mecanim Humanoid for the body and head (including IK), and legacy for the face. So ultimately I'm doing what Joe described -- controlling bones that are outside the Humanoid rig's definition. I don't assign the Jaw to Mecanim; I leave it under legacy control.

 

Sorry to necro this post, but it's high in the search results for people with this issue and never came to a satisfactory conclusion.

If you want to use a seperate facial animation from your body animation, here is an easy solution:
1. Open the Avatar your Animator is using. (click it in project folder > Configure Avatar in the Inspector)
2. Click the Head button (found in the left corner of the main image of the avatar)
3. Set the joints you want to use for facial animation to None
4. Create a separate animator for your facial rig

The downside of this method is you have to use 2 animators, which means calling animation state changes in two locations as well as I assume more overhead. However, its the only solution I've been able to find. If someone has a better method, please chime in.

 

Unity, please give us the ability to add custom bones checkboxes to a Humanoid Avatar Mask.

 

Any news about this request?

 

2011 and now 2018..come one Unity, give us the features we really need. We don't need to be able to deploy to a billion devices, we need fine grained control over the engine, which we just dont have.

 

also interested what would be the best solution. In my side i use faceware and i cannot support head tilt or movement

 

Oy, so this is the limitation of mecanim, eh?
Yup, generic rig and my faces animations show up.
If I use humanoid rig, I have to do some sort of magic and layers and masks, which I am trying to avoid because those hurt performance, and I am making a RTS.
Bummer.
Maybe I will just do generic, as I find most pre-made animations form the asset store to not be what I need, anyway.

 

is this ever fixed?

 

I'm not sure — I've moved on from the project where I needed this — but I might suggest looking at the Animancer plugin. I use that in most of my projects nowadays, and it does a good job of fixing or working around most of the boneheaded limitations in Unity's animation system. It performs well, and I also find it substantially easier to use.

 

Animancer won't solve this problem out of the box since it uses the same AvatarMasks as the Animator Controller system, but it supports animation jobs which should make it possible to implement. One of the Animation Jobs Samples is a weighted mask mixer which would let you not only mask individual bones but give them individual weights as well. I haven't tried using it myself, but I'd be happy to work with anyone who has a real use case for it.

 

I can't believe that we are in 2021 and such a simple thing can not be done in Unity. How can you make a serious production if you can't have a mask for the face bones only (without having to mask the head too). Does someone from unity read these post? 8 years and still no solutions. Really?

 

In the meantime I use UMotion Pro to copy blend shapes from one animation to the other but revised scripts is a common thing in the animation chain. replacing one facial motion capture with another revised one should be an easy process. Not rocket science...