I could use some help understanding Animation Layers

I have an InputController and an AvatarController. The former registers all the input and the latter contains all the animations. For that I used a simple base layer with animations such as idle, walk, run, jump, etc. Recently I added some IK to this model which (to my surprise) worked quite well out of the box. The IK nicely overrides any animations. I was planning to use the IK system to make the character hold things, such as weapons or a lantern. Maybe even aim a weapon along a vector.

The only problem I had with this is that the original animations determine whether or not the character has it's hands open or closed: I thought fixing this with a new Layer in my Animator. I couldn't get this to work, checked the docs, but couldn't find the cause / solution.

I've added some animations to a layer that only contain keyframes for the right hand fingers. When I give them a weight of 0 nothing happens, but at a weight of 1 it will also affect properties that aren't in the right hand: making my humanoid Avatar "float" in the usual bugged Avatar fasion. Anything in between 0 and 1 seems to blend into the base layers. So at 1 the right hand works but nothing else does, and vice versa at 0.

I've used the Additive setting on the right hand animation layer (Edit: this was wrong and I changed it to Override) and applied an Avatar Mask on it that contains only the right hand. I've only added keyframes for this hand's fingers. For both layers I've toggled the IK pass but that doesn't seem to affect anything so far.

I might be a bit confused on how these layers work exactly. My goal off course is to just stop the fingers from animating when the character holds an item or a weapon, in that case I would just want to instruct the right hand to be open or closed from code. In this approach I just have a bool property for the right hand on the Animator Controller.

PS. When I was experimenting with animators I added one to the characters spine to see what would happen if I animate it. Animators added to a specific part of the Avatar seem to overwrite whatever was in a parent animator, but to me it's unclear whether this is intended behaviour or just a hack. What I am looking for is a pragmatic way to do this, but also a sane one if possible.

 

I haven't used layers much, but I think you should be able to do this by putting your hand animations on an override layer (not additive) and using an Avatar Mask to have that layer only affect the hands. You mentioned that you tried it with an additive layer, but you don't want the hand animations to add their values to the other animations, you want to override the other animation values for specific bones.

 

Thank you. It doesn't seem to work though. Im testing this now on a project at work. Same idea. I'm seeing the Animator change states properly on both layers, I just don't see the keyframes of the right hand layer being applied.

Just for accuracy: the mask should be applied to the right hand layer right? And within this mask, everything should be red except the right hand, correct?

EDIT: got it Animation Layers working in a different prototype using the same method but a different body part (the head). This avatar is not humanoid though. Could this make a difference?

 

Yeah, the layer with that mask will affect green parts and not red ones.

Are you using a Humanoid Rig? The Humanoid section of the mask only works for Humanoids. If you're using a Generic Rig you need to use the Transforms section instead.

If that doesn't work, can you get a mask to work with something else like say playing a walk animation on the base layer and having another layer masked to only the upper body to attack or do some other action?

 

I am indeed using a Humanoid Rig and configured the Humanoid section of the Avatar Mask to just use the right hand. The right hand is green, the rest is red.

At work I opened up an old prototype with a Generic Rig and gave it an animation for just the head. The transforms are toggled on by default, and work out of the box. Toggling them off prevents that animation from playing, as expected. Now for the other way around

I'll try once I get home. I guess the difference would be that both animations would contain keyframes for all the parts, while in my current setup I just have keyframes on the fingers.

 

This is how I'd do it:
- Set the hand layer to override, weight 1. If you ever need to not override the hand animations, move it's weight down to 0.
- Set the avatar mask from the same model that you're animating, and mark just the hand transforms. This will also work for humanoid models! The requirement there is that the transform hierarchy matches all the way down, both in the mask, in the rig in the scene, and in the animation clips. This is only a requirement for the hand animation, not for the animations you play on the base layer, they can be retargeted.

You can use additive, but adding together finger animations from two states sounds like the recipe for very strange looking hands.

 

Thank you for your reply! I've done most of the things you suggested prior: the hand layer is set to Override and it's weight is 1. Just for clarity, could you elaborate on what you mean by:

Do you mean by that the Avatar Mask should be created by having the model selected at creation, or do you mean that the "Use Skeleton From" option should be set to the correct Avatar?

OK, thanks. I noticed the Avatar goes into it's fetal position (I do not know what to call this pose) because I only had keyframes for the hand, probably. I'll try what Kybernetic suggested to test the mask and work from there.

 

Want to know what turned out to be the solution?

Clicking on the small circle below the avatar mask's graphic fixed most of the issues. I guess it means I've deactivated the root, that was overriding the base layer? I completely didn't catch on it was a clickable element!

Clicking on the icons to deactivate IK on the masked layer made it play nicely with the IK implementation. The mask now only controls the fingers, and doesn't affect anything else. Base layer controls the hands position and rotation (exept when IK is applied), exactly what I wanted!

Thanks for pushing me in the right direction! I can't explain how happy I am with getting to grips with layered animations, humanoid avatars, and IK. This is a serious gamechanger for me!

Update: I managed to implement both the left hand, and create a "whatever" state for both hands that just follows the base layer after the open animation has ended. So whenever the characters hands are free and it sprints, it still balls up the fists. Very happy with this progress!