Question Facial animations: Jaw bone giving me trouble with blend shapes. Avoiding double motion

I have a character model that has a jaw bone along with blend shapes for facial expressions. If I move/rotate the jaw bone, the face's mech goes with it:



The mesh also has blend shapes. Here's the mouth opening via blend shape, without rotating the raw:



This blend shape doesn't look great on its own, because it doesn't move the jaw/teeth. So, it seems I need to both move/rotate the jaw and adjust the blend shape. However, because moving the jaw also affects the mesh, it ends up kind of doubling how much the facial mesh gets moved:



These blend shapes appear to be the default provided by Character Creator characters, so I don't think they were rigged "wrong". But I'm not sure how I can effective animate the character talking. I can't just move the jaw, since the lip sync software I'm using uses blend shapes. But I can't just use blend shapes, since the teeth with remain completely still.

Does anyone have experience with rigged models using blend shapes as well as a jaw bone, and getting these to play nicely together?

 

I believe the facial animation [at least from iclone] works by combining the jaw movement with the blendshapes. I have facial animations working on Character Creator characters, but I had to create a 2nd layer in the animator with an Avatar Mask applied to the head only, and blending set to override. Again, this is for a Humanoid character playing facial animations exported from iclone.

I would think a lip sync software would use a similar setup, but the authors of the software would know best.

 

That makes sense. But I guess that only works if whatever recorded the animations was also recording the jaw rotation/position change. In my case, all I'm getting is blendshape values.

I ended up realizing that I could just create blend shapes for the lower jaw itself. So, when the face's Mouth_Open blend shape is used, I'm also setting the jaw's Mouth_Open blend shape as well. This just moves the verts of the jaw, it doesn't rotate the jaw bone, so I don't end up with the double accounting on the face mesh. This feels a little cludgey, but in my case I don't have an easy way to recording the jaw position/rotation into my clips, so this will have to do.

 

In case anyone else runs into this (and so I don't forget how I did this in the future...) I made a quick video on how to get the jaw bone transforms to be incorporated into the shape keys. The end result is that the jaw can fully open just using shape keys, without the need for any extra code to move the jaw bone itself.

 

The problem is blendshapes are separated by different meshes/objects. So you have to animate multiple blendshapes I mean on different meshes actually

 

That's not the issue. Animation multiple meshes with the same shape keys is no problem. The issue here is that mouth animations (exported from tools like iClone) assume that you're both animating shape keys and moving the jaw bone transform. If you only do one or the other, you get an incomplete-looking animation.

The solution I described above (on possible approach) is to "bake" the jaw transform movement directly into the blend shapes, allowing you to only set the blend shapes. (This is useful because there are several cases where eternal tools only set shape keys, and they don't allow a simple way to also animate the jaw bine transform.)

 

You need to set the jaw.open blendshapes as a corrective shape. What you're seeing here are double transforms. I don't have any apps open, but the first thing I would do is open your character's jaw, do any kind of weighting, and then build a corrective blendshape for it.