Bug [resolved] Question regarding blendshape controls for character facial expressions

Hi Everyone,

The facial expressions of our character models are controlled using blendshapes.

Each time we want to apply an expression, a function is called, which prompts a calculation to determine the difference between the blendshape controllers initial values, and the target expressions required parameters. We then apply the values that are different, and leave the other values untouched. This runs every frame, but when there is no request received, the values are left as is.

However, we have recently encountered a problem where unchanged blendshapes seem to revert back to a specific value. Due to this, we are encountering a bug where, after applying an expression, values which should not have changed, are for some reason updated, thus resulting in the wrong, or a broken expression.

In addition, when we tried disabling the script, we could not move the sliders of the blendshape via the inspector UI. We assume that this is due to the aforementioned bug where certain parameters seem to get stuck at non initial values.

Therefore, I have some questions regarding the control of blendshapes, and the correct ways to go about doing so.

- If I don't change the value in the script during a frame update, is it correct to assume that there will be no change? Or will it try to return to a particular value?
- If the latter, is there a way to disable this behaviour?
- What might be the reason for the value being updated when no change is made?
- Is SkinnedMeshRenderer.SetBlendShapeWeight(..) the correct function to update a blendshape via a script?
- Are there any other ways to update one of these parameters?
- Is Update() the correct place to execute blendshape control logic?
- Is it perhaps required to set each parameter every frame even if I don't need to modify it from its current value?

Kind regards.

 

Hi All,

I'm being pressured to solve this problem ASAP, but I haven't really made any progress on my end.
I would greatly appreciate if someone could offer some advice, a hint, or even some insight into a past experience that might be of help.

Kind regards.

 

What version of unity are you using? With 2020.3 I have no issue, but I am mostly setting blendshapes few times to define a character face look.
Yes that is the correct method and there is no need to set the weight every frame if the value is not changing.
If you set it every frame of course you can't change the slider, and if you cannot change the slider it means you are still updating the weights, check for issues in your code. Try removing the component at runtime instead of disabling it, maybe you have some method still running even disabled(some monobehaviour methods still works, can't remember which).

I'm not sure because it has been a long time since I tested it and if I recall it was pretty limited with humanoid retargeting, but I think you can also drive blendshapes with the animation, check if there aren't frames setting blendshapes in the animation. Again not sure of this, you can see the animation window to see if there are values regarding the blendshapes.
I made it work some year ago but cannot remember If I was using a null transform to export the blendshapes values.

 

Thanks to your advice, I was able to identify the cause of the problem for now.
It is the inclusion of blendshape control in the animation.

As for the Unity version, I'm using 2021.1.12f1, but it may not matter much since the Unity version is the same as the one I used when I created the control script for the expressions.
At least when I created the script, it looked fine.

The problem is not solved yet, but we can discuss how to deal with it so we can move forward.

 

If the animation itself set the blendshapes your only chance it to override it every single frame. Try setting it in the LateUpdate method. It may have a performance hit since you set it every time. Otherwise remove the keyframe driving the blendshapes from the animation.

 

Based on the feedback received in this thread, and after discussing the issue with the 3D artist that created the models, we decided to tweak the implementation so that the value for each blendshape is set on a per-frame basis, including those that were unchanged. Fortunately, this did not impact performance in a significant way.

We also changed the code to execute during LateUpdate instead of Update.

Thank you all for your help with this issue!

 

Hi there,

Like @00christian00 said it the expected behaviour of the animator

if the skin mesh is driven by the animator animation system and you have some animation curve driving thoses blendshapes, we will write every frame all animated blendshape values regardless if the current state is playing a curve for thoses shape(we write the default value in this case). This is the Animator WriteDefault behaviour.

there is two options for you:
1. Like you already did you can write every frame in a LateUpdate(after the animation system) all blend shapes values correction.
2. Write an animation jobs that does write blend shape correction directly into the animationstream.

#1 is the easiest and will give you good result if the work load to compute the blend shapes is pretty small which seem the case for you.
#2 require more work to setup but if the work load is large, you can benefit from the animator multithread jobs support to balance the work load.

Unless you get perf issue I would keep solution #1 for now.