Mobile blend shape performance

I'm currently developing a mobile game in which I am aiming to make use of an array of blend shapes to control a characters facial animations.

In order to attempt to make this feasible on a range of different devices, I have done my best to keep my game scene and workings as simple as possible. My current scene has only around 12 draw calls and I am making use of simple mobile shaders for character and environment materials. I have also tried to optimize my character model as much as possible without losing too much detail.

Once this character is implemented into my game scene however, I am finding that my average frame rate (tested on multiple devices) drops from around 50+ fps to below 20. I would really like this to be a feasible option for facial animation but it isn't looking like one at the moment, despite my best attempts to reduce the complexity of the scene and game workings.

I'm wondering if anyone has used blend shapes within their game before (ideally mobile-based), and if so do you have any tips that could help me to increase my games performance. I have a fairly high number of blend shapes on my character, so I am thinking that perhaps this might not be an idea option for mobile devices.

I've searched through the forums and have found a few posts related to this, however unfortunately I'm still a little unsure of whether blend shapes are suitable for use on mobile devices.

Any help will be very much appreciated.

 

We’ve used blend shapes extensively in Secret Oops! for Apple Arcade, baseline hardware = iPhone 6s.

The cost of blend shapes is negligible, we did some stress test with a a great number of complex characters on screen with no problem.

Are you sure the frame drop are related to the blend shapes and not on something else?

 

We developed Lost Echo on iOS and Android. We have one character that has blendshapes (sort of by mistake... ) in the released version and that works fine on devices as old as an iPhone 5.

For our upcoming game where we make more extensive use of blendshapes, we haven't noticed any performance impact from blendshapes (we too have a baseline of roughly iPhone 6s, but I suspect that blendshapes don't really cause any issue even for older devices).

If your models are mobile appropriate (no crazy vertex counts), blendshapes should be pretty fast.

 

Thank you both for your replies, I really appreciate it and it's encouraging to hear that the use of blend shapes didn't have a significant effect on the performance of your games.

I've currently been testing a character model that is made up of 7097 vertices and a simple mobile shader. This model has 52 blend shapes in total. It seems that as soon as I trigger a blend shape animation the frame rate drops greatly, which is what led me to believe that the blend shapes are the cause of this issue.

The animation data itself is captured using facial motion capture data, so these are fairly complex in terms of the total number and frequency of blend shape keyframes.

Do either of you happen to recall a rough estimate of a model vertex count / number of blend shapes that you may have used?

I'm going to carry on testing today and will also check out both Lost Echo and Secret Oops! today.

Thanks again for your help.

 

@nicg220

Hello Nic,
I'm experiencing a very similar behaviour for our Quest 2 mobile game/interactive story (no link cable used). In our case though, we use a very high poly face (30K polygons) for emotional expression purposes. With a fair bit of optimization, I can get our game to run at 72FPS with 2 such characters on screen, proper lighting and limited transparency for some FXs.

However, as soon as we enable the animation (40 blend shapes), we can see a drop of about 10 FPS but it's extremely erratic. It appears to be tied to the density of the key frames in the animation but it's a bit difficult to say if it'S related. See the screenshot below with a hand drawn impact of the animation on framerate. We haven't looked into our animation export options yet but we expect the keyframe density to be quite high as we will be motion capturing our animations. Using a test mocap data and Faceware, the drop is closer to 25FPS bu it wasn't cleaned up and there is a lot of junk in that animation.

It's akward because it's the GPU that takes a hit, not the CPU and I wasn't expecting that. Have you found anything with regards to blend shapes performance that you can share?

Thanks

 

Hi LeRondPoint,

Unfortunately I can't offer a solution to you I'm afraid.

I've managed to improve the performance from my initial testing by reducing the mesh complexity, however am still working on trying to get the best performance with blend shapes.

I have also been using facial motion software to record the animation data, and these recordings output animations with very dense keyframes, so I can imagine that this may indeed be a part of the problem.

I'll make sure to post any updates if I do come across anything that might be of interest however.

 

Thanks for the update Nic,
I ended up removing the "Compute skinning" option in the player options because, in our case, we have a fair bit of CPU headroom.

 

Recently come across this as im working on optimising a Unity Dance App. Consists of 10 Dancers with 52 Blendshapes each. We captured the Facial expressions using motion capture even running 1 character on mobile with 52 blend shapes sees a huge drop in FPS on certain devices. Specially Andriod even on devices that are 2018 onwards.

52 Blendshapes i beleive now to be way too high for 10 characters on the screen at once, 520 blendshapes triggering at once is not the way, is there any other techniques for optimising blendshapes more? Keyframe reduction didnt seem to do much.