Facial Animation Suggestions Please!

Hello, I'm hoping to create a realistic character system (10 characters, up to 5 onscreen at once with minimal scenery) for use in a higher-end iOS/Android app. The characters want to have basic facial animation (lip-syncing and basic expressions) as well as rudimentary idle/walk/sit animations. They are intended to be as "real" as current tablet devices can deliver.

I've started to research some options but I'd really appreciate opinions on the best way to go about creating such a system as the last time I did something like this was before Mechanim existed and it didn't include facial animations.

Some of my questions are:

1. Should I use bone-based or blendshape (probably with Megafiers) )animation for the face? Many people suggest bone-based, but:
a) Unity still suggests 30 bones per character for mobile devices. That would allow for maybe 10 bones in the face - is this enough for the facial expressions required?
b) Would it be a problem that each vertex can only be influenced by two controlling bones on mobile devices?

2. I could wait until Unity 4.3 is released and see how it natively uses blendshapes...but is there a date for this?

3. Are there any good references for creating bone-based facial rigs/blendshape sets that you can suggest?

I'd welcome any other info and experiences people have had trying to do this. Some tools I've been looking at that might speed things up include iCLONE, FaceFX, MakeHuman, BonyFace, Megafier Blend Shapes and Mixamo FacePlus...if you have experiences with these I'd really like to know!

I'm interested in opinions on the whole process - Maya/Max, Blends/Bones, whether Mixamo auto-rigging is good enough...I hope I'm not asking too much

 

I have experience with FaceFX and Mixamo.

FacePlus currently does cartoony facial animation better than realistic, but it's really easy to get working. If you take full advantage of Mixamo's mocap library and characters, the cost of All Access is actually a really good deal, and they have fantastic support. FacePlus might be ideal if you plan to have a large number of custom facial expressions, and if you have a good face actor to record.

FaceFX plugins are also a good deal. FaceFX Studio is a little pricey, but depending on your budget and needs it might be better for your workflow. FaceFX has a lot of good batch-processing tools for quickly chomping through lots of lip sync. The FaceFX Default Character Setup page is a good reference for rigging face bones, as is this tutorial.

FaceFX is pretty much an industry standard. If you go with it, get the integration project here. In brief, you'll control your character's body using a Mecanim Animator, and you'll control the face using a legacy Animation component.

Another really nice feature of FaceFX, especially for mobile, is that it generates very compact curve-based animations. You'll get something like an 80% savings in animation size versus traditional keyframed FBX.

To get the most out of your limited bone budget, you should build a custom rig. If you don't have a good 3D person in-house, post in the Commercial: Job Offering forum.

Hope some of this helps!

 

Thanks TonyLi that's a really helpful answer.
I don't know if you saw the Unity 4.3 FacePlus demo, but I wondered if you knew much about the real-time face-tracking, which seemed to be from a camera or kinect. Basically I got excited at the prospect of having avatars that copied player's face movements and I'd like to know if this is actually something Face Plus does and whether it would run in real-time on tablet PC's. I also wondered if FaceFX had anything that did this.
Thanks for your post.

 

You might be able to license FacePlus from Mixamo for a (probably very substantial) cost for play-time face-tracking, but the All Access offering of FacePlus is for design-time face-tracking. It will record animations that you can later add to your characters. It requires a Windows 7/8 machine with 4+ GB of RAM, so it wouldn't be for tablets.

FaceFX is different. It's not for real-time tracking. You provide a face rig with poses for various mouth shapes. FaceFX then analyzes audio recordings and generates animations that make the face rig look like it's speaking those recordings. Once you've set up your face rig, you can give FaceFX, say, a thousand lines of dialogue and tell it to generate animations for each. It can also automatically inject head movement and blinking to make the animations look more natural.

Free demos are available for both products, so try them out and see what works for you.

There's also a Unity product called Adventure Creator that claims to do lip sync. I don't know anything else about it, except that the demo looks pretty polished.

 

Hi Yusefkerr,

A new solution just became available.

For this solution, you will be able to get realistic 3D-face-models, without the need of creating either bones or blend-shapes.

In addition, animation is built-in with the 3D face models, i.e. once a 3D model is created (from a photo), you can instantly get the 3D animation, including expressions, mouth-shapes and their blending.

You can create unlimited custom-built 3D-face-models, and you only need Unity, i.e. no other software is needed.