Mapping a user's face to an avatar model in Unity

Hey, guys,

I have a half-finished project that I made for a client that didn't go trought.

It's a project for Unity mobile, where the user uploads a picture of himself, and
maps to a 3D avatar's model face.

I made a blog post about it here: http://danielfortes.com/mapping-a-users-face-photo-to-avatar-in-unity/

You can find the sources and an apk binary there, if anyone's interested.

See ya!


www.danielfortes.com
www.twitter.com/danielfotres

 

I have a question: Can this piece of software detect points automatically or do you have to click them yourself? If the latter, how fast detection algorithm is so I could use webcam input (after porting it to desktop platform) to achieve FaceRig-like effect (by detecting points automatically at 60fps and moving face bones accordingly) so e.g. in a multiplayer game with voicechat character's mouth would be synced with audio if user has a webcam.

 

That can be done, but it's a little bit more complicated... You need to do some image/face recognition stuff.

First you need to have a face detector algorithm, and then a facial feature detector.

I think this can be done using OpenCV, which has a a wrapper for Unity: https://www.assetstore.unity3d.com/en/#!/content/21088

Here is a facial feature detector using OpenCV: http://cmp.felk.cvut.cz/~uricamic/flandmark/, also https://github.com/cxcxcxcx/asmlib-opencv, but they are in C++.

So it can be done... don't know how complex it would be. The C++ versions seem to be able to do this in real-time... I think the Unity Wrapper may be able to do it to, just don't know how it will play with doing this along with a game.

 

Hi got some errors.. i download the project sources and open in Unity editor in Wndows and play .. i load the photo but after click done i cant add points in the image as show in video.

 

faceunity do this job,but it's not free