Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Unity 3D + faceAPI

Discussion in 'Made With Unity' started by Raptor Claw, Nov 19, 2010.

  1. Raptor Claw

    Raptor Claw

    Joined:
    Jan 24, 2010
    Posts:
    11
    I became interested in incorporating the face tracking library faceAPI with Unity3D. Looking around I found tons of videos showcasing the connection but it was difficult to find any documentation, and some tutorials even required people to purchase a UDP script for Unity. I've just finished writing up my own tutorial with all the source code UDP script and all.

    You can find it on my blog at: http://www.andysaia.com/radicalpropositions/?p=197

    I hope somebody gets some good use out of it, any and all comments are appreciated.
     
  2. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    935
    Thank's a lot Raptor Claw! :)

    Your tutorial is really easy and usefull!

    I found only a little stability problem: when I don't move my head camera still move a little bit up-down and right-left; but I think that is a faceapi problem with my laptop webcam.
     
  3. the_gnoblin

    the_gnoblin

    Joined:
    Jan 10, 2009
    Posts:
    722
    thanks!

    I suppose this won't work in webplayer?
     
  4. Raptor Claw

    Raptor Claw

    Joined:
    Jan 24, 2010
    Posts:
    11
    @Kenshin
    Yes, there is a little bit of shake to it, as you correctly guessed the majority of the blame is given to the faceAPI streamer application, but certain factors like the lighting in the room or quality of the webcam play a role as well. Still I think there might be something that can be done inside of Unity to round the numbers in a way to prevent some jitter.

    @gnoblin
    No, currently it won't work in the webplayer. I have been thinking of some possible solutions though. One idea would be to have a web interface send the webcam data to a back end server that does all the calculations and sends the position and orientation information to the unity application. I think this would probably be really difficult if not impossible to do in real time but maybe somebody is smart enough to do it.

    The other idea (and probably more practical) is to build a faceAPI plugin for unity using .dlls. The company that makes faceAPI (http://www.seeingmachines.com/) is currently working on porting it to C# which may make this process easier still. If anybody has any better ideas than these I'd be interested to here them.
     
  5. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    935
    Raptor Claw, thank's for your answer.

    A c# porting will be very very interesting.

    Do you know prices for commercial version of faceApi library?
    Are you using this for any real project?
     
  6. Raptor Claw

    Raptor Claw

    Joined:
    Jan 24, 2010
    Posts:
    11
    I'm not sure how much the commercial license is. I've been planning on making something similar to this in unity
    http://www.youtube.com/watch?v=Phcdp7AjhBg

    The non-commercial license doesn't support lips and eyebrow tracking so I will have to fork over the doe eventually.
     
  7. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    935
    Really interesting! ...for manage advanced tracking features I was thinking about commercial license.
     
  8. Wolf Dreamer

    Wolf Dreamer

    Joined:
    Sep 2, 2009
    Posts:
    142
    Last edited: Nov 22, 2010
  9. Raptor Claw

    Raptor Claw

    Joined:
    Jan 24, 2010
    Posts:
    11
    The most important thing to keep in mind when buying a webcam for tracking is the frame rate. A crappy webcam would probably work but If you want to have your game running at 30 fps and the webcam only captures 12fps you might have some choppy tracking. I would recommend the PS3 camera(http://www.amazon.com/PlayStation-Eye-3/dp/B000VTQ3LU) It's not too expensive and is capable of a whomping120fps, which is pretty incredible for a webcam.

    You can find the driver for it here: http://thecodelabs.com/downloads/
     
  10. Wolf Dreamer

    Wolf Dreamer

    Joined:
    Sep 2, 2009
    Posts:
    142
    All webcameras have negative reviews.

    I read that the PlayStation Eye is only meant to work for PlayStation, not a personal computer, and you need a PlayStation to get the software.

    The site you link to says that software is only free software for 30 days.

    If I get a different webcamera, i won't need to buy that software to make this all work?

    Just need the free software at http://www.faceapi.com/Download/login.php?requires_login=true&page=/Download/downloads.php&query=

    If you release anything you have to give all the source code away free if its with the free version of their software. And they refuse to tell you how much it cost for a commercial license, or whether they'll let you distribute anything you make with it at all. http://www.seeingmachines.com/product/faceapi/licensing/

    Any other software out there that does the same thing?

    And for a webcamera, I only need it to see the whites of my eyes, which should stand out rather well from the rest of my face. Don't care about using it for any other reason.
     
  11. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    935
    I think you are wrong because only the product "CL Studio Live" require payment.

    PS Eye PC driver are free and you can connect this camera to your pc without any problem.
     
  12. Wolf Dreamer

    Wolf Dreamer

    Joined:
    Sep 2, 2009
    Posts:
    142
  13. shabazzster

    shabazzster

    Joined:
    Mar 17, 2010
    Posts:
    104
    This is Marvelous now lets get it ported over to the Mac please?
     
  14. okimoki

    okimoki

    Joined:
    May 13, 2010
    Posts:
    116
    Hmmmm.. when i try to open your project in Unity 2.6 it says that it was created with an older version of Unity.. and when i open the scene Target it is empty, so are all the prefabs etc.. any ideas?

    EDIT: it seems that Unity3 opens the scene but FaceAPICamera and FaceAPIController have missing scripts.. what scripts need to be attached to what?

    EDIT AGAIN: Never mind!!! Got it working in 3.1.. just needed to reatach scripts and textures.. scripts are named quite logicaly :) This is awesome, just need to do some anti-jittering-thingie to make the movement smoother, othervise its really amazing!!!!!!! :)

    Thanks for this, Raptor Claw
     
    Last edited: Dec 1, 2010
  15. priceap

    priceap

    Joined:
    Apr 18, 2009
    Posts:
    274
    I've been using the facetracking library for Processing (processing.org), and sending the tracking data using the osc .net implementation found elsewhere in the forums. There is a mac version also. Pretty straightforward and all free.
     
  16. 3Duaun

    3Duaun

    Joined:
    Dec 29, 2009
    Posts:
    600
    good ole' processing, so MANY MANY tools. I've been looking for ways to integrate it better into Unity. Would anyone have any suggestions on which OSC implementations are best for Unity?
     
  17. elbows

    elbows

    Joined:
    Nov 28, 2009
    Posts:
    2,501
    Im not sure which is best, there are several different implementations for Unity that have been shared on the forums, although searching for them can be tricky as OSC is too few characters for the search engine I think.

    Personally I used a modified version of OSCuMote:

    http://forum.unity3d.com/threads/21273-OSCuMote-Wiimote-support-for-the-free-version
     
  18. 3Duaun

    3Duaun

    Joined:
    Dec 29, 2009
    Posts:
    600
    @elbows
    Thanks a TON for sharing the OSCmodule link. I've encountered the 3 character search term limit, so this helps out a lot. Do you use VVVV too?

    Time to see if the data from the Kinect can be gotten into Unity ;-)
     
    Last edited: Dec 13, 2010