Search Unity

  1. All Pro and Enterprise subscribers: find helpful & inspiring creative, tech, and business know-how in the new Unity Success Hub. Sign in to stay up to date.
    Dismiss Notice
  2. Dismiss Notice

[RELEASED] Puppet Face - All-In-1 Facial Animation for Unity

Discussion in 'Assets and Asset Store' started by jamieniman, Dec 17, 2020.

  1. EddieChristian

    EddieChristian

    Joined:
    Oct 9, 2012
    Posts:
    723
    I bought this when it first came out and I finally have a project where I get to use it. But I just wanted to verify it was capable of doing what I need before I start. I just need an application that uses the webcam and tracks a person face. I realized that maybe this tool only did this in the editor and thought I would ask if it can do this in builds as well.
     
  2. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Is this in the demo scene? Could you try disabling and then reenabling the BlendShapeManager component?
     
  3. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Are you able to send me the BuyCookies.xml it created?
     
  4. BarnimPL

    BarnimPL

    Joined:
    Nov 22, 2020
    Posts:
    6
    Sure.
     

    Attached Files:

  5. thatnzguy

    thatnzguy

    Joined:
    Jun 8, 2015
    Posts:
    13
    I had a few difficulties with the blendshape editor.

    Some notes or ideas:
    • The tool values keep resetting. Values are the same per tool, it would be nice if they were separate.
    • If the radius is ever 0 or -0 then the face usually breaks right away (typing in 0.02 requires typing in 0.0 first..)
    • Mirror is misaligned in RP_Eric model
    • Pressing Ctrl-Z for undo also triggers the use of the automatic modifier Ctrl, which is either Reset or Pull modes
    • I would like more control on the brush, the fall off etc.
    • The vert display mode is helpful but appears really bright/large, a different style would be helpful
    • I'm accidentally moving the teeth, and it's really hard to target them to erase the changes to them. I suspect the skinned mesh rendedrer's physics collision is used for the brush targeting, making it hard to get inside the mouth
    Here's it in action:
     
    Last edited: Apr 27, 2021
  6. pab

    pab

    Joined:
    Jul 2, 2013
    Posts:
    6
  7. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Thanks for pointing these out, I'll take a look.

    Did you know if you hold down the "B" button you can change the radius by moving the mouse left and right. I find it much more intuitive than setting it in the inspector.

    Is he at 0 in the world x position?

    ah - let me take a look.

    yes this would be a good future feature.

    ill have a think,
    Did you know you can choose a submesh - this way you can exclude the teeth when making the blendshape :)
     
  8. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Ok this might be a Unity version issue - thanks for giving me the Unity version so I can look into it
     
  9. pab

    pab

    Joined:
    Jul 2, 2013
    Posts:
    6
    Another thing to check. Use a Mecanim Humanoid character with an Animator component playing an Idle animation. I think the lipsync clip will not play whilst the Animator is enabled. Disable the component and the lipsync track plays albeit with the fault already reported.
     
  10. Unique_User_Name

    Unique_User_Name

    Joined:
    Apr 29, 2021
    Posts:
    1
    @jamieniman below errors that I have on Unity version 2021.1.1f1 and 2019.4.20f1



     
  11. unity_ErOys4agLfdvKA

    unity_ErOys4agLfdvKA

    Joined:
    May 22, 2020
    Posts:
    1
    @BarnimPL @jamieniman all reported errors related to incorrect format are due to a problem with parsing float values from XML files. You are using dot (".") to separate values (total value from a fraction) but in some regions a comma (",") is used instead. You just need a better method for parsing float values.
     
    BarnimPL likes this.
  12. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Thank you! That'll help me figure a solution :)
     
    BarnimPL likes this.
  13. Clawdelle

    Clawdelle

    Joined:
    Feb 26, 2017
    Posts:
    3
    Hi, no markers are showing under wave.
    Unity 2020.3.6f1


    <?xml version="1.0" encoding="utf-8"?>
    <rhubarbResult>
    <metadata>
    <soundFile>C:\Users\Delle\Spiel-Neue-Projekte\puppyFace2\Assets/PuppetFace/Demo/Audio/delle1.wav</soundFile>
    <duration>5.30</duration>
    </metadata>
    <mouthCues>
    <mouthCue start="0.00" end="0.74">X</mouthCue>
    <mouthCue start="0.74" end="0.82">A</mouthCue>
    <mouthCue start="0.82" end="0.90">C</mouthCue>
    <mouthCue start="0.90" end="0.97">E</mouthCue>
    <mouthCue start="0.97" end="1.11">F</mouthCue>
    <mouthCue start="1.11" end="1.27">A</mouthCue>
    <mouthCue start="1.27" end="1.55">B</mouthCue>
    <mouthCue start="1.55" end="1.69">D</mouthCue>
    <mouthCue start="1.69" end="2.38">B</mouthCue>
    <mouthCue start="2.38" end="2.65">F</mouthCue>
    <mouthCue start="2.65" end="2.69">C</mouthCue>
    <mouthCue start="2.69" end="2.77">A</mouthCue>
    <mouthCue start="2.77" end="2.94">F</mouthCue>
    <mouthCue start="2.94" end="3.15">G</mouthCue>
    <mouthCue start="3.15" end="3.57">F</mouthCue>
    <mouthCue start="3.57" end="4.32">X</mouthCue>
    <mouthCue start="4.32" end="4.37">B</mouthCue>
    <mouthCue start="4.37" end="4.63">F</mouthCue>
    <mouthCue start="4.63" end="4.70">G</mouthCue>
    <mouthCue start="4.70" end="5.30">X</mouthCue>
    </mouthCues>
    </rhubarbResult>

    upload_2021-4-29_20-59-36.png
     
  14. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Could you try this for me - open the xml in an editor and replace all . with ,
    Then tell me if it works?

    Next thing could you reset that file, and then try adding the following script to the project - I want to see if this works:
    Code (CSharp):
    1. using System.Globalization;
    2. using System.Threading;
    3. using UnityEngine;
    4. #if UNITY_EDITOR
    5. using UnityEditor;
    6. #endif
    7. #if UNITY_EDITOR
    8. [InitializeOnLoad]
    9. public static class FixCultureEditor
    10. {
    11.     static FixCultureEditor()
    12.     {
    13.         Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
    14.     }
    15. }
    16. #endif
    17. public static class FixCultureRuntime
    18. {
    19.     [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)]
    20.     static void FixCulture()
    21.     {
    22.         Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
    23.     }
    24. }
     
    Last edited: Apr 29, 2021
  15. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    I believe I have sorted this, if you email me at Puppet3Dunity@gmail.com I can send you the fix (the xml reader needed to be culture invariant)
     
  16. Clawdelle

    Clawdelle

    Joined:
    Feb 26, 2017
    Posts:
    3


    Cool, comma (,) is working, so it's a German notation thing. r/Detlef
     
  17. Clawdelle

    Clawdelle

    Joined:
    Feb 26, 2017
    Posts:
    3

    Hi Jamieniman, I placed the script in rhubarb folder and it seems to work. Old Man shows the markers without intervention.

    thx/Detlef
     
  18. Miltan_

    Miltan_

    Joined:
    Sep 7, 2013
    Posts:
    2
    Hello, first of all, great tool,
    I have wave audio as byte[]
    Can you provide an C# Code example of how your tool handles this kind of approach on runtime.
    Can your tool perform lipsync animation if i download wave file on runtime
     
  19. paulacs

    paulacs

    Joined:
    Mar 4, 2021
    Posts:
    4
    Hello,
    I've got the following problem, when I try to use the animation clip capture with the PerformanceCapture component in a animator controller the application doesn't run out the rutine and, I've got the following warning:

    "Binding warning: Some generic clip(s) animate transforms that are already bound by humanoid avatar. These transforms can only be changed by Humanoid clips."

    I'm using and Humanoid rig so I don't understand why it doesn't work. Maybe the answer it is very obvious but i'm new in this field.
    Thank you so much!!
     
  20. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Currently there are a few hoops to jump through to get humanoid and performance capture to work together:
    1. import the fbx exporter package
    2. right click on your char (with its anims on an animator) and export as an fbx (with model and anims)
    3. Set this fbx to a humanoid (with the same avatar)
    4. Duplicate all the animations and add them to your animator.
    5. Copy all the blend shape keyframes across from the pre-exported anims to the exported ones.
    It's not ideal. I want to make humanoid specific performance capture - its on my list.
     
  21. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    At the moment the audio to lipsync conversion is editor time only.
    If you take a look at the lipsync script at line around 490, the conversion uses an exe, and is currently wrapped in If UNITY_EDITOR defines.
    I haven't looked into this much though, so perhaps its easy enough to make it work. Its on the the list of todos.
     
  22. Davon92

    Davon92

    Joined:
    Oct 24, 2014
    Posts:
    9
    Could this handle doing simple transitions as well between mouth shapes with no blending in between? im looking for something with this level of flexibility but that can also do static quick changes.
     
  23. paulacs

    paulacs

    Joined:
    Mar 4, 2021
    Posts:
    4
    Thank you!!
     
  24. appleboxman

    appleboxman

    Joined:
    Apr 17, 2020
    Posts:
    1
    Hi,
    I am new to Puppet Face. Using v1.4 today and I got some error messages:

    Assembly 'Library/ScriptAssemblies/Unity.PlasticSCM.Editor.dll' will not be loaded due to errors:
    Reference has errors 'unityplastic'.

    Assembly 'Packages/com.unity.collab-proxy/Lib/Editor/PlasticSCM/unityplastic.dll' will not be loaded due to errors:
    unityplastic references strong named System.Windows.Forms Assembly references: 4.0.0.0 Found in project: 2.0.0.0.
    Assembly Version Validation can be disabled in Player Settings "Assembly Version Validation"
     
  25. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Perhaps its an assembly definition issue.
    Could you try the following to see if it fixes it:
    1. Go to the player settings and make sure "Allow 'unsafe' Code is ticked on. (Allows the use of pointers)
    upload_2021-6-9_9-10-5.png
    2. Now search for the file "OpenCvSharp" and delete just this single file.

    Let me know if that fixes it?
     
  26. paulacs

    paulacs

    Joined:
    Mar 4, 2021
    Posts:
    4
    Hi! I've got a problem in the latest version, when I try to use the Blendshape manager.
    In the blenddshape manager when I try to create new blendshapes I don't get the circle that allows me to edit, it follows the normal mouse cursor, and even if I change pull/push/eraser, nothing, it doesn't change and it doesn't modify the mesh.
     
  27. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Make sure you have the gizmos button on (top right of the scene window) :)
     
  28. Xyth

    Xyth

    Joined:
    Aug 10, 2017
    Posts:
    3
    I am interested in this application to create facial blendshapes on 3D Unity based characters that are rigged but lack facial rigging. Is facial rigging required to create facial blendshapes? Also, I noticed this tool adds several new components in the editor. Do any of those components need to stay on the character during export to an asset file (.Unity3d for example). I would use this for adding characters to an existing game, and that game just uses only built in Unity components. Thanks!
     
  29. jamieniman

    jamieniman

    Joined:
    Jan 7, 2013
    Posts:
    890
    Its not required, you can make the blendshapes on any mesh or skinned mesh.
    The lip sync component is required to drive the lip sync animation.
     
  30. Xyth

    Xyth

    Joined:
    Aug 10, 2017
    Posts:
    3
    Perfect. Thanks for the info!
     
unityunity