Search Unity

[RELEASED] LipSync Pro and Eye Controller - Lipsyncing and Facial Animation Tools

Discussion in 'Assets and Asset Store' started by Rtyper, Mar 11, 2015.

  1. Aleksandr409

    Aleksandr409

    Joined:
    Jan 4, 2019
    Posts:
    6
    I'll send you a private message.
     
  2. Alvarezmd90

    Alvarezmd90

    Joined:
    Jul 21, 2016
    Posts:
    134
    Alright. I have a very annoying issue right now. it might have something to do with the model's face bone scale. But that shouldn't have to matter in my opinion.. EDIT - I tried resetting the bones scale in 3ds max and wasted one and a half hour for it to fail working again.
    Could you please upload the older version 1.5.1?

    EDIT 2 - Lmao, found the older version in my other project and it works flawless there. 1.5.2 has problems.
     
    Last edited: Feb 7, 2020
  3. Alvarezmd90

    Alvarezmd90

    Joined:
    Jul 21, 2016
    Posts:
    134
    Now that set emotion is working again I get this error on resetting after emotion:

    Code (CSharp):
    1. KeyNotFoundException: The given key was not present in the dictionary.
    2. System.Collections.Generic.Dictionary`2[TKey,TValue].get_Item (TKey key) (at <1f0c1ef1ad524c38bbc5536809c46b48>:0)
    3. RogoDigital.Lipsync.LipSync.SetEmotion (System.String emotion, System.Single blendTime) (at Assets/Rogo Digital/LipSync Pro/Components/LipSync.cs:491)
    4. RogoDigital.Lipsync.LipSync.ResetEmotion (System.Single blendTime) (at Assets/Rogo Digital/LipSync Pro/Components/LipSync.cs:505)
    5. ChangeTheEmotions.LateUpdate () (at Assets/Scripts/ChangeTheEmotions.cs:18)
    6.  
    Ah so borrowing that 'Set emotion' code from 1.5.2 works :D
     
    Aleksandr409 likes this.
  4. Aleksandr409

    Aleksandr409

    Joined:
    Jan 4, 2019
    Posts:
    6
    Hello again, I went through all the points of my character "blend shapes face" is missing, but there are bones they behave very strange when moving nothing to change. I attached a video of what I was saying. I hope I have any options to fix this. Thanks.
     
  5. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    2,463
    Edit: This was super simple. I'll leave the post here in case it helps anyone else. Anyway, the issue was that it seems I accidentally included "Blink_Left" in the "Rest" phoneme for Lip Sync. It seems the Rest phoneme takes precedence, and you probably shouldn't ever put your Blink blend shapes there.


    Original question:
    Has anyone experienced Eye Control only causing a single eye to blink? I've got blend shapes for both eyes, and if I adjust those blend shapes with the sliders, they each work fine. However, Eye Control is only adjusting the Blink_Right blend shape at runtime. You can see on the second screenshot a capture during runtime. Blink_right quickly goes from 0 to 1, and back to 0, But Blink_Left doesn't change numerically or in the model at this time. Anyone know what might be causing that?


    upload_2020-2-12_16-48-50.png


    upload_2020-2-12_16-48-17.png
     
    Last edited: Feb 13, 2020
  6. BriBill

    BriBill

    Joined:
    Oct 8, 2019
    Posts:
    1
    I'm using Lipsync Pro in my educational VR project and so far the results have been spectacular! In my project I have a coach that is providing dialogue, emotions, gestures, and eye movements which breath a lot of life into the character. The only piece I'm missing is for the head to also move and track the learner's movements. Is there any easy way to have the coach's head move and track the learner's movements along with they eyes with Lipsync Pro and Eye Controller or should I handle that separately? If so, can you recommend a solution? Thank you!
     
  7. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    2,463
    I'm not sure whether there's anything baked into LipSyncPro/EyeController for moving the head. However, it's pretty simple to do this using built-in Unity Animator methods.

    Take a look at Animator.SetLookAtPosition, and Animator.SetLookAtWeight. You basically tell the animator where to look, and how strongly it should look there, and the head (or body too if you want) will look at that position, overriding any other animations the model might be performing. You'll need to make sure your Animator layer has IK Pass ticked, and that you call the methods during the OnAnimatorIK.

    This thread shows a simple example of someone doing that: https://forum.unity.com/threads/mecanim-animation-setlookatposition.267200/
     
  8. DeadSeraph

    DeadSeraph

    Joined:
    Sep 8, 2017
    Posts:
    95
    Hi guys. Couple of questions:

    -Is there a way to set up multiple lip sync clips to play on command? Currently I have 15 or so lip syncs done but I'd like to be able to call them via code to play at this or that time. Is this possible?

    -We would like to be able to set a delay in the time when a clip plays. I've messed around with the settings a ton but the best I can tell, the delay time in the settings only affects when the animation starts, not when the animation / audio play in tandem. Would it be possible to simply add a delay value for the audio so that one delays the other? This seems like a pretty basic function?

    -Is the source code available anywhere for people who have purchased the pro version?

    Thanks for your time.
     
  9. Jakub_Machowski

    Jakub_Machowski

    Joined:
    Mar 19, 2013
    Posts:
    387
    Hello We use lipsynch and emotions with bones blend, The problem is that when Using bones blending system overall blend of emotion and look wear What I mean for example if lips corners are set is happy emotions more upper position and they are blended with phenomenes that also use that bones, IT dont take average position, but No add emotion on that bones at all, in reuslt emotions on mouth are visible only when character dont talk but even then for example smile has streght "picks" where it shows for a moment dissapear again shows. Is there chance to do it better to emotions get average position, if phenomene and emotion are using the same bone?
     
  10. mateuszjaworski

    mateuszjaworski

    Joined:
    Aug 8, 2019
    Posts:
    4
    Hey guys,

    do weights for phonemes work for you? I am trying to change them and even if I set weigth to 10% they are still played on 100%. For our timelines I am using Slate Sequencer and extension to make LipsyncPro compatible with it. Is there something I have to do to make it work properly? At this moment we don't have blendshapes only bone transforms.
     
  11. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    334
    Hi this asset supported webGL?
     
  12. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    334
    Hi were are planning to buy this asset, I was just wondering if I can load an audio clip from runtime? cause were using google cloud kit, from text to speech then we want to load that audio from this asset? is that possible?
     
  13. shivamnyr

    shivamnyr

    Joined:
    Jul 4, 2017
    Posts:
    1
    Is the limited run-time lipsync available now, where I can just push an audio and the transcript and have it processed at run-time?
     
    Benutzer1 likes this.
  14. minad2017

    minad2017

    Joined:
    Dec 1, 2016
    Posts:
    22
    An error has occurred in the [Slate Cinematic Sequencer] plug-in.

    Assets\ParadoxNotion\SLATE Resources\Extensions\LipSync\LipSyncSpeech.cs(14,57): error CS0535: 'LipSyncSpeech' does not implement interface member 'ISubClipContainable.subClipSpeed'

    Assets\ParadoxNotion\SLATE Resources\Extensions\LipSync\LipSyncSpeech.cs(14,57): error CS0535: 'LipSyncSpeech' does not implement interface member 'ISubClipContainable.subClipSpeed'

    Can you fix it?
    unity ver 2019.3.2f1
    lipsyncpro ver 1.52
    slate ver 1.9.8
     
  15. mdotstrange

    mdotstrange

    Joined:
    Sep 23, 2013
    Posts:
    222
    I also just tried to use LipSyncPro with Slate and got the same error-

    Open the script "LipSyncSpeech" and paste this on line 84 or anywhere in the script and everything will work after that.

    Code (CSharp):
    1.  public float subClipSpeed
    2.         {
    3.             get
    4.             {
    5.                 return 0;
    6.             }
    7.         }
    8.  
    9.         public float subClipLength
    10.         {
    11.             get
    12.             {
    13.                 return 0;
    14.             }
    15.         }
     
  16. minad2017

    minad2017

    Joined:
    Dec 1, 2016
    Posts:
    22
    Thank you very much.
    It works now.
     
  17. minad2017

    minad2017

    Joined:
    Dec 1, 2016
    Posts:
    22
    I am developing with Playmaker.
    I am having trouble calling SetEmotion from Playmaker.
    With "Slate plugin", it was possible to operate Emotion.
    As a test, I tried running a simple C # script, but Emotion still does not work.
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using RogoDigital.Lipsync;
    5.  
    6. public class SetEmotionTest : MonoBehaviour
    7. {
    8.     public LipSync lipSyncTarget;
    9.     public string emotionname;
    10.     public float blendtime;
    11.  
    12.     void Start()
    13.     {
    14.         lipSyncTarget.SetEmotion(emotionname, blendtime);
    15.     }
    16. }
    17.  
    Below is the error message.

    NullReferenceException: Object reference not set to an instance of an object
    RogoDigital.Lipsync.LipSync.SetEmotion (System.String emotion, System.Single blendTime) (at Assets/Rogo Digital/LipSync Pro/Components/LipSync.cs:611)
    SetEmotionTest.Start () (at Assets/---Script/-Sound/SetEmotionTest.cs:14)

    unity ver 2019.3.2f1
    lipsyncpro ver 1.52
     
  18. FirstTimeCreator

    FirstTimeCreator

    Joined:
    Sep 28, 2016
    Posts:
    729
    Hello,

    I have sent a separate support request in via email but figured I would ask here as well about this issue.

    1.When I setup the facial poses using the Bones system and try to preview a audio clip, all facial poses get reset to their default positions and all facial pose data appears to be lost.
    2. When I save a preset, nearly all scale, transform and rotation data is set to 0 when the preset is loaded.

    There are no errors in the console.

    The system actually works and moves the mouth/face joints until i click off of the preview window and everything is reset to the default position.

    This literally wasted an entire day of work for me because I finally got done setting up a complex face rig that is made using joints and not blendshapes went to test it, it was working great, clicked on hiarachy to go save the preset and literally ALL facial poses had been set to to the default rig position.

    I just about popped a blood vessel.

    This should not be happening.
     
  19. DeadSeraph

    DeadSeraph

    Joined:
    Sep 8, 2017
    Posts:
    95
    I wouldn't expect much support from the dev at this point. The asset doesn't seem to be supported at all anymore. I'll be seeking a refund and moving to Salsa, unfortunately.
     
  20. vorokoi

    vorokoi

    Joined:
    Oct 18, 2019
    Posts:
    28
    @DeadSeraph The questions you are asking are all answered in the documentation and in this forum thread. Yes you can call clips via code, trigger them via events, set a delay, anything via script. There is a completely documented API reference. If it was not possible, why would there would be an API? You are asking for the source code when it comes with the source code? What do you think all the scripts it comes with are for?

    Salsa is good for what it does, but doesn't offer the same flexibility and precision LipSync Pro does. It really isn't even close.

    Man, I would hate to be an asset publisher and have to deal with this kind of behavior from lazy/entitled users.
     
  21. msminotaur

    msminotaur

    Joined:
    Mar 27, 2014
    Posts:
    1
    Hi - I just started using Lip Sync Pro recently, and while I've been able to run autosync on some of my audio files, I've run into an issue where certain audio files fail with an error telling me to check our encoding or enable conversion. The audio files that DO work, are no different than the audio files that DON'T work (other than length and actual content....but they're all encoded the same), and we have conversion enabled. I've tried to look through all the posts on this thread, but I haven't seen much about autosync and this issue. Has anyone else encountered this inconsistency? Is there a solution I'm completely missing?

    Screen Shot 2020-03-05 at 3.31.11 PM.png
     
  22. DeadSeraph

    DeadSeraph

    Joined:
    Sep 8, 2017
    Posts:
    95
    1) If any of the things I have asked about are listed in the documentation, they certainly aren't clear or apparent, as I've been through the documentation multiple times, and I don't see a section that answers the questions I have asked. Maybe you could point me to the relevant sections?

    2) I'm not a programmer, I'm just doing some research for my team. We've purchased the pro version and it would be nice to not have to buy a different asset and relearn that one.

    3) If you purchase a product on the asset store, it is assumed that it is still being supported by the developers unless it has been deprecated. I'm not the only person who has had issues getting questions answered by the developer for this plugin.

    4) It's not a matter of being lazy. Support your asset or deprecate it. It's that simple. The developer has not been active in this thread for months and does not answer emails sent to their website for the product either. No need to be rude.
     
    Gregorik likes this.
  23. Gubendran_Vins

    Gubendran_Vins

    Joined:
    Mar 20, 2018
    Posts:
    3
    Hi,
    I am working with Lip Sync Pro plugin for Oculus GO VR application. Everything is working fine, But It reducing my FPS and hang up the application for a moment when my humanoid model start to speak.

    Is there any solution for this?

    Thanks.
     
  24. FirstTimeCreator

    FirstTimeCreator

    Joined:
    Sep 28, 2016
    Posts:
    729


    Fri, Mar 6, 2:43 PM (4 days ago)

    Support Request was sent 4 days ago. If I don't hear anything by weeks end, I guess we will have no choice but to change to Salsa because this is holding up development.

    This is a good product, If I can get some support it would be worth using. My last ditch effort to try to fix the problems with it will be to try to upgrade to the latest unity version. Surely others are experience these issues where the Data is not saving to the preset properly for Generic Rigs as well as the facial poses being basically "erased" when you test a sample clip in preview... again using Generic Rig AND Bones system (NOT blendshapes).
     
  25. Gregorik

    Gregorik

    Joined:
    Dec 11, 2014
    Posts:
    396
    @vorokoi You seem to be on a crusade to diss users on these forums, even when they come with legit queries, and even when the asset developers in question are clearly slacking. Please stop it.

    @FirstTimeCreator Same here. Lipsync Pro 1.52 has started to behave erratically on 2019.3 which is not surprising since 2019.3 final was released way after the last Lipsync version. It's not supported on recent Unity versions, meaning it's compromised. Hopefully it's not 100% abandoned yet, but I'm making the jump to Salsa too.
     
  26. vorokoi

    vorokoi

    Joined:
    Oct 18, 2019
    Posts:
    28
    @Gregorik I don't think you know what crusade means. I answered someone's questions, which I guess you could have done too, but you chose not to? If you aren't going to address a topic, don't tell someone who is not to.

    I am using LipSync Pro 1.5.2 on 2019.3.4f1 with absolutely zero issues.
     
  27. four-d-oz

    four-d-oz

    Joined:
    Aug 18, 2014
    Posts:
    9
    Hello everyone. We are having an issue with Lipsync Pro editor GUI not displaying properly. Specifically LipSync Data GUI. We are using Unity 2018.4.16f1 and have updated Lipsync pro to use the latest version 1.52 using the update guidelines. We were previously on an older version of Lipsync Pro and we where having component GUI issues where the menus were appearing stacked, but that now is fixed. The issue we have now is related to Lipsync data windows not displaying the GUI properly. Here is a screenshot of what I am talking about. If anyone has any idea how to resolve this, we would greatly appreciate it.

    upload_2020-3-16_14-25-46.png
     
  28. four-d-oz

    four-d-oz

    Joined:
    Aug 18, 2014
    Posts:
    9
    OK just wanted to answer my own question just n case anyone else runs into this issue. Our problem was that we moved the Rogodigital Editor Default Resources to a subdirectory in the project and did not notice. Once we moved all those resources to the proper Editor Default Resources directory on the root of assets, all the UI is working properly.
     
    CoyoteFringe and Gregorik like this.
  29. NawarRajab

    NawarRajab

    Joined:
    Aug 23, 2017
    Posts:
    20
    Will this ever be solved? We have this bug for more than 6 months now

    Code (CSharp):
    1. ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
    2. Parameter name: index
    3. System.ThrowHelper.ThrowArgumentOutOfRangeException (System.ExceptionArgument argument, System.ExceptionResource resource) (at <437ba245d8404784b9fbab9b439ac908>:0)
    4. System.ThrowHelper.ThrowArgumentOutOfRangeException () (at <437ba245d8404784b9fbab9b439ac908>:0)
    5. System.Collections.Generic.List`1[T].get_Item (System.Int32 index) (at <437ba245d8404784b9fbab9b439ac908>:0)
    6. RogoDigital.Lipsync.BlendSystem.SetInternalValue (System.Int32 blendable, System.Single value) (at Assets/DMAI/3rd Party/Rogo Digital/LipSync Pro/Classes/BlendSystem.cs:230)
    7. RogoDigital.Lipsync.BlendshapeBlendSystem.SetBlendableValue (System.Int32 blendable, System.Single value) (at Assets/DMAI/3rd Party/Rogo Digital/LipSync Pro/BlendSystems/BlendshapeBlendSystem.cs:67)
    8. LipSyncEditor.OnInspectorGUI () (at Assets/DMAI/3rd Party/Rogo Digital/LipSync Pro/Editor/LipSyncEditor.cs:840)
    9. UnityEditor.UIElements.InspectorElement+<>c__DisplayClass55_0.<CreateIMGUIInspectorFromEditor>b__0 () (at <6f28216fea9f453abf2e05b770ed3ee4>:0)
    10. UnityEditor.PopupCallbackInfo:SetEnumValueDelegate(Object, String[], Int32)
     
  30. Joshua_Bogart

    Joshua_Bogart

    Joined:
    Aug 5, 2015
    Posts:
    3
    Is it safe to say this asset isn't a viable option for WebGL builds? More specifically, any details on WebGL 1.0 vs. WebGL 2.0, asm vs. web assembly, etc? 2018.4.20f1 is the current target, but "AudioPlayback timing mode is incompatible with WebGL target." is all I see when adjusting Timing Modes. There doesn't seem to be much documentation for this and forums seem thin with details.
     
    Last edited: Apr 9, 2020 at 7:06 AM
unityunity