Search Unity

[RELEASED] LipSync Pro and Eye Controller - Lipsyncing and Facial Animation Tools

Discussion in 'Assets and Asset Store' started by Rtyper, Mar 11, 2015.

  1. Chris_Payne_QS

    Chris_Payne_QS

    Joined:
    Jul 16, 2016
    Posts:
    84
    Hi @Rtyper - great tool, but I'm having a nasty teething problem.

    In the phoneme preview, I'm seeing some hideous deformations because some of the bones are rotating the long way round between keys, as if it was blending eulers rather than quats. Some of my bones have got suspicious rotations in the phoneme list like 358 degrees, and it seems they're always being clamped to positive so I can't manually set them back to -2.

    All my phoneme poses have been imported from anims created by my Art Director BTW.

    Poking around in the code, it does seem to be using quats, although it looks like you're making a curve for each of the four quat components which I think might suffer from the same issue if you're blending them independently. Don't you need to slerp between entire quats to preserve the shortest rotation?

    [EDIT] Ah, I've found where AddKey converts positive eulers to negative eulers, so that's making more sense...unless the curves are operating on positive eulers.

    [EDIT] Oh look! FixQuaternionContinuity()...
     
    Last edited: Jun 27, 2017
  2. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    Sorry about this Chris! The code you've found there is supposed to get around this problem that animation curves have by default, and it normally works.
    I've just been experimenting with this to try and recreate it and I'm pretty sure it's caused by the interpolating of values done for the marker intensity and randomness settings. If you have all the markers in your animation set to 100% intensity and have randomness turned off on them all, do you still get the incorrect interpolation?

    EDIT: Yep, I've got that sorted now. It'll be in the next patch, but if you want to give me an email address (or send an email to contact@rogodigital.com) I can send you the fixed files now.
     
    Last edited: Jun 28, 2017
    Chris_Payne_QS likes this.
  3. Chris_Payne_QS

    Chris_Payne_QS

    Joined:
    Jul 16, 2016
    Posts:
    84
    You are an absolute star @Rtyper! I'll mail you...

    [EDIT] I should note that our character has ~80 bones in her face, so it's not surprising we hit a rare edge case with this new feature. Love the tool though!
     
    Last edited: Jun 28, 2017
  4. harry4951

    harry4951

    Joined:
    Oct 5, 2016
    Posts:
    8
    Hi, I am looking for the realtime lipsync, say, the micphone to lipsync or the random texts to lipsync animations. We have an AI virtual robot who will talk with human. Does the current version work on these functions now? Thx!
     
  5. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey All,

    We are currently using an Adobe Fuse character and cant get blinking to work quite right. The character currently blinks fine however his eyelashes remain stationary. While looking into this I discovered that the Left and Right Blink Blends are not the same between the 2 skinned meshes (different indices). Is there an easy way to fix this with the model or do I need to change the eye controller script to to accept the different blends for each skinned mesh?

    Cheers,
    Colton
     
  6. BigB

    BigB

    Joined:
    Oct 16, 2008
    Posts:
    672
    Hey everyone,

    I’m having a hard time figuring something out, it’s not in the documentation or in the forums.
    I have a character, with which I have to interact.

    There’s several dialogs to be performed with that character.
    I already have all the animation sync done, its 5 different talks, but because I cannot have them all in the same mesh, I have several copies of the same mesh, each with its own dialog.
    Is there a way to get all of this in the same character, and then just change at runtime what “talk/animation” I want ?

    How can I launch the animation from script ?
    I know I can do this : Play(defaultClip, defaultDelay);
    But where is the defaultClip defined ?
    And where or how do I add new clips ?

    Thanks !
     
  7. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    I'm afraid not. LipSync Pro 2's going to add runtime autosync, but there's no plans at the moment to make it totally real-time.

    This is something I should've added a while ago, really, but I couldn't come up with a good design for how it should work. Fuse models used to have the same blends on each mesh so it wasn't a problem until recently! I should be able to put together a blend system that will let you do this pretty quickly - like with the timeline support, if you give me an email address or send me an email I can get a patch over to you.

    The 'clips' are LipSyncData .asset files that are saved out of the clip editor. You can reference them the same as any other object to drop in in the editor, or you could use the Resources system. Something like this:

    Code (CSharp):
    1. using UnityEngine;
    2. using RogoDigital.Lipsync;
    3.  
    4. public class MyScript : MonoBehaviour {
    5.     public LipSync lipsyncComponent;
    6.     public LipSyncData clip;
    7.  
    8.     private void Start () {
    9.         lipsyncComponent.Play(clip);
    10.     }
    11. }
     
  8. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Thanks! I sent you a PM last week with my email address in it.
    Cheers,
    Colton
     
  9. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Has anyone experienced problems with the OnFinishedPlaying event? It's only working in the editor for me, not in the standalone windows build. Any thoughts?

    Cheers,
    Colton

    EDIT: Only works about 50% in the editor
     
    Last edited: Jul 7, 2017
  10. Chris_Payne_QS

    Chris_Payne_QS

    Joined:
    Jul 16, 2016
    Posts:
    84
    I've created TTS placeholders and auto-synced them for a whole chapter of dialogue, and now that I'm hearing the same samples over and over again, I've noticed significant pitch/speed variations in the audio playback, even halfway through a line.
    Might be framerate-related...are you feeding the AudioSource on the foreground thread?
     
    Last edited: Jul 9, 2017
  11. reza1313

    reza1313

    Joined:
    Jul 14, 2017
    Posts:
    7
    NullReferenceException: Object reference not set to an instance of an object
    RogoDigital.AudioUtility.GetWaveForm (UnityEngine.AudioClip clip, Int32 channel, Single width, Single height) (at Assets/Rogo Digital/LipSync/Editor/AudioUtility.cs:94)
    LipSyncClipSetup.Update () (at Assets/Rogo Digital/LipSync/Editor/LipSyncClipSetup.cs:2748)
    System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
    Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
    System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
    System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
    UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:262)
    UnityEditor.HostView.Invoke (System.String methodName) (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:255)
    UnityEditor.HostView.SendUpdate () (at C:/buildslave/unity/build/Editor/Mono/HostView.cs:330)
    UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at C:/buildslave/unity/build/artifacts/generated/common/editor/EditorApplicationBindings.gen.cs:249)
    i use unity 5.6.0 and lipsync pro 1.1.when i open lipsync editor i see this error.please help me thanks
     
    Qlevine20 likes this.
  12. Hamesh81

    Hamesh81

    Joined:
    Mar 9, 2012
    Posts:
    405
    Hi @Rtyper , I realize it has only been released fairly recently, but are there any plans to integrate this great asset with Timeline in Unity 2017.1?
     
  13. Devistute

    Devistute

    Joined:
    Aug 23, 2014
    Posts:
    32
    Hi, i noticed that the Release Notes for version 1.4 had talk about Auto Sync 3?
    What does this exactly do, is it way better than the previous one? Just curious.
     
  14. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    Do you think you could send me any more info or a smaller/stripped down project that demonstrates the problem? I haven't seen this myself, and OnFinishedPlaying is just a standard UnityEvent, so I'm not sure what could be going wrong there..

    Shouldn't be any issues to do with LipSync there. LipSync only does 2/3 things with the Audio Source: it sets the clip & calls .PlayDelayed when it starts playing, and (if you have scaleAudioSpeed checked) it sets the pitch to Time.timeScale in the Update call.

    I'd say turn off scaleAudioSpeed and see if it's still happening. Maybe check that nothing else is setting Time.timeScale strangely too. Otherwise, have a look at the Profiler and see if anything is causing serious lag spikes. If it is LipSync, let me know and I'll try to sort it out!

    Your LipSync version is well out of date, and it looks like you've got the package designed for a different version of Unity as well. Just update from the Asset Store.

    Yep! I'm working on a new extension package at the moment that will add a new track type etc to Timeline. If you're interested in technical details, I'm also planning on having version 2.0 use the Playables system as a base for LipSync clips, so the whole system will be much more integrated with Timeline in the future.

    Yeah, I've had to leave it out of this one, but it hopefully isn't too far off now. It's got two main advantages over AutoSync 2 really: quality and compatibility. It uses a different approach to phoneme detection which is more consistent and (usually) provides more accurate results, and it's also compatible with macOS Sierra (and High Sierra) which the current one isn't.
     
    Corvwyn and Hamesh81 like this.
  15. Jambamain

    Jambamain

    Joined:
    Jul 20, 2017
    Posts:
    1
    Hello,

    I contacting you because I 've got few issues with your plugin LypSync Pro I have bought. I guess it's because I'm using Unity 2017.1.0 but I just tested your new update and it's persiting.
    - When I want a delay before playing lipsync it doesn't work in "audio playback" timing mode. I have to do it in "custom timer".
    - "Emotion bar" is very hard to trim, displace and even sometime, impossible to delete.
    - I can't have emotion like "happy" during lypsinc when i'm using autosync mode. It's working when i do it manualy.
    Thanks you for your help,

    Benjamin
     
  16. viv710

    viv710

    Joined:
    Jul 29, 2016
    Posts:
    8
    Hi,
    i am using Lipsync Lite version, In build version lipsync is not working this is the error am getting
    Code (CSharp):
    1. ArgumentOutOfRangeException: Argument is out of range.
    2. Parameter name: index
    3.   at System.Collections.Generic.List`1[RogoDigital.Lipsync.Blendable].get_Item (Int32 index) [0x00024] in /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/List.cs:635
    4.   at RogoDigital.Lipsync.BlendSystem.SetInternalValue (Int32 blendable, Single value) [0x00000] in <filename unknown>:0
    5.   at RogoDigital.Lipsync.BlendshapeBlendSystem.SetBlendableValue (Int32 blendable, Single value) [0x00032] in D:\Unity Working\20_07_2017\01_retail\Assets\Rogo Digital\LipSync\BlendSystems\BlendshapeBlendSystem.cs:56
    6.   at RogoDigital.Lipsync.LipSync.LateUpdate () [0x00000] in <filename unknown>:0
    7. (Filename: /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/List.cs Line: 635)
    its working fine in Unity Editor...
    Thanks & Regards
    Vivek. D
     
  17. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    649
    Hi,

    I'm sorry if this has been asked before. Where can I find the workflow for getting Daz characters working with this? Is there anything which auto sets the phenome blendshapes, or do I have to do that by hand? If there is, is there any specific export process that is needed from Daz?
     
    andynatusch likes this.
  18. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    649
    Will this be frame-rate independent? I'm putting together a short film using timeline, and I'm trying to figure out how to handle facial emotions / speaking. I'm probably going to export the entire thing out as still frames, and I don't think I'm going to have it running at 24fps (its a bit prettier than my computer can handle at 4x ssaa). It would be great if this could still play the facial animations on their proper frames (much like a regular animation would) and then I could layer in the sound afterwards. If not, do you have any recommendations?
     
  19. Casanuda

    Casanuda

    Joined:
    Dec 29, 2014
    Posts:
    53
    Hi,

    Quick question if you don't mind.

    Does auto sync work with MacOS Sierra?

    Auto lipsync never completes and I have to force quit Unity. I am using MacOS Sierra, Unity 2017.1. I see a few comments back you mentioning Auto Sync 3 being compatible.

    Thanks
    Cas
     
  20. twinmatrix

    twinmatrix

    Joined:
    Sep 6, 2012
    Posts:
    38
    Hi There, I was considering buying Lipsync Pro after trying the Lite version. But than again, after playing with the standard animation and looking at the manual I could not find any way to start the lipsync animations from code, for a certain character at a certain moment. Also I see no way how to play it from the mecanim (animator). In the demo I noticed a (standard) animation file, but how do I produce such a file out of the Lipsync clips? Too many important questions, could nowhere find the answers (tried Lipsync. Pause() en Lypsync.Play, etcetera, after using RogoDigital.Lipsync; in the script), etcetera. So before I buy the product I hope to find/receive the answers and instructions on the matter.
    Thanks, Johan
     
  21. Teews

    Teews

    Joined:
    Jul 31, 2017
    Posts:
    1
    Hi there,

    I have a really basic question about LipSync Pro and Eyecontroller. I just can't make them work together. Can someone please tell me the right way and order of installation? I install LipSync and when I want to install Eyecontroller I get duplicated files and don't know at this point what to do...when I agree to override it won't work. Tried couple different things - with same results. I know it's a noob question but I'm new to Unity and couldn't find help anywhere else. I even wrote an email to Rogo tech support but had no answer for couple of weeks now.

    Thanks in advance, Greg.
     
  22. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Hi!
    Using Lipsync Pro 1.4 on Unity 2017.1.0f3, and every time I try to Recognise Phonemes, the importer windows freezes, and I have to kill Unity.

    I have tried to use solely 16bit, 16khz, mono, PCM wav-files, and I have also linked and verified the Sox plugin (green light), but no. I even tried to recognise the attached Gettysburg POCM files, but they also make the app crash.

    Anyone have a suggestion? @Rtyper ?

    EDIT: After some more research: 1.4 does not work on macOs Sierra.... So, when will we have support for that?
     
    Last edited: Aug 20, 2017
  23. simonejennings

    simonejennings

    Joined:
    Jan 11, 2017
    Posts:
    13
    Hi!

    We have been looking at using the morph3d characters with the lipsync pro plugin and as you can see in this image there are significant performance issues and when we check the profile we can see that the issues are with the lipsync.garbagecollect and lipsync.lateupdate. We've used your system with other characters and love it and have never had any issue before and after contacting morph3d they have told us the error must be on the lipsync end.

    Hope we can find a solution!

    Thanks,
    Simone
     

    Attached Files:

  24. Chris_Payne_QS

    Chris_Payne_QS

    Joined:
    Jul 16, 2016
    Posts:
    84
    Somehow I've got my install into a state where the LipSync editor window won't open. Focus shifts away from Unity but no window is visible onscreen. Tried resetting layout. Anyone seen this before?

    [EDIT] Tried creating a new project and installing LipSync in there. Still couldn't open the editor window :(
     
    Last edited: Aug 20, 2017
  25. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Maybe @Rtyper has something to contribute with?

    To the previous post also maybe, @Rtyper ?:

     
  26. Chris_Payne_QS

    Chris_Payne_QS

    Joined:
    Jul 16, 2016
    Posts:
    84
    My problem got fixed eventually, after a number of Unity restarts and a Library deletion and reimport. I was working on something else by then and the LipSync window surprised me by opening unexpectedly on startup, in the middle of the screen. Behaving normally again now...
     
  27. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    Hello everyone - really sorry for all the missed messages! I've had a combination of being busy with work/life things and technical problems that meant I haven't been working on LipSync for a few weeks now. If you sent an email to contact@rogodigital.com before the 11th of August and I haven't replied, please send it again! I've changed web hosts and a number of emails got lost in the process. I can promise support will be back to normal from now on!

    I'll go through the messages here over a couple of posts, so please bear with me.
    Thanks for being patient!
     
  28. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    Did you have any errors in the console during these problems? I can't say I've come across any of them myself, so I don't have a solution off the top of my head. The delay with Audio Playback mode used to be a problem, but was fixed several versions ago - does this still happen with 1.4?

    As for the AutoSync issue, that only affects the phoneme markers in the editor. There's no difference at runtime between a LipSyncData clip that was made using AutoSync and one that was made manually. Again, if you're still having this problem could you give me any more information about it?

    Are you using Morph3D's MCS character? If so, I'd recommend downloading the Morph3D blend system and using that instead, as their characters can have a different number of blendshapes attached at runtime, and the Morph3D blend system is designed to deal with that.

    If it's not an MCS character, I'd need a little more information: is it a standard 3D model you've made yourself or from some kind of character creator/asset? Whatever the cause is, it seems like there are somehow fewer blendshapes on the model when you build it.
     
  29. ImFromTheFuture

    ImFromTheFuture

    Joined:
    May 21, 2015
    Posts:
    30
    Hi @Rtyper,
    I've just checked out the asset store page and was demoing the lite version of this tool. It doesn't seem to work with Morph3D for me. I've followed the video guides correctly. I tried the same process with a fuse character and it was working but it fails to do so with Morph3D.
    Is there a way to solve this? I'm possibly going to purchase this in a few hours so I need to know if this works with it properly or not.
     
  30. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey @Rtyper,
    I think there is a bug at line 59 in AdvancedBlendshapeBlendSystem.cs. The original line was

    manager.blendShapes[blendable].mappings.skinnedMeshRenderer.SetBlendShapeWeight(blendable, value);

    but I think it should be changed too

    manager.blendShapes[blendable].mappings.skinnedMeshRenderer.SetBlendShapeWeight(manager.blendShapes[blendable].mappings.blendShapeIndex, value);

    Before making this change I was having array out of index exceptions because it wasn't using the mapping, I have made this change in my project and everything works perfectly!
    Cheers,
    Colton
     
  31. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    There's no preset for Daz characters, but it's possible to get morphs in Daz to import into Unity as blend shapes. I've found a tutorial covering that here, but I haven't personally used Daz so I can't really give any more specific help. Once you've got your poses set up for one character, you can create a preset to use for others.

    Not yet, sorry. AutoSync 3 may still be a while away, so I'm looking into getting the current one updated now. There should be more info on that very soon!

    I'm assuming that the animation files you're talking about are LipSyncData assets? Those are created in the clip editor (the process is covered in our tutorials), and are played by calling Lipsync.Play on any LipSync component. You just pass a reference to a LipSyncData object to that method. Playing from mecanim isn't supported out of the box, but it should be possible to write a StateMachineBehaviour script to do that if you needed it.

    LipSync Pro actually includes Eye Controller already, so there's no need to install it! The standalone version of Eye Controller is only designed to be used separately or with LipSync Lite.
     
  32. ImFromTheFuture

    ImFromTheFuture

    Joined:
    May 21, 2015
    Posts:
    30
    Hi,

    I just bought the asset. It's wonderful and very easy to set up.
    I have one problem that I need to solve right now, when I use the asset with the timeline feature, it doesn't work. I've made buttons that play the lipsync data and when I press the button, the Morph3D character slides all over the scene. I've found that when I disable the timeline gameobject and then press the button, it works fine.
    How do I solve this?

    Regards!
     
    Last edited: Sep 12, 2017
  33. HaraldL

    HaraldL

    Joined:
    Sep 12, 2017
    Posts:
    1
    Hi,

    I'm working on a project where we would like to use Lipsync in the timeline, Is there an ETA on the extension for timeline?
     
    fildax likes this.
  34. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    649
    I was able to very easily create a timeline plugin for this that only affects the morphs of the character (does not play the sound) by using the following code. I'm using it to render out .png sequences using Octane to create a video in Premiere. You could probably just add another audio layer to sync up the audio with the lipsync in the timeline the same way I'm doing it in Premiere.

    Code (csharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEngine.Playables;
    6. using UnityEngine.Timeline;
    7.  
    8. using RogoDigital.Lipsync;
    9.  
    10.  
    11. public class LipSyncPlayable : BasicPlayableBehaviour
    12. {
    13.     public ExposedReference<LipSync> Target;
    14.     public ExposedReference<LipSyncData> Data;
    15.  
    16.     private LipSync _target;
    17.     private LipSyncData _data;
    18.  
    19.     public override void OnGraphStart(Playable playable)
    20.     {
    21.         _target = Target.Resolve(playable.GetGraph().GetResolver());
    22.         _data = Data.Resolve(playable.GetGraph().GetResolver());
    23.     }
    24.  
    25.     public override void ProcessFrame(Playable playable, FrameData info, object playerData)
    26.     {
    27.         _target.PreviewAtTime((float)playable.GetTime() / _data.length);
    28.     }
    29.  
    30.     public override void OnBehaviourPlay(Playable playable, FrameData info)
    31.     {
    32.         _target.PreviewAtTime(0);
    33.         _target.TempLoad(_data.phonemeData, _data.emotionData, _data.clip, _data.length);
    34.         _target.ProcessData();
    35.     }
    36. }
    37.  
    Code (csharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEngine.Playables;
    6. using UnityEngine.Timeline;
    7.  
    8. using System.Linq;
    9.  
    10. [TrackColor(0, 200, 0)]
    11. [TrackClipType(typeof(LipSyncPlayable))]
    12. public class LipSyncTrack : TrackAsset
    13. {
    14.     public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
    15.     {
    16.         return ScriptPlayable< LipSyncMixer>.Create(graph, inputCount);
    17.     }
    18. }
    19.  
    20. public class LipSyncMixer: BasicPlayableBehaviour
    21. {
    22. }
    23.  
    Basically, I took how they do the Preview mode, and plugged that into timeline :).
     
    jeromeWork likes this.
  35. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    240
    I just bought it, looks very promising, have one issue though (probably noob thing): when choosing Advanced Blendshapes, my second Skinned Mesh Renderer (teeth) does not seem to use its blend shapes, although I built blend shapes from names using BlendShape Manager. Teeth blendshapes seem not to be receiving any input. Any idea?
     
  36. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    240

    now I see this:
    Array index (16) is out of bounds (size=3)
    UnityEngine.SkinnedMeshRenderer:SetBlendShapeWeight(Int32, Single)
    RogoDigital.Lipsync.AdvancedBlendshapeBlendSystem:SetBlendableValue(Int32, Single) (at D:/projects/Unity_BrekelThomasWLipSync/Assets/Rogo Digital/LipSync Pro/BlendSystems/AdvancedBlendshapeBlendSystem.cs:59)
    LipSyncEditor:OnDisable() (at D:/projects/Unity_BrekelThomasWLipSync/Assets/Rogo Digital/LipSync Pro/Editor/LipSyncEditor.cs:251)

    my second Skinned Mesh Renderer has different amount of blendshapes (for obvious reasons). Any patch for this?
     
  37. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey Artumandas,

    Change line 59 in AdvancedBlendshapeBlendSystem.cs too :

    manager.blendShapes[blendable].mappings.skinnedMeshRenderer.SetBlendShapeWeight(manager.blendShapes[blendable].mappings.blendShapeIndex, value);

    This should fix your issue.
    Cheers,
    Colton
     
  38. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    240
    Hi, that does not work, do I have some obsolete code here? This is the method in question:

    Code (csharp):
    1.  
    2. public override void SetBlendableValue (int blendable, float value) {
    3.             if (!isReady)
    4.                 return;
    5.  
    6.             if(manager.blendShapes[blendable].mappings != null) {
    7.                 for (int i = 0; i < manager.blendShapes[blendable].mappings.Length; i++) {
    8.                     SetInternalValue(blendable, value);
    9.                     manager.blendShapes[blendable].mappings[i].skinnedMeshRenderer.SetBlendShapeWeight(blendable, value);
    10.                 }
    11.             }
    12.         }
    13.  
    14.  
     
  39. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    My method looks like this and it solved the issue for me

    Code (CSharp):
    1.         public override void SetBlendableValue (int blendable, float value) {
    2.             if (!isReady)
    3.                 return;
    4.  
    5.             if(manager.blendShapes[blendable].mappings != null) {
    6.                 for (int i = 0; i < manager.blendShapes[blendable].mappings.Length; i++) {
    7.                     SetInternalValue(blendable, value);
    8.                     manager.blendShapes[blendable].mappings[i].skinnedMeshRenderer.SetBlendShapeWeight(manager.blendShapes[blendable].mappings[i].blendShapeIndex, value);
    9.                 }
    10.             }
    11.         }
     
  40. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    I guess you've found out yourself now, but yeah, Morph3D works fine with Pro now! The Lite version needs updating.
    I'm a bit confused about your second problem though... LipSync shouldn't be able to change the position of the whole character object at all, and if it's fine when you disable Timeline my first guess would be that it's something about your timeline object rather than LipSync.

    Could you check if it keeps happening if you remove LipSync from the project?

    It's nearly done! Aiming for the start of next week to release it, though @DGordon's one looks like it'll work well too if you don't need audio. :)

    @arturmandas & @ColtonK_VitruviusVR Thanks for spotting that Colton, you're right that should fix arturmandas's issue as well. I'll get that in a patch ASAP!
     
  41. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    649
    Do you have properly scrubbing audio in the new one? If you do, I'm very interested in seeing how you accomplished that ... assuming your new timeline code will be available. The built in Audio timeline stuff doesn't scrub, and although I built my own which does, I still haven't got that working flawlessly ... just good enough for our current needs.
     
  42. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Any chance of getting a pre-release version with timeline support this week?
     
  43. Pierrein8

    Pierrein8

    Joined:
    Nov 7, 2016
    Posts:
    6
    Hello,
    i send an email to contact@rogodigital.com but with no response so i try here !

    i'm working with lipsyncpro and autosync. But i have realy strange result (with default setting or high setting).

    I have empty spaces zone ( no marker detected ) in my audio at some place,
    Some place in the audio have too much marker.
    And some marker are detected while no sound...

    Do you have a solution for this problem and increase quality?

    thank you !
     
  44. harry4951

    harry4951

    Joined:
    Oct 5, 2016
    Posts:
    8
    Hi,

    I am using Lipsync Pro 1.4 and the latest downloaded Morph3D character. I already downloaded the blendshapes extensions for Morph3D. When it came to lipsync blendsystem to my character, it told me "M3D Character not set..."

    upload_2017-9-15_11-42-41.png
    I have no idea how to fix it. It would be good if you could help me to figure it out. Thanks!
     
  45. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    Mine doesn't currently have audio scrubbing, but it does have audio playback at runtime. I had been playing around with trying to have it work the same way in the editor and at runtime but audio really ends up being a problem.

    Yeah, I don't see why not - I'll send you a PM later today. Bear in mind there might still be some errors here and there!

    I have come across this before, it seems to happen only on some (usually short) clips. Can you try going to the AutoSync settings page in the clip editor, enabling advanced options and trying the following values:

    lipsync_advanced_options.PNG

    I've had some success with those settings on a clip that previously showed up with a lot of blank spaces.
    EDIT: Sorry, I've just seen your email as well - those settings might improve things a bit, but I wouldn't expect great results on French dialogue. The language model is only designed for English. I'm hoping to be able to create additional language models for the next AutoSync version, but the only one available for the current version is En-US.

    This may sound like a obvious question, but have you pressed the continue button? I've just tried replacing this myself (re-downloading everything into a clean project) and it works fine.
     
    Last edited: Sep 15, 2017
  46. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey @Rtyper,

    Think I can get a hold of this today?

    Cheers,
    Colton
     
  47. minad2017

    minad2017

    Joined:
    Dec 1, 2016
    Posts:
    50
    hi,
    I am using "Lipsyncpro 1.4" with "Unity 5.6.1 and 2017.1.1".
    Although it was able to be used without problems until now, suddenly trouble has appeared, so please consult with us.

    When reading the audio file with the lip sync editor, "Clip Length" is not automatically adjusted and it is troubled.
    For example, even if you read a 3-second audio file, the default value of 10 seconds is read, and if you do Auto Sync as it is, an incorrect lip sync will be generated.

    The same problem has occurred in Unity 6.1 or 2017
    It happens even in Lipsyncpro 1.3 or 1.4.
    Is there a solution?
     

    Attached Files:

    • 1.png
      1.png
      File size:
      23 KB
      Views:
      1,205
    • 2.png
      2.png
      File size:
      37.5 KB
      Views:
      914
  48. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey @Rtyper

    Really trying to get a beta version of the game out.. Whats the word on the timeline support?

    Cheers,
    Colton
     
  49. rsouza

    rsouza

    Joined:
    Dec 1, 2011
    Posts:
    67
    Hello there.

    I purchased your LipSync plugin but I'm have some problems with expressions,

    I made several tests. First I created my own expression (it works ok in the preview) but when I run it by code it becomes like the screenshot.





    After this I decided to try a fresh version in a new project, using the pre built expressions, but the result is the same.

    My code is the following:

    Code (CSharp):
    1. public class OpenMouth : MonoBehaviour {
    2.  
    3.     public RogoDigital.Lipsync.LipSync woman;
    4.  
    5.     void Start () {
    6.         Open ();
    7.     }
    8.  
    9.     public void Open(){
    10.         woman.SetEmotion (woman.emotions[0].emotion, 1f);
    11.     }
    12. }
    If I click in the expression name it shows correctly.



    Am I doing something wrong? Please answer me as soon you can. I'm working in a really time sensitive project.

    Thanks in advance
     
  50. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    452
    This is very weird, the clip editor sets the length based on the audio clip chosen. I haven't seen it fail to do that before... Could you possibly give me steps to recreate the problem? I've tried myself but I can't get it to happen.

    Thanks for this! I'd fixed this bug in the main part of LipSync but I'd overlooked SetEmotion, and it was still using the old code. I'll send you a PM in a minute with a fix in it.