Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Discussion in 'Assets and Asset Store' started by Crazy-Minnow-Studio, Apr 23, 2014.

  1. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I'm fully into Salsa right now.
    I just wanted you to know I LOVE IT.
    I love how flexible and how easy it is to customize it to my needs!
    I suppose the majority of your users will just use the default one-click configs and call it a day.
    I don't. That's why I love you for doing something that flexible.
    I could kiss you now.
    Not really, but still. :D

    YOU'RE AWESOME!
     
    Crazy-Minnow-Studio likes this.
  2. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    hello there,

    i'm just using salsa a few days, but i run into a problem with some costum animations that interfere with the headbones.
    i have CC3 character and use the one-click setup and are humanoid setup

    for example.
    i have a character that moves the head 45 degrees left.
    but when using salsa that head movement is blocked. (i also turned of "head configuration")
    looks like it has something todo with Axis-fixed, Axis offeset scripts that are added to the rig.

    any clue how i can fix this?
     
  3. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    and how about adding the facial expressions from CC3 to salsa
     
  4. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hello Object-Null,
    Please check the documentation for Eyes. It explains the fix-axis bones and how to remove them. Just search for Fix Axis or Restore Axis in the Using Eyes documentation.

    Good luck on your project.
    D
     
  5. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    I am not entirely sure what you mean by this. All blendshapes from the CC3 models should be available to you in SALSA.

    D.
     
  6. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Glad you are enjoying SALSA!

    /*kiss kiss*/
     
    atomicjoe likes this.
  7. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    Thanks. I'll check that one then
     
  8. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    Yes they work great. But what I mean is dat cc3 and I guess incline also. Have alot of emotion presets. Like happy, sad, angry etc. Maybe there is some way to export them to salsa / unity aswell
     
  9. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    You know what I miss from Salsa?
    A preview option to run it without playing the scene.
    Because right now, adjustments must be done in play mode.
    This is not very convenient since the scene could be doing lots of other things and, also, adjustments are lost when exiting play mode.
    I use a neat asset to save play mode changes or right click and copy the component values and paste them once I exit play mode, but it's giving me issues with EmoteR when I do this and then replay the scene:
    Code (CSharp):
    1. NullReferenceException: Object reference not set to an instance of an object
    2. CrazyMinnow.SALSA.QueueProcessor.Register (CrazyMinnow.SALSA.IExpressionController controller, CrazyMinnow.SALSA.ExpressionComponent+ControlType controlType, System.Single durDelay, System.Single durOn, System.Single durHold, System.Single durOff, System.Boolean persistence, CrazyMinnow.SALSA.ExpressionComponent+ExpressionType type, LerpEasings+EasingType easing, CrazyMinnow.SALSA.ExpressionComponent+ExpressionHandler handler, System.Boolean animatorControlled, System.Boolean useOffset, System.Boolean isActivating, System.Single frac) (at <973273dd609b4cb0b3e1e4d2a714605f>:0)
    3. CrazyMinnow.SALSA.Emoter.EmoteComponentRoundTrip (CrazyMinnow.SALSA.ExpressionComponent emoteComponent, System.Single durationHold, System.Single frac) (at <973273dd609b4cb0b3e1e4d2a714605f>:0)
    4. CrazyMinnow.SALSA.Emoter.EmoteRoundTrip (CrazyMinnow.SALSA.EmoteExpression emote) (at <973273dd609b4cb0b3e1e4d2a714605f>:0)
    5. CrazyMinnow.SALSA.Emoter.RandomEmote (System.Single chance, System.Int32 numEmotes) (at <973273dd609b4cb0b3e1e4d2a714605f>:0)
    6. CrazyMinnow.SALSA.Emoter.LateUpdate () (at <973273dd609b4cb0b3e1e4d2a714605f>:0)
    7.  
    To reproduce this, just right click on the EmoteR component in play mode and press "Copy Component", exit play mode, and right click again on the EmoteR component and press "Paste Component Values".
    Then play again the scene and this error will appear when emotes are fired.

    Also, could you let the blendshapes be set to more than 100%?
    Unity manages unclamped blenshapes: you can put a number higher than 100% or even a negative one and the blendshape will continue in that direction, so Salsa can do it too. (in the skinned mesh renderer, the slider only goes from 0 to 100%, but you can actually specify any number you want)
    This is VERY useful to create emotes and visemes combining several blendshapes past their original limits.
     
    ammaralisabri and wetcircuit like this.
  10. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I would fix this myself, but since Salsa is inside a DLL... :(
     
  11. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Apology if I am misunderstanding, but there are toggles for preview in both SALSA and EmoteR.

    Screen Shot 2019-10-15 at 9.55.46 AM.png Screen Shot 2019-10-15 at 9.58.57 AM.png Screen Shot 2019-10-15 at 9.58.08 AM.png
     
    Crazy-Minnow-Studio likes this.
  12. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    wetcircuit likes this.
  13. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Ahh. I thought it must be something more. :)
     
  14. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hey @atomicjoe,
    I am looking into the error created when pasting values from a copied component. At this point, I am unsure what is causing it. EmoteR uses all the same subsystems SALSA uses. My immediate thought is the controller data is not being rebuilt appropriately on startup but I am not finding any evidence of that thus far. I will try to get that nailed down.

    As far as a preview mode for SALSA with audio, this may be possible, but would add some additional code to the main SALSA loop, regardless of whether this feature is being used or not -- due to the difference in time calculation options between editor and runtime. Although I could probably remove most of that with a compiler directive. Also of note (since I just built out a test system for trying this), as expected, the refresh rate is much more erratic due to the inspector refresh cycle -- it's a lot heavier. It still works, but I am not sure if it works well enough. I'll look into this some more, but no promises.

    Ah, the ol' blendshape overdrive discussion. We wondered how long it would be before that came about. Not sure on this one. We already had the core engine written when Unity announced this would maybe be a thing...and it was very experimental. We do have the capability to work with negative values, but it is all still within a normalized -1 .. 0 .. 1 range. This is mainly due to how all of the ExpressionControllers are integrated. The QueueProcessor handles all of the lerping computations and has no understanding of the individual controllers, by design, allowing a flexible plugin system.This one is a big maybe. Depends on how much re-writing of existing code this would need. I honestly haven't looked at it in a while. Might be easy...might be a lot of roto-rooting and potential breakage.

    D.
     
    atomicjoe and wetcircuit like this.
  15. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Great! :)
    I suppose you know it already, but you can easily write code that will only be compiled and run in editor by simply using:
    Code (CSharp):
    1. void SomeMethod() {
    2. #if UNITY_EDITOR
    3.    code that runs only for editor
    4. #else
    5.    code that runs only for builds
    6. #endif
    7.    code that runs on both
    8. }
    I use this all the time for debug code that I don't want to be compiled on builds and for running editor only methods from monobehaviours. It's super handy.
    More info HERE.
    Oh, I understand...
    That's the kind of stuff that seems simple to implement but in actuality is at odds with the internal organization and implies rewriting and breaking stuff.
    Personally I have found another way to fix my issues with blendshapes higher than 100% by defining the blendshape weight to 50% at creation. This makes the blendshape to be at full effect at 50%, giving me double the effect at 100%.
    (I can do that because I recreate the characters meshes and blendshapes completely in Unity on import, in order to merge materials and create atlases.)
    So don't bother to implement this just for me, it's ok :D
    Negative blendshapes would be cool though :p
     
    Crazy-Minnow-Studio likes this.
  16. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Yup, I'm aware of compiler directives -- but thanks! Trying to figure out some time calc nuances. The callbacks can be enabled in the Editor for non-editor scripts, but they don't fire on a normal game loop cycle. They update when the Inspector updates which is not remotely sufficient. So have to trick the inspector into updating more regularly, which i have done, but Time.time doesn't spin like it does at runtime. It only updates with certain calls apparently. Still trying to work out what triggers it to update. And I'd rather leave footy prints on as little of the original code as possible. Also, Unity appears to deprecating [ExecuteInEditMode] for [ExecuteAlways], which isn't a thing until 2018.3. Which means more compiler directives to work with, so I'm trying to minimize the use of it. Anyway, I've shelved it for now in a feature branch, will look at it with fresh eyes later.

    I believe I've fixed the copy/paste issue in EmoteR. The action was mucking around with the specialty EmoteR pools, so I've forced some cleaning in the startup routines. I will send you a build to test shortly. I was able to reproduce your issue and it is resolved in this fix.

    I'll take a peek at the blendshape overdrive feasibility (maybe) tomorrow.

    Thanks,
    D.
     
    atomicjoe likes this.
  17. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    You mean in the blendshape list? If so, yes, you can add them to the EmoteR config if you like and then trigger them randomly, repeatedly, as emphasizers, or manually (works great with the Timeline addon for SALSA/EmoteR).

    D.
     
  18. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    Edit: i got the fix for me. i deleted the bones that where added (CC_Base_Head_OffsetAxis and CC_Base_Head_FixedAxis) and reconnected the head bone in the head configuration. Now it does what i want.

    Hello there,

    well i checked the documention but i cant figure out how to make it work.
    i use another rig fro creating my animations and use a animator to put them on the characters.
    the issue here is that the bone structure hierachy changes when i add the one click CC3.
    this means that there head bones added that are not same as in my animation rig.
    (CC_Base_Head_OffsetAxis and CC_Base_Head_FixedAxis)
    i also can't change this option humanoid setup from the character because the bones are added after importing.

    if you want i can send you a project where you can see the difference in animations.
     
    Last edited: Oct 17, 2019
  19. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
     
  20. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hey there!

    Glad it's working for you now! Good luck on your project. Remember, you can easily remove the fix-axis bones from within the Eyes module.

    D.
     
  21. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    @Object-Null BTW, are you good on this or am I still not on the same page?

    D.
     
  22. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I think the problem was Salsa is inserting new Transforms that weren't there originally and he couldn't use animations that were done with the original character hierarchy in another model.
    Now he has removed manually the fix-axis transforms that Salsa inserted and it's working great.

    Why does Salsa change the original hierarchy anyway?
    It can generate lots of issues for character animations in Unity.
     
  23. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    The [Axis Fix / Axis Restore] mechanism is provided as an option to add corrective hierarchy above bones that are not correctly aligned to Unity's left handed coordinate system. When active, mecanim animation is canceled out on the original child bone so that control can be managed by the Eyes module. We've found there to be a lot of artist that don't factor the coordinate system in when designing characters.

    The correct solution is to always make sure your models bones are oriented correctly to the system you plan to use them in, but this provides an option when that criteria isn't met. Of course, if your characters bones are correctly aligned to Unity's left handed coordinate system or you have mecanim animations that you wish to control your character instead of the Eyes system, you can simply select the [Restore Axis] button in the Eyes inspector for each of the systems where you don't need it.

    Most of the mainstream character generation systems seem to be on board now with correctly oriented bones so it might make sense to default the one-clicks to be not fixed. Of course, you can also modify the setup script to match your needs as well. The fix is applied in the setup script using the Fix Axis API that's documented in the manual.
    https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/eyes/api/
     
    atomicjoe likes this.
  24. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    your explanation is working great :)
    but well it works for 'me' now.
    i lost the part from salsa where it can look to a target.
     
  25. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    ty for the information. so only by pressing the restore button should fix my issue?
    anway ill take a closer look after the weekend. and let you knoe if i did figure it out.
     
    Crazy-Minnow-Studio likes this.
  26. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    Hi I using salsa lipsync,and it work greate in editor.. my problem was when build in device.. the mouth wont sync like in the editor and its like in slowmo.. any fix? or I missed a setting
     
  27. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I'm making my own OneClick for my characters.
    So far so good, but I can't find the way to set a specific easing method for a lipsync shape.
    Actually, if I could just set them globally like with the Global Overrides options but by code, that would be great :p
    I want to change the default Cubic Out easing to Linear for each shape in my OneClick.

    Does the voice sound in slow motion too?
    On my side it works great on builds for PC and Android.
     
  28. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    audio was okay, just the mouth of the character wont open the way it was shown in the editor..its slower in the device., here my sample apk you want to try
    https://www.dropbox.com/s/cwmzcqoffx6ujcv/elon musk c.apk?dl=0
     
    Last edited: Oct 21, 2019
  29. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi,

    Please send your SALSA invoice number to assetsupport@crazyminnow.com. Also include the following:
    • Desktop environment
    • Unity version
    • SALSA version
    • Are you building to Android or iOS?
    • Are you using any SALSA add-ons, which ones and what versions?
    • Are the lips moving slow on mobile or not moving at all?
    • Have you tried a minimal scene with nothing but your SALSA-enabled character to isolate the for troubleshooting?
    Thanks!
     
  30. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    I'm using old version
    SALSA with RandomEyes
    version 1.5.5
    this was the problem
    lips moving slow on mobile and yes I already tried even the sample 3D_Public_Functions_CSharp scene
     
  31. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I can't find the way to set a specific easing method for a lipsync shape.
    Actually, if I could just set them globally like with the Global Overrides options but by code, that would be great :p
    I want to change the default Cubic Out easing to Linear for each shape in my OneClick.
     
  32. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi, You can find an example of setting the easing in code on the API Configuration Example page in the documentation.
     
  33. LightningIsAwesome

    LightningIsAwesome

    Joined:
    Nov 14, 2016
    Posts:
    3
    Hello. I bought your plugin and also Amplitude plugin(because I need to use it in WebGl). Any docs or setup guides how to implement Amplitude plugin for SALSA?
     
  34. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi,

    The documentation can be found on the product web page.
    https://crazyminnowstudio.com/unity-3d/amplitude-webgl/
     
  35. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Please send your invoice number to assetsupport@crazyminnow.com and answer all of the bullet point questions. We can't help you without a valid invoice and sufficient information to troubleshoot your problem. SALSA works fine on mobile, so we need as much information as possible to find out what's effecting normal operation in your project.
     
  36. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hello @LightningIsAwesome,
    Did you already read the documentation for AmplitudeSalsa on our site?

    We will be re-making the Amplitude video very soon. Currently it demonstrates using with the old version of SALSA, however, you will likely find the setup process works very similarly.
     
  37. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    its okay now, I cant get an invoice cause my boss purchase it, now i'm using v2 and works better now in devices
     
  38. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Hey @Crazy-Minnow-Studio , I have an interaction problem between the Eyes and Emoter modules.
    Let me explain this in detail because it's a little tricky:

    I have several Emotes that require the eyelids to be partially or totally closed, like: smiling, laughing, crying or meditate.
    If I just make a single blendshape for the Emote with the eyelids partially or totally closed baked into the blendshape mesh, then I have issues when the Eyes module blinks or tracks the eyes, because the blendshape of the Emote and the blendshapes of the upper and lower eyelids accumulate and make the eyelids pass through each other. (this is already noticeable with the default "Squint" emote when the Eyes module makes the character blink: the eyelids poke through each other)

    Of course, this is an inherent limitation of baking partially or totally closed eyelids into an emotion blendshape and then applying additional blendshapes over them, so, to fix this, I remade my emotion blendshape without any eyelid movement on it and then created an Emote on Emoter with 3 components combining 3 blendshapes: the emotion blendshape (without eyelids) plus the upper and lower blendshapes adjustments to fit the emotion.
    This way, the eyelids movement is always dependent on the upper and lower eyelid blendshapes, both on the Eyes module and on the Emoter module, avoiding stacking blendshapes for the eyelids and thus avoiding them to "over-morph" and pass through each other.

    Since Eyes and Emoter are part of the same system, I was hoping that Salsa would apply the Emote, move the eyelids to the adjusted Emote position by applying the amount of upper and lower eyelid blendshapes needed, and then the Eyes module would just use the current upper and lower eyelid blendshape values as the starting value for the blinking and eye tracking. That would make the "open" eyelid position the current Emote eyelid position and the "closed" eyelid position the Eyes eyelid upper and lower max positions.

    However, this is not the case: Currently, as soon as the Eyes module is activated, the upper and lower eyelid blendshapes are 100% controlled by it, canceling any Emoter adjustment to this blendshapes. This behaviour continues even after disabling the Eyes module: the eyelid blendshapes can't be set by Emoter.

    THE SOLUTION:

    We would need for the Eyes module to use the current Emoter blendshape value for the upper and lower eyelids as the MIN value of the Upper and Lower eyelid components inside the Eyes module.
    This way, the Emotes could close or open the eyelids using the upper and lower eyelid blendshapes and this wouldn't affect the Eyes functionality to blink and track the eyes motion. :)
     
    Last edited: Oct 24, 2019
  39. Parsec3d

    Parsec3d

    Joined:
    Apr 2, 2015
    Posts:
    27
    Hi!
    seems 2d texture mode is not working under LWRP and we are forced to use LWRP since we have a lot of things under
    shadergraph..

    is there a workaround?
     
    Last edited: Oct 25, 2019
  40. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    The suite uses the following prioritization:
    • LipSync is king and overrides all other expression types.
    • Emotes override everything but LipSync.
    • Eyes/Heads/Eyelids are all on equal footing but will be overriden by LipSync or Emotes.
    Blink is actually registered as an emote and when emotes conflict, the last registered wins.

    What you would probably want to do is disable the Blink sub-module prior to setting your emote in EmoteR. This would prevent the blinking mechanism from canceling out your emote settings. Then just re-enable it when your are finished emoting.

    Likewise, you could control blinking entirely within EmoteR if you wanted to, using either the Random or Manual pool options. Or a hybrid, using Eyes to control blinking, but disable the blink when you're working with your emote, and add blinking during the emote with a manual blink from EmoteR where following the blink you fire your emote again to make sure it is set back up.

    There are a lot of ways to handle this, but it is a complicated situation when several things are trying to control the same blendshapes.

    Hope that helps,
    D.
     
  41. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hello,
    We responded to your email on the same question.
     
  42. Parsec3d

    Parsec3d

    Joined:
    Apr 2, 2015
    Posts:
    27
    Thanks ! email with information sent.
     
  43. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Thanks for the deails but sadly it's not helping:
    As soon as the Eyes module has been activated once, it will never let any other Emote modify the eyelid blendshapes, even after disabling the Eyes module all together.
    Disabling and reenabling other Emotes doesn't change this, so I can't control eyelid blendshapes myself.

    Anyway, disabling blinking and eye tracking when using Emotes wouldn't be a solution anyway, since the Emotes stay activated for long periods of time and aren't just little oneshots (a character should keep it's emote expression until another one is fired) and having eyelid movement only when the character is emotionless and not when it's smiling is really weird.

    As you can see, it's not an easy fix on the user side and you should really consider implementing it on your side.
    I can do it myself if you tell me how to modify the eyelids Min values in the Eyes module on runtime. (the editor is ignored)
     
  44. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Eyelid tracking is persistent just like eye and head movement since it has to be always active. They can't be done intermittently or any animation that includes these control surfaces will conflict and you'll see undesirable movement between our processing tick cycle and mecanim's late updates. We are working on an update that will allow these persistent queue registrations to be unregistered when disabled, but the bottom line is that you can't have two systems trying to control the same control surfaces at the same time. If you want to control the same blendshapes that are used for blink, you'll need to disable eyelid tracking, or use different blendshapes and work around the additive effects of over driving the same vertices with multiple blendshapes.
     
  45. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    We're looking into possibly tagging blinks as their own type in the queue so they could merge back to where they were before the blink. We're also investigating an option to open access to change the min/max values at run-time on the blink run-time controllers. No promises on any of this since it's all just being investigated at this point, but we believe this would offer options to address the points you discussed.
     
  46. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Thank you very much for taking the time to address this issue :)
    I have made my own custom fix for this and it's working fine for me, but it has some prerequisites that not everybody can meet: you need 2 additional empty blendshapes in your character mesh. (although it could be redone to work without them, using a transform to store the values instead)
    I share this here to show how I have fixed it myself and in the case it could be useful for others

    In the 3D character model, I make 2 additional empty blendshapes to match the upper and lower eyelid blendshapes. This blendshapes don't move any vertex: they are empty. Their purpose is just to match the emotes eyelid blendshape values.
    The Eyes component only moves the regular upper and lower eyelid blendshapes.
    In Emoter, I set the emotes to use both the regular upper and lower eyelids blendshapes plus the new upper and lower empty blendshapes with the same values. (let's call them Dummy Blendshapes)
    Then I use a custom script to always correct the eyelid positions: if the eyelid blendshapes have values greater than the dummy blendshapes, it does nothing (that's when Eyes is blinking or moving the eyelids to track the eyes) but if the regular eyelid blendshapes are smaller than the dummy eyelid blendshapes value (which were set by the emote) then it corrects the eyelids blendshapes to equal this value instead. This way, the eyelids blendshapes will never have smaller values than the emote specified values but can have greater values for blinking or tracking the eyes.

    Here is the custom script that must be placed in the gameobject with the skinnedmesh renderer you want to fix (the face or whole body gameobject):

    However you have to set this script to execute AFTER salsa in the script execution order menu, in the project settings. (I have another custom script that makes this for me by code. That's the commented line there)

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5.  
    6. [ExecuteAlways]
    7. //[ScriptExecutionOrder(10000)]
    8. public class JoeSalsaFix : MonoBehaviour
    9. {
    10.     public string eyelidUpperBlendshapeName = "head__CTRLEyeLidsTopDown";
    11.     public string eyelidLowerBlendshapeName = "head__CTRLEyeLidsBottomUp";
    12.     public string dummyEyelidUpperBlendshapeName = "head__DummyCTRLEyeLidsTopDown";
    13.     public string dummyEyelidLowerBlendshapeName = "head__DummyCTRLEyeLidsBottomUp";
    14.  
    15.     private SkinnedMeshRenderer skinnedMeshRenderer;
    16.     private int eyelidUpperIndex;
    17.     private int eyelidLowerIndex;
    18.     private int dummyEyelidUpperIndex;
    19.     private int dummyEyelidLowerIndex;
    20.  
    21.     private void Start()
    22.     {
    23.         OnValidate();
    24.     }
    25.  
    26.     private void OnValidate()
    27.     {
    28.         skinnedMeshRenderer = gameObject.GetComponent<SkinnedMeshRenderer>();
    29.         if (skinnedMeshRenderer == null || skinnedMeshRenderer.sharedMesh == null) { this.enabled = false; return; }
    30.         eyelidUpperIndex = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex(eyelidUpperBlendshapeName);
    31.         eyelidLowerIndex = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex(eyelidLowerBlendshapeName);
    32.         dummyEyelidUpperIndex = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex(dummyEyelidUpperBlendshapeName);
    33.         dummyEyelidLowerIndex = skinnedMeshRenderer.sharedMesh.GetBlendShapeIndex(dummyEyelidLowerBlendshapeName);
    34.         if (dummyEyelidUpperIndex<0 || dummyEyelidLowerIndex<0 || dummyEyelidUpperIndex > skinnedMeshRenderer.sharedMesh.blendShapeCount - 1 || dummyEyelidLowerIndex > skinnedMeshRenderer.sharedMesh.blendShapeCount - 1)  { this.enabled = false; return; }
    35.         skinnedMeshRenderer.SetBlendShapeWeight(eyelidUpperIndex, Mathf.Max(skinnedMeshRenderer.GetBlendShapeWeight(eyelidUpperIndex), skinnedMeshRenderer.GetBlendShapeWeight(dummyEyelidUpperIndex)));
    36.         skinnedMeshRenderer.SetBlendShapeWeight(eyelidLowerIndex, Mathf.Max(skinnedMeshRenderer.GetBlendShapeWeight(eyelidLowerIndex), skinnedMeshRenderer.GetBlendShapeWeight(dummyEyelidLowerIndex)));
    37.     }
    38.  
    39.  
    40.     private void LateUpdate()
    41.     {
    42. #if UNITY_EDITOR
    43.         if(!Application.isPlaying) {
    44.             if (skinnedMeshRenderer == null || skinnedMeshRenderer.sharedMesh == null) { this.enabled = false; return; }
    45.             else
    46.             if (dummyEyelidUpperIndex < 0 || dummyEyelidLowerIndex < 0 || dummyEyelidUpperIndex > skinnedMeshRenderer.sharedMesh.blendShapeCount - 1 || dummyEyelidLowerIndex > skinnedMeshRenderer.sharedMesh.blendShapeCount - 1) { this.enabled = false; return; }
    47.         }
    48. #endif
    49.         skinnedMeshRenderer.SetBlendShapeWeight(eyelidUpperIndex, Mathf.Max(skinnedMeshRenderer.GetBlendShapeWeight(eyelidUpperIndex), skinnedMeshRenderer.GetBlendShapeWeight(dummyEyelidUpperIndex)));
    50.         skinnedMeshRenderer.SetBlendShapeWeight(eyelidLowerIndex, Mathf.Max(skinnedMeshRenderer.GetBlendShapeWeight(eyelidLowerIndex), skinnedMeshRenderer.GetBlendShapeWeight(dummyEyelidLowerIndex)));
    51.     }
    52. }
    53.  
     
    Last edited: Oct 27, 2019
    Crazy-Minnow-Studio likes this.
  47. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    By the way, for Daz characters, this can be done in Daz Studio by selecting the face object, Pose Controls, right-clicking on any blendshape -> EDIT MODE -> right-click again -> Create new Property -> call it what you want -> EDIT MODE again to exit edit mode.
     
    Crazy-Minnow-Studio likes this.
  48. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    The iClone OneClick has been updated to v2.1.6

    Changes:
    • Fixed bone rotation on OO viseme.
    • Added support for latest free Izzy model (apparently the one we had was a really old model).
    • Updated Eyes module bone and shape searching.
    • Fix-Axis bones have been removed for the head module. NOTE: if used with an animation, you will need to unlink the jaw bone in the mecanim setup/configuration.
     
    Last edited: Nov 1, 2019
  49. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    This is happening for Daz models too: by default Unity includes the lower jaw bone in the Animator Avatar.
    Because of this, when using motion capture animations with the character, the lower jaw bone was incorrectly rotated more than it should be. This was only noticeable when the character opened it's mouth, because the lower jaw bone doesn't affect the face but only the lower teeth and tongue inside the mouth. (at least in Daz characters)

    For reference, to correct this, you have to go to the character import settings, Rig tab, Avatar Definition -> Configure.
    Once in the configuration scene, you have to click on the "Head" button at the green character scheme in the inspector, select the "Jaw" press delete to remove the assigned bone.
    This way the lower jaw bone will no longer be affected by the Animator animations.
     
    Crazy-Minnow-Studio likes this.
  50. Object-Null

    Object-Null

    Joined:
    Feb 13, 2014
    Posts:
    70
    Hello,

    is it somehow possible to slower the speed for the overall mouth movements?
    (im using CC3 one click)