SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

When 2.5 is released, we will also release updated one-click add-ons to address changes to the Eyes module. CC3 3.3 base and game models have already been successfully tested with the updates.

 

Just a quick nitpick, in SalsaTextSyncEditor, around line 35, EditorStyles.textField.wordWrap is set to true, but it's never set back to its original value, so all Unity text fields are wordwrapped if that inspector is viewed at some point, which almost made me go crazy trying to find a bug in one of my editor tools Would be nice to fix that in future versions.

 

Hi DED-Games,

Thanks for your feedback, we'll look into this.

 

I bought the SALSA Lip Sync asset from the Unity Asset Store. I was wondering if anyone could point me in the right direction to use the suite for FBX for unity animations downloaded from Mixamo.

 

It works on models not animations. You will need to use models that have all the head/mouth bones. Older fuse models needed to be downloaded with the extra bones.

 

Hi,

Is there a way to tell SALSA to ignore timeScale setting same way it works for Animator?

Best regards,
Nikita

 

Hello @OneCpublishing. I had an old branch that was worked up long ago to experiment with this. It never made it into the main branch, mainly from lack of customer support. However, it is a pretty simple implementation so I'll put it in for v2.5.0. It is highly experimental and pretty much untested, but should give you the option to ignore time scaling for the suite.

D.

 

Оk, thank you!

Best regards,
Nikita

 

Hi,

I have an issue at the moment where the eyes are not looking at the target that has been set. I set the target with lookTarget, the head rotates to this point but the eyes look above and to the right.

I've tracked it down to a clash between Salsa Eyes and the Animator as the eyes do look at the correct point when the character animator is disabled. I've ruled out any particular animation by clearing out the animator and just using a dummy animator.

Does Salsa Eyes use the Animator for IK eye movement and it's not reset in certain circumstances? Is there a correct way to set a new target for head & eyes that I'm missing?

Thanks,

Dave

 

Hi Spabbage,

Sorry about this. This is a bug where SALSA Eyes was unintentionally allowing the animator to slip in influence in the final calculated value. Barring any unforeseen issues, we are planning to submit our 2.5.0 update today or tomorrow, which resolves this issue. This is a pretty big update and introduces some breaking changes that will require you to reconfigure your character. After we submit, we'll publish the updated manual and support packages for 2.5.0.

 

Version 2.5.0 Submitted to the Asset Store!!!
SALSA LipSync Suite v2.5.0 has been in the works for a few months and is the culmination of many new features, usability and functional enhancements, bug fixes, and a near-complete re-write of the Eyes module. See the Release Notes and previous forum posts for full details. It looks like the Asset Store has been experiencing some lengthy turn-arounds. We did have to apply for a manual review this time so we are unsure how long it will take Unity to approve and release this version. We will keep you posted with anything we hear.

IMPORTANT NOTES:
As mentioned, the Eyes module has received a lot of changes to implement a more streamlined processing structure that doesn't rely on bone-axis-fix architecture. These changes result in breaking changes to current configurations and API usage. If you wish to use v2.5.0 in an existing project, please understand it will be necessary to reconfigure your Eyes modules and if you are using the API, it will be necessary to make a few code changes. The axis-fix option will be completely removed in the near-future and has only been left in the current version to allow customers an easy method to remove the "fix" scaffolding from their existing configurations.

Also mentioned in the previous post on this new version, we are incrementing the minimum Unity version to 2017.4.1 as per Unity recommendations. Likely in the near-future the minimum supported Unity version will increase further to 2018.4 as Unity removes support for older versions and begins to enforce minimum version requirements on published Assets.

There are a couple of conflicting legacy package files that need to be removed with the new version and we take care of that for you; however, you may see a Unity notification that a .meta exists without the base file. This is simply informational and when you clear it, you will not see it again. This notification will only appear when upgrading an older version.

Add-Ons and OneClicks:
Due to the new Eyes configuration requirements, all OneClicks have been updated and it will be necessary to ensure you are using the correct OneClicks with this new version. The appropriate OneClicks will be like-versioned with this 2.5.0 release. If you have a previous OneClick version in your project, you will see errors in the console until you get them updated.

There are two simple rules to keep in mind when browsing the Add-Ons/OneClicks:

1. The latest available Add-On version available is required for the latest version of SALSA LipSync Suite.
2. If an Add-On is versioned 2.5+ it requires SALSA Suite v2.5+. Several Add-Ons did not require updates to work with Suite v2.5.0.

OneClickBase is now included in the Core package -- you should always use the latest version available.

We will publish Add-on/OneClick updates shortly after the update is live on the Asset Store.

EXPECTATIONS:
If you are upgrading an existing project, make a backup of your project before you upgrade SALSA Suite!

We have spent a lot of time testing and trying to make this the best version of SALSA Suite ever. But please keep in mind, there are a LOT of changes -- if you find bugs or documentation issues, please let us know, we will knock them out as quickly as possible. Please ensure you've checked the Release Notes and latest documentation and ensure you have the correct Add-On/OneClick versions. It is best if you email us (assetsupport@crazyminnow.com), including as much detail as possible, screenshots and/or video, any errors received, and always include your Unity Invoice Number.

Documentation updates will be uploaded to our site shortly.

Thanks and Enjoy!
Mike and Darrin

 

And....version 2.5.0 is live! Remember, this update has a lot of changes -- SALSA and EmoteR should be drop-in replacements but Eyes *will* require reconfiguration and if you are using the Eyes API there are some deprecated calls that had functionality that simply did not exist any longer. Backup any existing project before upgrading to version 2.5.0! Read the release notes!

If you run into any issues, please contact us via email (assetsupport@crazyminnow.com) and include details (OS, screenshots, video, error-messages, invoice number!).

I will be loading add-on and OneClick packages to the downloads area today. Make sure you grab the correct version for the version of SALSA Suite. The 2.5.x add-on/OneClicks require SALSA Suite v2.5.x. If you have the wrong version, you will get errors and/or broken functionality. Read the download descriptions and notes for details.

The documentation is already live. If you find errors or information seems to be missing, please let us know.

Thanks and enjoy!
D.

 

All add-on/OneClicks that were updated and require SALSA LipSync Suite v2.5.0+ have been uploaded and linked. Remember...version numbers 2.5.x mean SALSA Suite 2.5.0+ is required. If there is not a 2.5.x version of the add-on, it did not require an update for operation with SALSA Suite 2.5.0. Please check the file descriptions and notes for more information on requirements. And check the linked blog/documentation post for each file for more details and instructions.

Thanks and enjoy!
D.

 

New SalsaMicInput Available (v2.5.0)

Hello everyone!
I've uploaded an update to SalsaMicInput that leverages one of the new features of SALSA v2.5.0. No longer is it necessary to have a pointer updater script. MicInput now plugs into SALSA's clipHeadPointer delegate and SALSA polls this value from MicInput instead. Simpler and more efficient -- removes a component and subsequently an Update() loop and only processes during the SALSA timing pulse. I also added some defensive code in MicInput and fixed a few potential logic conflicts.

There is also some new OnEnable/Disable() code that allows a cleaner disconnect from microphone input to switch to regular AudioClip input and back. Now you can just disable the component to switch the mic off for clip playback and then re-enable it to turn it back on again if you wish, or not.

NOTE: Please remember, the source is included in SalsaMicInput and this should be considered example code that you can use to jump-start your project. It is not intended to fit all scenarios or requirements -- you are free to update it as necessary for your needs.

Enjoy!
D.

 

Hey there, 2.5 is great, thanks for keeping the magic alive.
Still, is there a way you could unify the 2 separate install directories into 1 in future updates? It's a bit confusing to uninstall/reinstall Salsa since its files are distributed both in the root and the plugins folder. Thanks.

 

Hello @AGregori,
Glad you are enjoying v2.5. Thanks for the feedback. We will consider this, but I can't make any promises -- there are a few reasons why we chose this structure. Primarily, this is a Unity recommendation; of course, Unity is always changing their recommendations, but they have remained steadfast in keeping the core code in Plugins. There is also a benefit to having core code in the Plugins folder as far as efficiency and recompilation times -- plugins are in a separate assembly from the root Assets files. When we published v2, we had to relocate some of the add-on integrations into the root folder since some publishers do not adhere to the Plugins requirement and our add-ons could not see the integration from Plugins since it is an upstream dependency. Since we had to move some, we chose to move all add-ons into the Assets root to reduce confusion. So technically, if we moved to consolidate, we'd have to move the core out of Plugins and that is not beneficial or recommended.

One thing we have tried to do to help in this regard is make upgrading as easy as possible, generally not requiring a removal of previous versions prior to upgrading. In fact, v2.5.0 did introduce a zombie file from the previous version which we remove for you eliminating the need to have customers delete some or all of the files first. Additionally, the only files that should be removed prior to a build scenario should be the Examples folder under Assets/Crazy Minnow Studio/Examples (as long as they are not being used). All other files are required and adhere to Unity's folder structure for automatic build exclusion. Perhaps we could somewhat facilitate your request by adding more detailed documentation about the folder structure, both online and in the readme file? Let me know your thoughts.

Thanks,
D.

 

Makes perfect sense, thank you for the overview.

 

Hi there. First off, I just wanted to tell you how impressed I am with what you have created. This is a fantastic product. I am creating a new VR world in which I plan to use AIs hooked up through characters via your wonderful engine. I could probably spend three days figuring out my tiny problem, but I decided I'd ask you guys first. When I hit play, dude's mouth hangs open as if he is a 'mouth breather' when he speaks. I'm sure there's a quick fix to this, but I'm obviously too tarded to have found it. Dude is a CC3 2.1 character, exported with mouth open. I did apply the brand new one click- which worked like a charm- but for the mouth breather problem. I am using your awesome new 2.5 and Unity 2019.3- for the CC3 export files they created. Please help dude stop being a mouth breather. Lol. Thank you.

 

Hello, glad you are enjoying SALSA. Sounds like you probably have an animation export that has the mouth in the open state. If you play your scene with SALSA disabled, is the mouth open? If so, you'll need to eliminate that animation element. The other thing you could do is persist the SALSA expression components in the Queue -- under SALSA's settings section. I would call this more of a stop-gap measure until you fix the real problem in your character.

If that doesn't fix it, send us your Invoice number and some screenshots/video of the issue and we may also need to see your model or project.

Hope that helps,
D.

 

Thank you so much for the speedy reply. You are correct. I have an animation element causing the mouth to hang open. The problem was that in CC3, I forgot to add the T pose in the motion tab on export. Since I didn't manually add the t pose file, the program was 'remembering' when I had the mouth open. Lol. Thanks for getting back to me so fast. Problem solved.

 

We are not CC3 users ourselves, but I believe there is an option on the export to include animations. I know there are quite a few SALSA/CC3 users out there that will have much more experience and know-how than us, hopefully someone can help out in this thread. Otherwise, the CC3 forum should be able to help.

D.

 

I figured this out. One can export still animations from CC3 to Unity. Those animations must start with the t pose at the top of the list. For instance, one could have the t pose, then a standing pose, then a sitting pose, and so on. I learned after hours of experimentation that CC3 characters imported into Unity will not work with Unity animations. One gets the dreaded 'mouth breather' pose on the mouth. This means the pipeline must go CC3 to iClone to Unity- using iClone animations on the CC3 character. I guess if you guys feel like creating a fix for this it might be cool. It would save CC3 users having to go from CC3 to iClone to Unity. Apparently the animations that work on the CC3 characters are specific to iClone, which is fine. They are super awesome characters. Honestly the only issue with using 'regular' Unity animations with your system is that 'mouth breather' issue. I hope I just saved everyone hours. Lol. Until there's a fix, sorry my dudes but if you want CC3 animations you gotta do the pipeline.Luckily, I already have tons of iClone animations so it's not a problem. Just thought you might like to know. Your work absolutely rocks, by the way. None of this takes anything away from it. Keep making it better and better. You have a hit on your hands.

 

This is a CC3 character I created and exported to Unity 2019.3 in a standing pose, animated with SALSA 2.5 and the CC3 2.5 onelick.

 

As an addendum to my previous posts concerning the pipeline from CC3 to Unity, I just learned something wonderful. If one has newer versions of 3dxchange and IClone 7 installed, one can actually export animations directly from CC3 to Unity. The extra step of using either iClone or 3dxchange does not appear to be needed, however, I do think both programs must be installed. I was able to export an idle animation under the t pose animation in CC3. When I imported it into Unity via the file system they made for 2019.3, it worked perfectly. Dude is not 'mouth breathing' and speaks perfectly with your software. This means we can all finally achieve 'the dream' of having CC3 characters walking and talking properly inside Unity. In terms of adjusting or editing motions, I would suggest UMotion Pro. So in short, we can actually skip the iClone/3dxchange step with what I just mentioned. Just put your existing iClone animation UNDER the t pose animation when you export to Unity. That's critical. Like I learned the hard way, the CC3 animations are indeed specific to CC3 characters. Regular Unity animations will leave the jaw hanging open- which looks super odd. This is a bit inconvenient in that only iClone animations can go on CC3 characters but that's the way it goes right now. I have not experimented to see just how many iClone animations I can fit on a CC3 export. That will be next. Hope that helps. If anyone can find a way to make the regular Unity animations work properly with CC3 characters- without the dropped jaw- please let us know. That would be awesome.

 

Got dude walking and talking in Unity. Very happy with it. Just one strange thing. If you zoom in on dude's lips as he talks, he looks somewhat jittery. Are the meshes fighting each other? It's not perfect yet, but it's not horrific. I literally cheered out loud when I hit play. Lol. One other question for you geniuses. Is there any way to have it so the first words don't get cut off when one hits play? I seem to miss part of the first sentence as it 'fires up.' At any rate, this is moving along super well- due to your awesome software.

 

It look as though your animation may still be conflicting. You could edit your animation file in Unity and delete all the key frames that are applying undesired results (i.e. animation to parts you want animated by SALSA). This is the more efficient approach since you eliminate the conflict. Alternatively, you could possibly use SALSA persistence (see the manual), but this is not as efficient as simply solving the conflict. Also, you can unlink the jaw bone from the character avatar as another option to prevent mecanim from animating the jaw open based on your animation file values.

 

You can hit pause, then play once all of Unity's systems have spun up. The problem occurs because Unity spins up the audio subsystems first (and you hear sound) prior to releasing other systems, namely the one where we can access the audio data in the AudioClip...the pointer for the playhead doesn't update for a second or two. This shouldn't be an issue in a build.

Hope that helps!
D.

 

Thanks for pointing out the link. Unity broke all of their AssetStore links recently and tracking down relevant links has been fun...

Yes, we are working on a product demo video and realize it is an important part of customer decision-making.

 

Thank you for all the suggestions. I will be working on them shortly. That's great that you'll be working on the 'firing up in runtime' issue, as well. The reason that fix will come in handy is because I want to get my characters triggered by AI chatbots in game. Like say the NPC is sitting at a desk in a room and you, the player in VR walk in. You say hello to the chatbot- which is my dude or another character I create. That character will definitely need to respond quickly to not break the illusion. As for the animation problem, I am going to be working on your first suggestion. That one sounds like it will work. One quick question. I am working on a female character and I was wondering if you have a 'promo voice- female' audio clip I can use to test with. I am betting you do, since you named the current one male. Thanks again for your speedy and dead on replies.

 

Hi,

is it possible to use SALSA with Game Creator?

Thanks

 

Hey @mmckenney, just to clarify, we are not working on the issue with running in the Editor. This is a Unity thing that is just a thing. In a build, there should not be any issue. And, once Unity is up and running, everything is good. The only delay is when you hit play from within the Editor. After that, it's all good -- or at least it should be. You can test this by attaching a GUI button (or some other mechanism) to your scene that starts your character up.

Yup, in the Example Pack available in the SALSA Suite downloads area on our site.

Hope that helps,
D.

 

Hello,
There is currently no specific integration for Game Creator and it is not currently on our roadmap. However; if you can control an AudioSource within Game Creator, you can control a SALSA-configured character's ability to start/stop speaking.

D.

 

Anybody getting this on daz oneclick
ArgumentNullException: Value cannot be null.
Parameter name: componentOrGameObject
CrazyMinnow.SALSA.OneClicks.OneClickDazEditor.OneClickSetup () (at Assets/Plugins/Crazy Minnow Studio/SALSA LipSync/Editor/OneClicks/OneClickDazEditor.cs:51)

 

That error generally happens when you haven't selected your GameObject in the hierarchy prior to attempting to apply your OneClick.

D.

 

I need a holiday

 

Still getting this even with lower jaw deleted the mouth is working low eye lids using 1 click Dax with Daz gen

 

Hi Gatskop_Software,

Can you verify that you are using SALSA 2.5.0 and the latest Daz One-Click setup package? If so, please send your DAZ model and SALSA invoice number to assetsupport@crazyminnow.com and we'll take a look.

Thanks!

 

Yes I use 2.5.0 and did send you my invoice number

 

Great, and the model?

 

Daz Genesis 2

 

Right, but we need a copy of your model since SALSA has already been thoroughly tested with DAZ. We need to see if there's something different on your model.

 

Did mail google drive share to model check email

 

See my email response. The model works fine for me using the DAZ one-click setup.

Michael

 

Hi,

I cant find any help for Make Human, no youtube video no post, so please i want to use salsa with makehuman

thanks

 

Hi ELDEVELOPER,

There is no official one-click support for Make Human. With interchangeable rigs, there's simply no way for us to know what the structure of your characters rig will look like. That being said, as long as your character has parts that can be animated (i.e. bones, or blendshapes, etc), then you can likely animate it using SALSA. Additionally, once you get a manual configuration working the way you like, you can also build your own one-click setup using your preferred parameters.
https://crazyminnowstudio.com/docs/salsa-lip-sync/addons/custom-one-clicks/

As for how to configure your character, similar principles apply to all 3D characters. The general overview section of our documentation provides a good look at the general movement blendshapes we use for all of our one-click setups.
https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/overview/

 

Version 2.5.1 Available on the Asset Store!!!
SALSA LipSync Suite v2.5.1 has been released on the Asset Store and primarily fixes a couple of bugs, but also removes all of the underlying Eyes axis-fix architecture. Please see the Release Notes for more details. This is the recommended version for all new projects. For existing v2.5 projects, we also recommend upgrading to this version. For v2.4 and earlier projects, we recommend upgrading to this version, but understand it will be necessary to reconfigure Eyes modules and if you are leveraging the Eyes API, there are a few small changes to make.

IMPORTANT NOTES:
As a reminder, the Eyes module has received a lot of changes to implement a more streamlined processing structure that doesn't rely on bone-axis-fix architecture. This resulted in breaking changes to current configurations and API usage when upgrading from the 2.4 and earlier branches. If you wish to use v2.5 in an existing v2.4 (or earlier) project, please understand it will be necessary to reconfigure your Eyes modules and if you are using the API, it will be necessary to make a few code changes. The axis-fix option has been completely removed in this version. If you have existing 'fixed-axis' configurations, please remove them via the Eyes module option prior to upgrading to v2.5.1. If you do not, you will need to manually remove the fix-axis bones, re-parent the original bones, and remove any invalid components on the existing bones left by the deprecation of the fix-axis feature.

EXPECTATIONS:
If you are upgrading from a version prior to v2.5.0, read the Release Notes, make a backup of your project BEFORE you upgrade SALSA Suite, remove existing axis-fix implementations, and please ensure you are using the latest version of all add-ons and OneClicks for compatibility.

Enjoy,
D.

 

Hi, I want to use SALSA with UMA but I have my own UMA race (Fuse Character). The problem is that I can't add the blendshapes before the start because the UMA character is generated at runtime. Is there a simple way to load the blendshapes at runtime into the Viseme Configurator ? It is simply a UMA based fuse character with blendshapes.

 

i have written this little script that loads all skinmeshes into the viseme controller. it seems to work too, all viseme have the right skinedmesh now but unfortunately the mouth doesn't move. i hope you understand what i want to say, my english is not very good.
https://ibb.co/GPgS6ny

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using CrazyMinnow.SALSA
5.  
6. public class UMA_Fuse_Salsa_Handler : MonoBehaviour
7. {
8.     [SerializeField] private Salsa salsa;
9.     [SerializeField] private SkinnedMeshRenderer UMAMesh;
10.  
11.     public void InitializeSalsa()
12.     {
13.         foreach (Transform eachChild in transform)
14.         {
15.             if (eachChild.name == "UMARenderer")
16.             {
17.  
18.                 UMAMesh = eachChild.GetComponent(typeof(SkinnedMeshRenderer)) as SkinnedMeshRenderer;
19.                 Debug.Log("Child found. Mame: " + eachChild.name);
20.             }
21.         }
22.         foreach (var viseme in salsa.visemes)
23.         {
24.             foreach (var cntrlVar in viseme.expData.controllerVars)
25.             {
26.                 cntrlVar.smr = UMAMesh;            
27.                 {
28.                     cntrlVar.smr = UMAMesh;
29.                 }
30.             }
31.         }
32.     }
33. }
34.  

 

Hello! OK, a couple of things to note...first our UMA implementation expects a Dynamic Character Avatar to work with and subsequently the UMA Expression Player, which works with the UMA bone rig to simulate a blenshape-like atmosphere for UMA characters. If you have blendshapes on your character already that you want to use, you can still do that, but you won't use the SALSA Suite UMA controller type -- which from your second message sounds like you are not doing, so good. Probably you should take a look at the runtime documentation to get a feel for how to setup a runtime character. You'll have to combine normal runtime character setup with UMA character setup since you'll want to respond (I would think) to the 'character created' UMA event. Ultimately, it would go something like the following:

1. Spawn your UMA character (wait for and then respond to the UMA character created event).
2. Configure your character for SALSA, EmoteR, Eyes as needed.
3. If your character is truly a Fuse character, you can probably apply the Fuse OneClick via script.

Check out the documentation for more details:
https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/further-reading/runtime-setup/
https://crazyminnowstudio.com/docs/salsa-lip-sync/addons/one-clicks/

Hope that helps or at least gets you started in the right direction.
D.