SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

I'm not familiar with that asset but after taking a quick look at it, it seems to modify the audio after the AudioSource has processed it, then output to the speakers. This means the lip-sync would not be in sync since the Voice Changer Filter asset add's roughly a half second delay, while SALSA operates in real-time. Also the asset is compiled, so this is an assumption of what's happening. You'd have to talk to the developer of that asset to know for sure.

Michael

 

Could you at least add pitch as an optional Parameter to your Play method? Or maybe even more? Something that changes the voice but not the Speed and delay?

That would be great! If not, it will be okay, too, I like my lypsynch UMA characters

 

You can already adjust pitch using an Audio Mixer and a Pitch Shifter effect, these are built into Unity.

• Window -> Audio Mixer
• Click the + to add a new Mixer
• Select Master Attenuation in the new Mixer to view the Master inspector.
• Click Add Effect -> Pitch Shifter
• Move the Pitch slider to your preference
• On your Audio Source where you want to adjust pitch, select your new Mixer Master as the output.
• There are several other effects available in the Master inspector.

https://docs.unity3d.com/Manual/AudioMixer.html

Michael

 

Thanks a lot Michael! If you ever run out of new ideas for salsa, some voice-changing stuff would be great and even fun for others, too I guess. Thanks a lot for your hint, I will see if this works out for me.

 

Hello,
Couple of questions... Is SALSA compatible with dearVR?

And next, is it possible to use it in realtime with multiplayer voice chat as the sound source? such as photon voice or other implementations?
Thanks,
Garrett

 

Hello grahag11!

Unfamiliar with dearVR, however, if you can grab an AudioSource from it, it should work with SALSA. Keep in mind, 3D audio tends to reduce the dynamic nature of the audio (diminishing the amplitude) the further the 'source' is from the 'listener' -- in turn, reducing the perceived lip-sync (IOW - the lips don't move as much).

Yes, it works in a photon voice configuration. We have a blog post and video demonstration of it on our website.

Hope that helps!

 

Hello. Wonderfull package. But I need a litle help. I didn't find a workflow with lod characters. We use Autodesk characters which have 3 lods. I added Salsa on lod 0, work excelent, but if the quality is low default is loading lod 1 which haven't salsa. I have to put salsa on all lods, or on lod groups, it's not inefficient? Thank you.

 

Hi Bulisor,

Thanks for your purchase, please send your invoice number to assetsupport@crazyminnow.com and we'll be happy to help.

Thanks,
Michael

 

Last update was January 2016. Does Salsa work with Unity 5.5?

 

Hi Deckard,

SALSA is tested and compatible with Unity versions from 4.6 to 5.5, and on the following platforms: Windows, Mac, Linux, Android, IOS, Windows Store (8 & 10 [.NET Core]), WP8, and WP10. It is not compatible with WebGL due to limitations in the Web Audio API that Unity uses for their WebGL implementation. A minor upgrade to 1.5 is nearing completion, and a major upgrade to 2x is in the works. We'll be discussing the upcoming features of the 2x path soon.

Thanks,
Michael

 

Hi, thanks for the information.

 

Hi folks,

It was brought to our attention that Morph3D released a 1.6 update the includes some pretty major changes. We have identified that they are no longer using Unity's built-in mesh-based BlendShape system, but rather a custom component system. This completely breaks our MCS 1-click add-on for these characters since it mapped to standard BlendShapes. We are evaluating the feasibility of a revised add-on to support the new system, and will keep everyone posted here.

Sorry for the inconvenience, and thanks for your patience as we dig our way through the revised system.

Michael

 

Morph3D 1.6 does indeed still use standard mesh-based BlendShapes, they just stream in only the shapes you wish to use to help optimize the heavy load of so many shape options...a very smart option. We'll tweak the current 1-click setup to adapt to the changes and get it posted to the downloads soon, and the upcoming SALSA 2x branch should be able to handle the changes natively with loads of new options and without any add-on's.

 

That made me salivate.

 

LOL...good, we're pretty excited about it to.

 

The new MCS 1-Click setup for MCS v1.6+ is now available. After logging into our downloads page, you will find two versions:

MCS pre-v1.6
MCS v1.6+

Visit our MCS 1-Click setup blog post for more details.
https://crazyminnowstudio.com/posts/using-mcs-characters-with-salsa-lipsync/

 

Hi,

I downloaded Uma 2 package and i want to use my microphone for lip synchronisation. I saw this video
But i didn't found "CM_UmaLiveMic" script in any unitypackage.
Someone knows in which unitypackage i can find it?

Thanks

 

micInput-lite is available in the downloads section of our website. See the minInput-lite blog post for details and a download link at the bottom of the page.

http://www.crazyminnowstudio.com/posts/real-time-microphone-input-for-salsa-lip-sync/

Michael

 

Thanks! it's work well.

 

We have discovered a bug/behavior in MCS’s 1.6+ update that reorganizes BlendShapes when new shapes are added to a Morph 3D character. The shape order is not maintained upon Unity Editor reload. This behavior breaks our 1-click SALSA setup. We have released an updated 1-click (v1.2) for MCS 1.6+ which addresses the issue by adding a “Remap Blendshape Indexes” button above the shapes in McsSync.

If you add new morphs to your character, after you close Unity and open it again, the shapes will likely not be mapped correctly. Simply click the button to remap the shapes if the order has changed.

https://crazyminnowstudio.com/posts/using-mcs-characters-with-salsa-lipsync/

Hope this helps you out!

 

Hello Michael,

I have problem with a blendshape, as you can see on the picture when the mouse is open the skin is not separated properly. It is approx. 1 year old Fuse 1.3 Character with blendshapes from the mixamo pipeline.
Do you have a suggestion how i can fix this issue easily?

 

Hi Hormic,

You could just exclude the problematic shape from CM_FuseSync, otherwise you'd likely need to import the model into your favorites 3D editor to actually fix the bad BlendShape.

Michael

 

Ok, thank you,
Unfortunatly i have no really clues about blendshapes on how to locate and exclude them. I guess exclude something in CM_FuseSync won't help me that much, cause excluding the body makes no sense and opening the mouth is crucial so cant exclude that anyway. So the solution i'm capable of is try to reimport the model and hope for the best.

 

I meant you could remove the problematic shape from the CM_FuseSync inspector, by clicking the [X] button next to a given shape. This won't fix the shape, it simply makes it so that SALSA doesn't use the shape. MouthDown is used in the SayMedium shape group and MouthOpen is used for SayLarge. Your screen shot didn't specify which of those two shapes broken so I'm including info on both. If it's MouthOpen, you will likely need to select another shape that still gives you a lip-sync look you like since it's the only shapes used in the SayLarge group.

 

Ok, thank you very much, yes mouth open is the problem, but i think its hard to find a satisfying workaround.
Cause you could see every time the mouth opens a bit that there is something wrong. so i guess i will evaluate the model and reimport it.

 

Ok, just to let you know and maybe others with the same issue.
I did a model optimization with the fuse decimator -> https://www.mixamo.com/decimator
I guess this was the reason that the blendshapes screwed up, all is fine with the original model.

 

Hello,

I just bought salsa and maybe your videos on how to use it with character creator assume I know more than I do. I imported my character from Autodesk character creator and dropped it in my scene and then ran the one click setup. I dropped my audio file into the correct location like you did on the video but I do not have a play button..... I attached a photo so you can see what I am talking about.

 

Hello sunseeker1988,

Thanks for your purchase. The play button appears when you run the scene.

Michael

 

Thanks for the quick response. I am very new to character mouth animation. I have the option to select the type of facial expressions in Character creator as, Facial blendshapes or facial Bone Rig. Which one should I choose to be generated with my character to work with salsa? after I drop this character into my scene, what do I have to set up on him as far as animation settings ect to get him to talk with salsa?

 

Hello! You would want to select blendshapes. For lipsync, you simply need to export your character with blendshapes, add it to your scene, then add SALSA via the 1-click option just as you saw in the video. There are no other settings to configure.

Hope that helps!
Darrin

 

To those using SALSA with MCS characters, changes to the MCS armature required us to change the way we search the hierarchy for the left and right eye bones to ensure they map correctly to CM_MCSSync.

Please download the MCS 1-click setup version 1.3.

https://crazyminnowstudio.com/posts/using-mcs-characters-with-salsa-lipsync/

Thanks,
Michael

 

Super cool stuff. I ts working now. Amazing how this software works. I am having an issue trying to get my characters mouth to animate a little better. hes barely opening his mouth and seems to have no to poor saymedium, saylarge blend shapes. I played with every property and I cannot get him to open his mouth wider or with the other blend shapes. Your video for a CG character looks much better than mine. Will the audio file im playing affect the blend shape look depending on the audio quality? not sure what im doing wrong. I want to show this software off at a unity meetup im going too this weekend.

 

Hello sunseeker1988!
Yes, the audio file will certainly have an affect on lip synchronization. The more dynamic your audio, the better the lip-sync effect. You can compensate for this to a certain extent by adjusting the triggers for saySmall/Medium/Large. If your audio lacks good amplitude dynamics, adjust the triggers (medium & large) to lower values. Also ensure your range of motion is set to an appropriate number, typically 100% -- if this is too low, it will impact the extents of the blendshapes used for animation.

You can try using one of the included audio files (military moves is a good choice) to see if the character responds better. This will aid determination of whether it is a settings or audio issue. Even with poor audio, the available settings options can generally be used to great effect.

Hope this helps!
Darrin

 

Greetings,

Excellent product. However, I'm trying to get Salsa3D to initiate a 'play audio' (thus triggering the lipsyncing) with a compatible character using Slate's cutscene creator. Do you know how I might go about doing this? I've created an actor properties track, Salsa3D shows up, but none of the properties seem to fire the audio.

 

Hello Mythcons,

We're glad you're enjoying SALSA. We have created a custom Action Clips add-on for Slate that has not been released yet. If you'd like to participate in a closed beta, please send an email to assetsupport@crazyminnow.com requesting access and include your SALSA invoice number.

Thanks,
Michael

 

SALSA 2.0 isn't released yet, but here is a quick comparison of the quality improvements to our real-time automatic lip-sync processing in 2.0 prerelease.

 

Hey, we recently bought SALSA and RandomEyes for a project, and I'm very impressed! I had thrown something simple together previously based on amplitude analysis and a linear blend between two animations, but this does a much better job.

I have one major blocking issue though - I can't seem to get it to work alongside Mecanim on our 3d character at all. If the animator is on, it completely overrides SALSA. I've tried masking parts of the mesh (from the head up), but it won't work unless the animator is disabled entirely.

Am I missing something? I'd love to be able to additively blend SALSA on top of our hand-crafted animations.

 

Hi SoapboxCloudBuild,

This issue, along with a few others, and some usability improvements are included in our 1.4.1 patch release. I'm working to try to get it submitted to the Asset Store this weekend. If you need it sooner, please send your invoice number to assetsupport@crazyminnow.com.

Thanks for being a SALSA customer!
Michael

 

Great, and thanks for an incredibly speedy response! Look forward to the update!

 

Looks great. Is 2.0 a regular free upgrade? When can we expect a release?

 

We believe it’s important to do our best to minimize the impact to existing SALSA implementations when rolling out new capabilities, and we have worked hard with the 1.x line to preserve backwards compatibility throughout its development, while adding new features, add-on's, and workflows. We understand the importance of a stable base to build upon, and SALSA 1.x has provided that for three years.

However, this hard line approach can also be an obstacle to rolling out product enhancements. We believe the design improvements and feature and flexibility gains in SALSA 2.0 are significant, and warrant our first departure from this hard line stance on maintaining backwards compatibility. It will not be possible to upgrade to SALSA 2.0 without breaking existing project implementations, and we definitely do not want people accidentally upgrading to 2.0 without being fully aware of the new implementation requirements. For this reason, and because of all of the work going into 2.0, our plan is to increase the base price from $35 to $39 for new customers, and offer upgrades to existing customers for the $4 difference. We consider this a safety mechanism to prevent accidental upgrades, and a modest and justifiable price increase for all of the work being done.

Thanks,

Team Crazy Minnow

 

No problem. I will consider the $4 well spent.

 

I have an issue. As you may know, Adobe now owns Mixamo. They also have an new version of Fuze. I created a character, and when I try to use RandomEyes, when I click the Auto-Link on the second RandomEyes, nothing happens.
I also tried the 1-Click Fuse workflow, and again there is nothing linked. Thanks Jd

 

I use Adobe Fuse. I think maybe the problem is that one of the Fuse base bodies has renamed the eyes "Default" and the others still use "Eyes". I noticed the difference in some of my male/female figures…. I don't know if that is your specific issue, with the 1-click.

 

Hi jdscogin,

When using any Fuse character, the Auto-Link button on the RandomEyes3D inspector is not applicable since RandomEyes3D uses BlendShapes for eye control and Fuse uses eye bones. The eye bones are instead linked in CM_FuseSync and the RandomEyes3D BlendShape values are retrieved and translated to bone transform translations in CM_FuseSync. As wetcircuit said, if Adobe renamed the eye bones, that would certainly make the CM_FuseSync script fail to find them since it's looking for bone names ending in "LeftEye" and "RightEye". If eye bone naming is the issue, please let us know which Fuse base model you're using so we can investigate and make applicable changes to the 1-click add-on.

Thanks,
Michael

 

Hi wetcircuit,

Do you happen to remember which base model you were using when you discovered the "Default" eye bone name change?

Thanks,
Michael

 

Ahh, I do not mean to add misinformation. The eyeball mesh itself is renamed "default", but the bone name is still the same (as far as I know). From your description, it should not matter to RandomEyes.

 

Still having issues. Should the One-Click version still have shapes? Because I see none. An when I press the Play button on the Salsa 3D (Script), it plays the audio, but no movement at all on the character. I tried changing the Default to Eyes and changed the Mesh to Eyes, and the character's eyes bugged out. Kinda of funny though.

 

Hi jdscogin,

SALSA does not add BlendShapes to your character, they must already exist on the character. Since you're using Fuse, you need to upload the character to the Auto-Rigger and select the option to include BlendShapes on the Adobe/Fuse website.

Thanks,
Michael