SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Also make sure you've changed the figure's rig to Humanoid

 

Thanks so much. I thought there was something I was forgetting. Been awhile since I used RandomEyes. Thanks again!

 

Yes, that part I got. Thanks!

 

Perfect! That was the problem. Thanks again, so much! Totally my fault.

 

Yes, that was the issue. I had not added blend shapes to the model. I did that and everything worked. Totally my fault. I had not used RandomEyes for awhile and forgot to do that. I thought there was something missing, but forgot how to activate it. Works great now.! Thanks!

 

Hello all,

Our web host is experiencing an outage, and Crazyminnowstudio.com is currently down. We're working closely with our web host to resolve the problem as quickly as possible. Sorry for the inconvenience and thank you for your patience.

Team Crazy Minnow

 

Crazyminnowstudio.com is back online, thanks for your patience.

Team Crazy Minnow

 

SALSA 1.5.0 has been submitted to the Asset Store. See the release notes for details.

https://crazyminnowstudio.com/unity-3d/lip-sync-salsa/release-notes/

 

We're very excited to announce the public beta release of our new SALSA ActionClips for Cinematic sequencer - SLATE. This free add-on provides essential SALSA and RandomEyes function access to the fantastic cinematic sequencer by Paradox Notion. If you're not familiar with SLATE, check the developers website for more information.

Check our blog post for all the details and links to the free download. We welcome feedback for suggestions and improvements.
https://crazyminnowstudio.com/posts/salsa-actionclips-for-cinematic-sequencer-slate/

 

We offer 1-click synchronization scripts for many of the mainstream character generation systems, but what do you do if you want to use SALSA with your custom made characters that don't have SALSA spec BlendShapes, or a character system we don't have a 1-click setup script for? This is why we created SalsaSync, a flexible universal free add-on for mapping SALSA and RandomEyes processed values onto your characters BlendShapes and bones. We love the ease of our 1-click setup scripts, but it's not possible for us to provide that for every custom character combination. SalsaSync trades 1-click ease of use for maximum flexibility, and a focus on providing you a tool to support the widest possible range of character configurations.

If you're already familiar with SalsaSync, or interested to know more, check the blog post below for details about our latest version 2.0.
https://crazyminnowstudio.com/posts/salsasync-flexible-salsa-map-for-custom-characters/

 

We posted here a while ago about doing some SALSA tests with character models from Freedom's Gate. These characters look pretty good and use our SalsaSync add-on (mentioned in the previous post) to create the linkage between SALSA and RandomEyes.

Check our blog post for all the details and don't forget to grab our SalsaSync add-on if you decide to give these models a shot.
https://crazyminnowstudio.com/posts/salsa-lipsync-with-freedoms-gate-characters/

 

TextSync is a free add-on for SALSA lipsync that adds text-based simulated lipsync to your SALSA enabled characters. Simply add TextSync to your SALSA enabled character, set the desired words-per-minute rate, and pass text to the TextSync.Say method. SALSA will simulate lipsync to the text for the duration calculated from the words-per-minute setting.

We also built a prefab to add to an Adventure Creator scene that hooks into the AC dialogue events and allows you to use the standard AC Dialogue Play speech action to deliver dialogue audio or dialogue text, and SALSA and TextSync work together to perform lipsync either way. We'll post more about this when get it wrapped up, published, and the blog posted.

Anyway, here is the blog post for TextSync with links to the download.
https://crazyminnowstudio.com/posts/salsa-text-to-lipsync-using-textsync/

 

Our Cinematic Sequencer - SLATE beta add-on has been updated to version 0.3. We added custom inspectors for ActionClips that have a length so that the width on the timeline matches the length or duration of the ActionClip.

Get the update from our downloads page using the link at the bottom of the page linked below.
https://crazyminnowstudio.com/posts/salsa-actionclips-for-cinematic-sequencer-slate/

 

Adventure Creator added official lip-sync support for SALSA in version 1.43. While this release added integrated support for Salsa2D, Salsa3D already worked in previous releases of Adventure Creator through the Dialogue : Play speech Action. Click here to read the announcement from ICEBOX Studios, the developer of Adventure Creator. In addition to the native support capabilities, we're excited to announce our new free SALSA Custom Actions add-on for Adventure Creator. This free add-on exposes essential SALSA and RandomEyes function access to the fantastic Adventure Creator system. We're also pleased to announce a TextSync integration option using our new Adventure Creator listener for TextSync add-on. Together, these systems offer automatic lipsync of dialogue audio and dialogue text using Adventure Creators native Dialogue : Play speech Action, and we think that's pretty cool!

Read the blog post and grab the goodies from the download link at the bottom of the page.
https://crazyminnowstudio.com/posts/latest-version-of-adventure-creator-supports-salsa/

 

Hello,

I just installed the latest UMA Version (2.5) and since then, all my UMA in the Scene have the character Name "UMA_Salsa_1" in the Scene and the first one takes all the speaking now...

I do not have the latest salsa installed. Should I do that now or will there be another update soon?

Thanks a lot and have a nice Weekend!

 

Hi Firlefanz73,

It sounds like you're using the [Mode : Runtime] option on the CM_UmaSync component, along with an UMA character generator script. If so, please be sure to set a unique name for the [Character Name] field on the CM_UmaSync component. If this doesn't resolve the issue, please send an email to assetsupport@crazyminnow.com with a detailed description of your setup.



Thanks,
Michael

 

SALSA 1.5.0 is now available on the Unity Asset Store.
https://www.assetstore.unity3d.com/en/#!/content/16944

Check the release notes here for details.
https://crazyminnowstudio.com/unity-3d/lip-sync-salsa/release-notes/

 

Hello everyone! We finished the initial release of our free SalsaDissonanceLink add-on!

We have been working with Placeholder Software to integrate lip-sync with network voice-chat. Dissonance Voice Chat is fairly new to the store, but they have great reviews so far and a product that is flexible and voice-chat-feature-rich. You will need the latest versions of SALSA (v1.5.0+) and Dissonance (v1.0.6+) to hook the two together using SalsaDissonanceLink.

Our first tutorial on setting up Unity Networking, SALSA, and Dissonance has just been published and provides a fairly detailed look at the process. Check out our blog post for links to the add-on download and the tutorial documentation.

 

Hello,
I've just updated to SALSA 1.5.0, bur now I'm getting this error:

Code (CSharp):

1.  
2. NullReferenceException: Object reference not set to an instance of an object
3. CrazyMinnow.SALSA.Salsa3D.TalkStatusChanged ()
4. CrazyMinnow.SALSA.Salsa3D.LateUpdate ()

I can't find where it's coming from, would you have an idea about it?

Thanks

 

Hello ahmidou,

Please provide a detailed description about your setup:
In what ways are you using the SALSA OnTalkStatusChanged events?
What add-on's or other assets are you using with SALSA?
What are the sequence of events to reproduce the error you're getting?

Thanks,
Michael

 

Hi Michael,
I'm using it with the microphone plugin, also I'm not using any blend shape, but instead I'm using the API to send the different "say" and "look" values in Playmaker that way:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. using CrazyMinnow.SALSA;
4. public class Lipsync : MonoBehaviour {
5.     public Salsa3D salsa;
6.     public RandomEyes3D randomEyes;
7.  
8.     //private Animator animator;
9.     public PlayMakerFSM playMaker;
10.     public float smoothFactor;
11.     // Use this for initialization
12.  
13.    
14.     // Update is called once per frame
15.     void Update () {
16.         string isTalking = salsa.say;
17.         if (isTalking != "Rest") {
18.             playMaker.FsmVariables.GetFsmFloat("saySmall").Value = salsa.sayAmount.saySmall * 0.01f;
19.             playMaker.FsmVariables.GetFsmFloat("sayMedium").Value = salsa.sayAmount.sayMedium * 0.01f;
20.             playMaker.FsmVariables.GetFsmFloat("sayLarge").Value = salsa.sayAmount.sayLarge * 0.01f;
21.         }
22.         else
23.         {  
24.             playMaker.FsmVariables.GetFsmFloat("saySmall").Value = Mathf.Lerp(0.0f, playMaker.FsmVariables.GetFsmFloat("saySmall").Value, smoothFactor);
25.             playMaker.FsmVariables.GetFsmFloat("sayMedium").Value = Mathf.Lerp(0.0f, playMaker.FsmVariables.GetFsmFloat("sayMedium").Value, smoothFactor);
26.             playMaker.FsmVariables.GetFsmFloat("sayLarge").Value = Mathf.Lerp(0.0f, playMaker.FsmVariables.GetFsmFloat("sayLarge").Value, smoothFactor);
27.         }
28.         playMaker.FsmVariables.GetFsmFloat("lookUp").Value = randomEyes.lookAmount.lookUp * 0.01f;
29.         playMaker.FsmVariables.GetFsmFloat("lookDown").Value = randomEyes.lookAmount.lookDown * 0.01f;
30.         playMaker.FsmVariables.GetFsmFloat("lookLeft").Value = randomEyes.lookAmount.lookLeft * 0.01f;
31.         playMaker.FsmVariables.GetFsmFloat("lookRight").Value = randomEyes.lookAmount.lookRight * 0.01f;
32.         playMaker.FsmVariables.GetFsmFloat("blink").Value = randomEyes.lookAmount.blink * 0.01f;
33.     }
34. }  
35.  

I't happening if I modify any parameter in either Salsa or RandomEyes while playing.

 

ahmidou,

Thanks for the details, we're investigating now. Please send an email to assetsupport@crazyminnow.com with your Skype account in case we need to have a look as your setup.

Thanks,
Michael

 

Cinematic Sequencer - SLATE just updated to 1.7.0 on 04/06/2017, and added an option to the Actor Audio Track to [Use Audio Source On Actor]. This option checks for, and uses, an existing AudioSource component on the Actor instead of instantiating one dynamically. It allows SALSA to work with SLATE's native [Actor Audio Track] [Audio Clip] ActionClip.

 

SALSA add-on for UMA 2.5+ Dynamic Character System (DCS) is now available. DCS seems like a much needed layer of abstraction to simply entry into using these characters.

https://crazyminnowstudio.com/posts/salsa-add-on-for-uma-dcs/

 

Time for another SALSA 2.0 pre-release demonstration. This one shows the results we're currently getting on a Fuse character, and demonstrates the new (work in progress) 1-click setup. You might notice that no sync script (i.e. CM_FuseSync) is needed in 2.0 since native mapping is possible on all of the character systems we currently support, with the exception of UMA (so far).

 

interesting… you have 8 phonemes in that video….

Is this all "just" audio detection, or are you also adding a text script to supplement the audio?

 

Yes both the MCS demo and this Fuse 1-click are using an 8 phoneme set that we think produces a fairly convincing result. However, the number is completely up to the end user depending on the desired look.

There is no text supplement, it's 100% lightweight run-time processing.

 

*swipes monitor with creditcard*

 

The SALSA 2.0 pre-release video posted above also shows a new component near the top called SalsaQueue, this is a new central processor for all blend activities (speech, eye movements, blinking, expressions, etc.). Moving blend processing to a dedicated queue provides several advantages:

• Shape conflicts are automatically resolved by priority. Fuse characters provide a good example of this advantage. Since they don't include enough BlendShapes to create a complete set of visemes using unique shapes, shapes must be reused in other visemes, and the queue automatically prioritizes resultant conflicts.
• A central queue for all blending means the blend code structures are unified for speech, eyes, and expressions. This results in more flexibility with less code.
• Blend activities from speech, eyes, and expression are registered to the queue only when blending is needed, which means the performance footprint consists of actively blending shapes only. In testing, we've found the queue usually contains around 3-5 shapes at any given time during speech on a Fuse character (lower with characters that have dedicated visemes shapes). Fuse male characters demonstrate this advantage quite well. They separate body, mustache, and beards into separate object with separate BlendShape sets. As you can see in the image below, this results in a massive 173 BlendShapes to make up 8 visemes that include BlendShapes from the body, mustache, and beard. Despite this ridiculous number, the queue is only processing a tiny subset at any given time.

 

Hi,

I have what is probably a silly question here, but I am using the latest version of the mic input script ("CM_Mic Input") and after updating to the Windows 10 creator update (It seemed to be working fine before) I can hear the audio from the mic input being played back.
Turning on the "Mute Microphone" toggle prevents this, but it also prevents the lip syncing from working. I am using a standard Morph 3D character system character (latest version) and SALSA seems to be working fine otherwise.

Any idea what's causing this?

 

Hi PhoenixAdvanced,

You should be able to use a mixer group to pass the audio data without the sound. We have a blog post on the topic at the link below.

https://crazyminnowstudio.com/posts...e.getoutputdata-no-data-when-source-is-muted/

Michael

 

That worked perfectly, thank you for that!

 

am getting this when importing salsa addons
type or namespace name `RandomEyesLook' could not be found

 

Hi zeb33,

SALSA 1.5.0 introduced a new random eye movement algorithm that looks more realistic. This change deprecated the old look method. It sounds like your using one of our add-on's and possibly don't have the newest version.

Can you send an email to assetsupport@crazyminnow.com with your SALSA order number and as much detail as you can about any SALSA related add-on's you're using?

Thanks,
Michael

 

strange, using salsa1.5.0 and only download addon from your site yesterday..will email you, thx

 

Hi, guys. I'm using SalsaSync2 and the new Playmaker actions.

I'm trying to build my character expressions, and it looks like I need to create RandomEyes Groups and trigger them with the RandomEyes 3D Set Group Duration action…. My issue is the face props (teeth, eyelashes) can't be added to groups. Is there maybe another way to do this? Maybe a setting in SalsaSync?

 

SalsaDissonanceLink add-on now supports Salsa2D as well as Salsa3D.

Hi wetcircuit,

Which character system is this?

Michael

 

Fuse.

 

The CM_FuseSync.cs script normally handles the synchronization of expression shapes to the associated eyelash and facial hair SkinnedMeshRenderers on a Fuse character. This gets a bit more difficult when the character system standard starts to drift as it has been doing with Fuse. SALSA 2.0 addresses this completely by mapping every shape to it's own SkinnedMeshRenderer reference, but it's difficult to implement something like that in SALSA 1.x without breaking a bunch of other stuff. I'll look into the feasibility of adding a mechanism to SalsaSync to see if we can bridge the gap until SALSA 2.0 is ready. Is this one of the newer Adobe Fuse characters with missing shapes, and that's why you're using SalsaSync instead of the Fuse 1-click addon?

 

It's not missing anything, but the teeth are in the hierarchy as their own object. (the eyelashes were always an issue for RandomEyes, I think)


(left is the built-in figure, right is imported basemesh figure)

This is a re-sculpted basemesh, but presumably any imported figure using the "creator kit" basemeshes (whether or not it's been resculpted), seems to be Fuse's import process converting the mesh regions into separate meshes.

Also SalsaSync is just better-er than Fuse 1-click…. More options. For some reason I lose RandomEyes control over many of the blendshapes with the Fuse 1-click.

(A workaround for me might be to trigger all the blendshapes with playmaker like your old Expressions action, but I was trying to do it the new RandomEyes way….)

 

Does the teeth object include BlendShapes and do they match the body BlendShapes, or is it bone controlled? Can you send the FitA14 and GalenaA1s models to assetsupport@crazyminnow.com? I'll have more time to explore options this weekend.

 

The teeth have 4 blendshapes (Smile_Right is useless, lol but the tongue is actually part of the teeth, so TongueUp works on the Teeth but not the Body)



I'll zip them up and send you a pm.

 

I can't believe you guys already have a fix for this up on your website….

Testing the new version of SalsaSync2.1 and it works perfectly! Simple solution. (I'll test the new version of Fuse 1-Click tonight but I know you wouldn't have published it wasn't working, lol...)



You guys are a credit to the asset store and the Unity community in general!

I have to applaud you, when I saw you had a Dissonance scene working, I was so happy – I haven't even tried it, but I was like YES!…. You guys have partnered or supported every voice, chat, and dialog option in the store, and all the figures too – maybe second only to Playmaker in partnered support...!

SALSA is the BEST!

 

Hello,

I am already using Salsa with UMA for my game, that works fine!

But now I want to create a Little presentation, where a guy is just standing on a platform and talks.
I cannot do this with UMA, because the target platform is Universal 10 (Hololens) and UMA does not Support it.

Can you point me to a free male human Avatar or if not some other Avatar that works together with unity? It is just a Little presentation, I would not like to have to spent Money or too much effort on it but it would be a nice Little Project with the hololens, thanks!

PS: The new Ethan demo Looks nice! But is there some other Avatar that Looks a Little more seroius maybe?

 

Got it working in no time, salsa is really great and the demo Scenes are really helpful!

You can tell everybody, Salsa works with the hololens!

If you know a more serious looking free or maybe cheap Avatar that works with salsa, please let me know.

Thanks

 

Firlefanz73,

Very cool! Here is a list of all the systems that have been tested, and work with SALSA.

http://www.crazyminnowstudio.com/unity-3d/lip-sync-SALSA/

 

I make tutorial
Import DAZ3D to Unity 5 with SALSA and integration with my asset ALVI (Game Maker for FPP/TPP game)

Alvi
https://www.assetstore.unity3d.com/en/#!/content/32323

 

Looks like a nice system, thanks for the SALSA workflow tutorial.

Michael

 

Will Salsa 2.0 be backward compatible with 1.5 setups? What's the ETA on 2.0?

 

Hello magique,

We don't have an ETA yet, but we have posted a SALSA 2.0 FAQ blog where we'll keep the latest info until release. See the section on "Will upgrading to SALSA 2.0 be difficult".

http://www.crazyminnowstudio.com/posts/salsa-v2-devblog/

Michael