SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

dialogue system and UMA integration?

 

I'll be working on Dialogue System integration this week.

 

UMA-DCS integration is planned and in our queue of to-do's.

 

Awesome. BTW are you familar with CharacterCreator3? I'm using CharacterCreator3 and UMA. Is it possible to use Salsa for CC3 UMA characters? UMA traditionally use bones for facials but CC3 uses blendshapes. I guess it's possible to use blendshapes but setting them up properly will be painful. I'm wondering if you will provide some sort of blendshape templates for CC3 to be used for UMA. Thanks.

 

We are familiar with CC3, our iClone 1-click for SALSA v1 was adapted to also work with CC3 back in October 2018. CC3 integration is also planned and in our queue of to-do's for SALSA v2. SALSA v2 works with bones, blendshapes, sprites, materials and textures.

 

Will there be a new box head?

 

Awesome! I'm really looking forward to it. I hope supporting UMA version of CC3 is not much different from vanilla CC3. Cheers!

 

Hi, is there an easy way for one object to mirror blendshapes of another?
I'm thinking of using Broadcasting to target the other object and then a script to update the blendshape status of it to match the original.

 

Hi,

The following property can be used to read the processed blendshape output values for saySmall, sayMedium, and sayLarge.

Salsa3D.sayAmount (SalsaBlendAmounts)

• Salsa3D.sayAmount.saySmall (float)
• Salsa3D.sayAmount.sayMedium (float)
• Salsa3D.sayAmount.sayLarge (float)

A custom type that contains the say BlendShape amounts. It's updated in realtime and provides direct access to the values even when no SkinnedMeshRenderer is linked. This is useful for asset integrators that want to use Salsa3D as a lip sync engine with their own integration.

 

The long drawn out process of getting Salsa 2 live......

 

Thanks, but in my case I'm trying to capture Custom Shapes

 

In SALSA v2, Custom Shapes will be handled in EmoteR. EmoteR has the same sort of flexible configuration as SALSA with its emote expressions. You could configure any number of SkinnedMeshRenderers on an emote and effectively fire the shapes off from one instance. This would work pretty easily in small configurations, but may become cumbersome depending on the number of shape components used. Additionally, it does increase the queue length, which means more processing during shape registrations. So, all of that said, it is possible to implement in v2, but may not be practical, it just depends on your situation.

 

My characters have A LOT of blendshapes and I would like to make some custom combined ones and delete all the rest.
MorphMixer seems perfect for the task, however:
- Does it support Daz3D Genesis 1 characters?
- MorphMixer supports making presets for combining blendshapes, which is AWESOME, but does it support DELETING blendshapes once the combined ones are done? (this would be really cool in case I would need to reimport the character from scratch in Unity because of a change in design)

I know it's off topic, but since this is related to Salsa, I thought no one would mind

 

MorphMixer works by creating an asset copy of your original mesh, and then remapping the copy to your Renderer automatically. It does not modify the original mesh. It should work on any character that imports into Unity with a SkinnedMeshRenderer and has blendshapes. On the asset copy you can create new blendshapes by combining one or more existing blendshapes of varying amounts, or delete any blendshape. The only issue we're aware of is on CC3 characters, the size of the created asset mesh file is about twice the size of the original, we aren't sure yet what causes this since it doesn't happen on other characters, possibly something within the CC3 system.

FYI MorphMixer and Amplitude also have their own forums found at the links below.

• MorphMixer
  • https://forum.unity.com/threads/morphmixer-create-new-blendshape-mixes-in-unity3d.504283/
• Amplitude
  • https://forum.unity.com/threads/amplitude-for-webgl.507835/

 

Where can I get the update purchase? I just saw 2.0 in the store…

Thanks

 

Hello Everyone!!!

Looks like SALSA LipSync v2 Suite is now live...WOW that was fast!

To get the upgrade, you'll need to be logged in with the account you purchase SALSA v1 with and you should see update pricing. We're seeing quite a few $4 purchases, so this appears to be working.

We thought we were going to have 2 or 3 weeks to get everything lined up here and that turned into 3 days, so please be patient and bear with us. Some things of note:

1. v1.x addons do not work with v2! I'll be getting the downloads site operational and we'll get the packages uploaded that are ready. Please consider everything beta at this point and let us know via assetsupport@crazyminnow.com if you have problems.
2. I will be linking the documentation we have ready on the site shortly. In the meantime, you can get to it directly from this link: https://crazyminnowstudio.com/docs/salsa-lip-sync/
3. We will be updating the site to reflect SALSA LipSync v2 Suite as we move along.
4. Addon development will continue as well.
5. We will also get some video tutorials out there.

We honestly thought we'd have at least a couple of weeks so please be gentle.

Thanks a bunch!
Darrin

 

mmm... now I don't know if I'll buy it.

Just kidding, I just did it.

 

Ditto! I bought it as well! Good things do come to those who wait! @TonyLi Now Tony has to work his magic for his wonderful assets!

 

@SickaGamer - I'll post here when it's done. Should be a few days more.

 

Making some progress:

• The downloads site is live. Currently ONLY the TimelineSalsaCore package is uploaded.
  • We will continue to upload packages and expect to have several available by tomorrow.
  • Most packages will leverage a core EXAMPLES package for demo scene usage. I will try to have this package up shortly (and we may have to iterate this process a couple of times to get it right but the hope is a unified package will keep the core product light and nimble as well as the add-ons).
• We are updating the site as we go, deprecating the old information and adding new links and images reflecting the new version.
  
• For SALSA v1 users that don't wish to upgrade, the legacy downloads and documentation are still available, links at the bottom of the side menu (see the image below). As a reminder, SALSA v1 is now deprecated. It will receive no further updates. You are free to continue using it for as long as you wish, but we recommend upgrading to the new version for the new shiny (currently only $4).

Thanks so much for your patience and support!
Darrin

 

Forget unlimited visemes…



WE GET UNLIMITED HEADS???

 

We aim to please!

 

Crazy Minnow...delivering all of your Hydra animation needs.

 

All SALSA LipSync Suite 2.0 compatible add-ons will have the initial release version of 2.0.0. The following compatible add-ons have been posted to the downloads page.
https://crazyminnowstudio.com/unity-3d/lip-sync-salsa/downloads/

• Playmaker 2.0.0
• Cinema Director 2.0.0
• Adventure Creator 2.0.0
  • SalsaTextSync Adventure Creator Listener 2.0.0 (SalsaTextSync will be uploaded soon).
• NodeCanvas 2.0.0
• SLATE Cinematic Sequencer 2.0.0

We're working very hard to get everything updated as quickly as possible. We hope to have several of the 1-click add-ons available in the next couple of days. Thanks for a successful launch day turn-out, and your patience as work to finish the remaining add-ons, documentation, and website updates.

 

Now counting the days for the UMA Plugin

 

Is unity timeline supported in salsa 2.0?
The Basic timeline support in v1 was a great start, and really useful to time emotes and play audio.
V.

 

Hi Salsa people,
do you support Amazon Polly Speech Marks (JSON format) to avoid doing the viseme/phoneme timing placement manually with an Amazon Polly generated audio file?
For those who don't know what this is: https://aws.amazon.com/blogs/aws/new-amazon-polly-speech-marks/

Thanks!
Phil.

 

Hello! We do not support this at this time. It is currently unclear if/how we would leverage this into our system, but we are always open to new options. And to clarify, SALSA is not a phoneme mapped solution. We analyze audio in real-time and approximate lip-synchronization based on a few different factors.

 

Yes, Timeline is supported and the core add-on is available on the downloads page. Although, I'm going to update it one more time very shortly, so wait for v2.0.2. Updating the Examples pack and will need to update the Timeline pack to the new GUIDs for the example scene. If you don't need the example scene -- go grab it now!

 

We have posted up a few more add-ons for v2:

• SalsaTextSync v2.0.0
  
• TimelineSalsaTextSync v2.0.0
• SalsaRtVoice v2.0.0
• TimelineSalsaCore v2.0.2
• Examples-Pack (has some audio files and 2D assets used in sample scenes for some add-ons).

The blog posts for these have received a very fast update for SALSA LipSync v2, but proper documentation will eventually be completed and moved to our docs repository.

Hopefully coming by early tomorrow will be:

• Dissonance
• micInput
• MasterAudio

These are completed, just need a final test on the new base code and a proper export. And yes, we are hoping to get the OneClicks out as well.

 

Faster!

 

The following add-ons should now be available:

• SalsaMasterAudio
• SalsaMicInput

A quick pass has been completed on the original blog posts for info/instructions. We will be moving documentation to the /docs repository in the near future.

NOTE: SalsaMicInput no longer needs to be configured to use an attenuated mixer group. With the new analysis engine in SALSA LipSync v2, you can mute the microphone at the AudioSource and lip-sync will get along just fine!

 

OK! This should solve an issue I had with the MicInput… I always got a late echo like a weird sampling loop that came back 20sec later, but pitched slower… I could never solve it. I also route my mic through an audio app to delay it a fraction of a second to match the SALSA delay, but that weird sampling loop was a spoiler…. This new way should let me just mix my live Mic with Unity audio output (externally) and avoid the sampling issue.

 

Oh dang! That totally works!


BUY THIS THING, Y'ALL! MUTE IS WORTH ANY PRICE!

 

Will the old toon box head work with salsa 2? I've grown quite fond of him.

 

Our Dissonance Voice Chat integration add-on, SalsaDissonanceLink is now live in downloads (and docs should be good to go as well). https://crazyminnowstudio.com/docs/salsa-lip-sync/addons/using-with-dissonance/

 

Yep, he should work just fine.

 

Unity's Microphone input is a touchy little beastie. Input data is stored in a circular AudioClip of configurable length and if the AudioSource play-head gets ahead of the Microphone record-head, you get that weird delay of how ever many seconds the clip buffer is set to. I believe the old version of micInput was set to 10 seconds. I can't explain the slower/lower pitch - that's weird. It is also pretty easy for the heads (pointers) to get out of sync. One of the biggest culprits is application focus and pausing -- at least from what I've seen. If the application is paused for whatever reason, it's best to simply restart the micInput recording using StopMicrophone() & StartMicrophone().

Regardless of where playback thinks it is, SALSA LipSync v2's analysis is continually calibrated to the microphone's record position. That doesn't solve the audible playback issues, but it does make the analysis as close to real-time as it can be. So, if you don't need the microphone playback to be audible, it works about as good as it can. For those that need to output to loudspeakers, it may work best to split microphone output to an amp (preferably with a small or adjustable delay to really dial things in) and avoid using the AudioSource output from Unity's microphone input.

Thanks for the feedback wetcircuit!
Darrin

 

Is there an equivalent to the properties for in v1? I want to trigger an event and none of the component types look particularly suited for that.

 

Hello,

Which v1 properties are you referring to? Can you tell us more about what you're trying to do?

 

Our blog posts for the following add-ons have been updated for SALSA LipSync Suite:

• Playmaker
  
• Cinema Director
• Adventure Creator
  
• NodeCanvas
  
• SLATE Cinematic Sequencer

 

Wow, SALSA LipSync Suite is currently listed as the 3rd most popular asset on the store. That's amazing...thanks everyone!

https://assetstore.unity.com/?orderBy=1

 

Um, not really, I can't find the original documentation and don't see what I'm looking for in v1. I had a foggy memory of there being a public property or properties set to the current "state" (saySmall, sayMedium, and so on.) I had planned to use that to drive properties on materials and VFX for animation.

 

The v1 documentation is still available.
https://crazyminnowstudio.com/unity-3d/lip-sync-salsa/manuals/

In v2, you can drive textures and materials directly from the UI by changing the animation controller type.

 

Dialogue System integration for SALSA LipSync Suite is now available. Please see this post for details.

 

Since I upgraded to V2 I got this:
Assembly 'Library/ScriptAssemblies/Assembly-CSharp-Editor-firstpass.dll' will not be loaded due to errors:
Reference has errors 'SALSA-LipSync'.
Before I was OK. Please help.
Thank you,
CJ

 

Hi metafunnel,

It sounds like you might be using the unsupported Unity beta version 2019.2? Is this in a new project? SALSA v2 is not backwards compatible with v1 since it is a complete rewrite and require a new setup on all SALSA enabled character.

 

Congrats on the V2 release, I just bought it. Very excited to get started. I wanted to know if you have a demo video showcasing the new head and eye animations? Curious to see how they look. Great job btw on the new version.

 

Yep. that'll be good for a wholesale change to a from one material to another, but won't let me change their exposed properties as I understand it. Neither is any help with VFX graphs.

--- Edited above to be slightly more clear about what I DO want to change using an expression component or whatever it would take. Also sent mail to support. Thanks.

 

Send us all of your pertinent information (in an email to assetsupport@crazyminnow.com), including your invoice number and a reference to your ID in this thread. It will, of course, be necessary to have sufficient detail about what you want to accomplish. SALSA v2 is very flexible in its underlying design, so we'll see what we can figure out for you.