SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Only 9.5 hours left to get our highly rated SALSA Lip-Sync solution for 44% off. Want to add lip-sync to your game or art project without spending hours, days, or more mapping phoneme shapes? SALSA is completely automated, requires no audio pre-processing, and works with the following systems:

Real-time microphone lip-sync
Fuse Character Creator
Autodesk Character Generator
DAZ3D
Dialogue System for Unity
Adventure Creator
Master Audio Playlists
Playmaker
Custom character models

https://www.assetstore.unity3d.com/en/#!/content/16944

 

I am having an issue when selecting the Auto-Link option for the Custom Shape RandomEyes3D Script

It doesn't seem to Auto Link.

Using the latest version of Fuse 1.3 Unity 5.1.1f1

am I missing something simple?

 

Hello JeffG,

Thanks for your support!

For a fully setup Fuse character, you should have two RandomEyes3D components, one for eye control where you leave the Skinned Mesh Renderer field empty and another for facial animations. To auto-link the facial animation BlendShapes, link the Fuse [Body] child object to the Skinned Mesh Renderer and check the box for [Use Custom Shapes Only].

I demonstrate this at 2:38 seconds in the following video.

Michael

 

and I get nothing. Click the button and nothing happens. The list of custom shapes doesn't get populated. Nothing in the console either.

Now, I am on Windows 10. Not sure why that would make a difference, but I can try it on my laptop (Windows 7) tomorrow morning

 

Can you look at the Skinned Mesh Renderer on your [Body] object, do you see a list of BlendShapes?

We've tested up to Windows 8.1.

 

Stupid User trick. Blend shapes for facial animations are disable by default when doing the auto rig. Thanks for the info though. Gave me a the right place to look.

 

Glad it's working Jeff, let us know if you have additional questions.

 

SALSA with RandomEyes v1.3.2 Submitted to the Unity Asset Store

Read about the details of this minor patch release on our blog.
http://crazyminnowstudio.com/posts/salsa-with-randomeyes-v1.3.2-submitted-to-the-unity-asset-store/

 

Result from our 24:HOUR SALE



WOW, we sold 251 copies of SALSA with RandomEyes, to 29 different countries, in 24 hours!

We are a very small team and put in tremendous effort to provide a great product at a fair price. We want to thank all of our customers for supporting our product, offering great suggestions to help us improve SALSA, and spreading the word about our lip-sync solution. Finally, we’d like to welcome all of our new customers into the fold. If you love SALSA, please spread the word, and rate or review if you haven't already. Your word-of-mouth praise and great reviews gives others the confidence to give our solution a try.

Click below to read our entire blog post about our 24:hr sale experience.
http://crazyminnowstudio.com/posts/results-from-our-first-24-hour-sale-on-the-unity-asset-store/

 

I'm attending the SIGGRAPH 2015 conference in LA on Tuesday, August 11th. I'll be walking the exhibit floor in a white polo with the SALSA logo on the front and the Crazy Minnow Studio logo on the back.

If you see me, say hello or stop for a chat.

Michael

Registration
Exhibitors

 

hi Michael, is it possible to use Salsa with a textfile instead of an audiofile. i would need this very much and in the tutorial videos i only saw audiofiles so far. thank you!

 

I bought EasyVoice asset to create text-to-speech audioclips inside Unity. It works well with SALSA, but does not make TTS voices at runtime "on the fly" - that would be very dependent on the end user's installed voices (there are a few TTS assets but I have not used them)....
https://www.assetstore.unity3d.com/en/#!/content/22328

 

Hello Hormic,

SALSA cannot read text files directly, it uses real-time waveform analysis to drive BlendShape value transitions. Without the data from an audio clip, SALSA has nothing to process.

Depending on what you're trying to do, it might be possible to use a 3rd party text-to-speech software, like the one below or another, to generate the audio files SALSA needs ahead of time. However, this still results it SALSA processing audio clips instead of text files.

Though not tested, maybe something like this? It seems to leverage the Windows Speech API to generate spoken audio clip files using Microsoft's text-to-speech engine. The caveats with an approach similar to this include Windows dependency, and a lack of rapid real-time processing needed in most game scenarios.
https://www.assetstore.unity3d.com/en/#!/content/22328

Michael

 

Jinx! Thanks wetcircuit.

 

LOL

While you are here, I have a quick question. In your Playmaker actions I'm a little confused over Duration and Blend.

They are both time(?) settings but Duration is "x seconds" and Blend is what exactly? Fade In/Out time? A proportional relationship of the overall duration? a "speed" setting?



I feel like I am using a Ouija board to control my face, LOL. (It's not just your actions, I think this is a convention of Unity or maybe Playmaker that I just don't know when we are talking "seconds" or "speed" or what...

 

@wetcircuit

Duration is the total number in seconds, that the expression will be enabled. The Blend fields represent a step increment towards the max Range. In your screenshot the expression will be enabled for 5 seconds, the Smile_Left and Smile_Right shapes will blend towards 100 at increments of 5, and the Squint_Left and Squint_Right shapes will blend towards 100 at increments of 20. You won't see actual stepping because the values are blended using linear interpolation.

So stepping a smile shape from a value of 0-100, at a blend increment of 5, will occur over 20 iteration. A squint from 0-100, at a blend increment of 20, will occur over 5 iteration.

If the Duration were set very short (1 second) and the Blend increment very small (0.001 or whatever), the shape will never reach it's max Range, instead it will move towards it's max range for 1 second, then use the same Blend (step size) to decrement back to 0.

I hope this helps,

Michael

 

Lemme see... If one train leaves Pittsburgh traveling 45mph....


j/k the iteration steps never would have occurred to me....
But now some of my previous tweening issues might need reviewing... lolol

 

And another train leaves New York at 2:30pm...LOL

 

Hmmm, thank you for your suggestions (also @wetcuircuit i will look into EasyVoice), but is it so complicated to integrate an uncomplicated behavior for textfiles?
Not every game in the world needs and wants to use spoken audio files.
So, for instance when a line of speech is visible on screen, create a random motion for the mouth if no audio file is available. So you have mouthmotion during reading a line of text on the screen. I can imagine that this is not soooo hard to implement. With this behavior Salsa would be the perfect solution for every cicumstance.

 

You just need the mouth moving silently? I think I would probably just create a short animation clip keyframing the mouth blendshapes in a few spots and then loop it. There is no reason to use SALSA (or any component) for that. It's overkill if there is no audio... That would be my suggestion. It is actually simpler than setting up SALSA.

Although I would still use RandomEyes for the facial expressions and blinks, etc

 

Yes thank you, that is possible and is of course a good suggestion, but when i already have a software for it with salsa, i would reccomend that it can also do this.
Or at least has the functionality to manually control start and stop mouthmotions and blendings so that i can use it in one flow with Random Eyes in my project, when i don't have or want audio speech for every single line of text.

 

Ahh, I thought you were asking for accurate text lip sync which would be a lot of unnecessary processing. As wetcircuit said, there are definitely other ways to handle this but I can see the advantage of having an option in our API so that if you're already using SALSA for your spoken dialogue workflow you wouldn't need another workflow to handle a simple "jabber mouth" style animation. It's an interesting idea, it's still common for games to have only some of the in-game dialogue with audio, and some text only.

We'll look into it, thanks for the suggestion!

 

Thank you, i'm looking forward to it!

 

Im interested in lip sync using real-time audio from text-to-speech. Is it possible? I noticed that you have microphone support so I'm assuming it is. I bought RT-Voice and I would love to hear your thoughts on getting your asset to work with theirs. If it is, I would definitely invest in SALSA!

 

Hi adamz,

That looks like a very cool solution. We are looking into it to see if it will work with SALSA. We'll post a reply here as soon as we have an answer.

 

Fantastic!

 

I mentioned your asset to the creators of RT-Voice, so perhaps you'll both be able to work together to integrate it!

Are you planning on integrating phoneme-based lipsync matching, to get a more accurate animation to the audio?

 

Hello adamz!

We purchased RT-Voice yesterday to take a look at it and also reached out to crosstales. As it turns out, RT-Voice uses the computer system's TTS engine to create audio (outside of Unity), so at this point, the two won't work together. crosstales is looking into it to see if there's a solution available.

BTW, microphone input works in SALSA because the Microphone system links up with Unity's AudioSource.clip and that's what SALSA attaches to.

At this point, we don't have any plans for phoneme mapping features. There are already some very good solutions available for that sort of workflow. SALSA fits a niche that provides very nice lip-sync with very little effort (no need to create dozens of phoneme shapes, nor manually map or pre-process audio files) and it works in real-time and on all platforms. Of course, we are always looking for ways to keep to SALSA's goals and improve upon its tried and tested technology. If you have not yet seen our latest promo video, please check it out. It uses DAZ3D models and demonstrates the power of SALSA with RandomEyes. Our blog post gives a high-level overview of how we set it up.

If working with RT-Voice and phoneme mapping are your requirements, it certainly looks like SALSA may not be the asset you're looking for. Hope this answers all of your questions.

 

Thanks for looking into this for me. Im trying to come up with a system that will allow me to take dynamic text created at run-time, convert it to audio, which then will allow me to use SALSA. Would something like Easy Voice work? I'm not sure how long it will take to create a audio file at run-time, but that could be another solution I could use...

 

Ahhh yes, Easy Voice. We took a peek at this a while back. If I recall, they create audio files in batch. So, I don't believe it's a run-time solution, but it's certainly a quick way to produce some TTS audio. This would certainly work with SALSA since the end result is an audio file.

In script, you'd either attach to your AudioSource and set the clip or do it directly in the SALSA API. [NOTE: if you attach to the AudioSource, in v1.3.2 and earlier of SALSA, you'll need to collapse the SALSA inspector or SALSA will override the AudioSource -- this will be addressed in the next release of SALSA.] Again, it's not a run-time solution, but this option should work. It may be possible for crosstales to add a similar feature to RT-Voice at runtime -- if you ask them nicely.

Your mileage will vary on how good the audio files will look in SALSA based on the voice quality used. We have tested some nice TTS output (Acapela Box) with SALSA in the early days and it looked good.

 

Our newest promo video is out, featuring DAZ3D characters EmotiGuy and Dragon. This video demonstrates real-time lip-synchronization using SALSA technology in a cinematic scene. It also fully leverages the RandomEyes asset (included with SALSA) for eye-movement, blink, and custom expression triggers. All lip-sync is completely controlled by SALSA, in real-time, simply by swapping audio clips for each model in a dialogue style. As mentioned, we used DAZ3D models in the promo along with our DAZ3D workflow script. The whole process is super easy to setup and achieves great results.

Our blog post provides a little more detail on how we put it together. We're discussing whether or not we want to put out a more detailed "how-we-made-it" video/article.

For those that are interested, we used Simple LOD by Orbcreation to create the flying dragon LODs. Simple LOD is pretty awesome! We also used Camera Path Animator by Jasper Stocker to a great extent, controlling the flight paths of the dragons and all of the camera movements/settings.

 

Experiment: SALSA real-time lip-sync with RT-Voice runtime text-to-speech

This is a quick experiment to test SALSA real-time lipsync with the RT-Voice runtime text-to-speech system. This test was performed using pre-release beta versions for each system.

SALSA version 1.3.3 (beta)
RT-Voice 1.0.10 (beta)

 

SALSA v1.3.3 has been submitted to the Asset Store!

We're happy to announce our 8th SALSA release. This is a minor release containing a few new features to enhance usability, and provide greater flexibility across implementations. If you want to read the highlights or check out the full release notes, follow the link below.

http://crazyminnowstudio.com/posts/salsa-with-randomeyes-1.3.3/

 

SALSA and RT-Voice...a winning pair!

It's official, with the release of RT-Voice v1.1.0 from Crosstails and the recently submitted SALSA v1.3.3, Unity developers can now have lip-synced text-to-speech at run-time with no pre-processing. Follow the link below for all the details.

** This workflow will work using SALSA 1.3.2, but you will have to collapse the SALSA inspector. This caveat is removed in v1.3.3, which has been submitted to the Asset Store for approval.

http://crazyminnowstudio.com/posts/salsa-and-rt-voice-for-runtime-lipsynced-text-to-speech/

 

SALSA with RandomEyes v1.3.3 is now live!

If you're using SALSA with other assets (like RT-Voice or Adventure Creator) we recommend updating to v1.3.3 for best compatibility. This update relaxes the coupling of SALSA with the AudioSource component.

Full release notes.

 

Dialogue System users: Some of the example assets have new metadata in SALSA v1.3.3. The Dialogue System's SALSA Support example scene needed to be updated to correctly reference the example assets. If you're using SALSA 1.3.3+ and want to play the example scene, you can download an updated support package here:

SALSA_Support_2015-08-19.unitypackage

The next release of the Dialogue System will also include a SALSA + RT-Voice integration package!

 

We just finished the 72 hour Ludum Dare game jam. It was exhausting, but also loads of fun. Click the link below if you want to check out our contest entry game.

http://crazyminnowstudio.com/posts/run-ogre-run/

 

Ahh poor Ogre! hahaha

Works great on Firefox
(not so much on Safari)

 

The WebGL version worked just fine in Chrome on Windows. What a fun little endless runner! Great job!

 

Thanks, we're seeing issues on Safari as well and trying to figure it out. We also just posted a Mac desktop version that is working well for us. If you're interested in trying it, we'd love to hear if it's working on more than just our one test Mac.

(sorry, the previous file link had changed - this link is to the page with the files)
http://crazyminnowstudio.com/posts/run-ogre-run/

 

Thanks! We're pretty beat after the 72 hour marathon, but it was fun, and nice to give our full tool chain a workout.

 

link gave me a 404 error...

 

Sorry, we re-uploaded the file and the name changed. I've put a link to the page with the downloads instead to make sure that doesn't happen any more.

http://crazyminnowstudio.com/posts/run-ogre-run/

 

Hiya

I purchased this asset last week ( it's really neat!) and am trying to set it up with Autodesk Character Generator. I downloaded the scripts and it all works neat except for one issue. The character has different blendshapes for the right and left part of the face, e.g. a smile is divided in 2 files, right and left. As a result instead of a complete smile my character does a smirk with only one side of her face. Is there a way to set multiple expressions as 1 custom shape?

Thanks!

 

Hello SheridaH and thank you for your purchase. There are a couple of options for this scenario. Please send your invoice number to assetsupport@crazyminnow.com and we'll see if we can find a good solution for you. Thanks!

 

The Dialogue System for Unity version 1.5.5 is now available on the Asset Store with an integration package and example scene that uses SALSA and RT-Voice. SALSA made it really easy to set up: Just drop an RT Voice Actor component on your SALSA-configured speakers, and they'll automatically speak Dialogue System conversation lines and barks with full text-to-speech lipsync!

 

Unity 5.2 AudioSource.GetOutputData Bug Work-Around:
There is currently a bug in the AudioSource.GetOutputData component in Unity 5.2 that fails to produce data output when the AudioSource is muted. This becomes problematic when using something like SALSA's addon product micInput that relies on a muted AudioSource when collecting microphone input data to prevent echo from your voice. Luckily, there is a work-around until Unity can change the operation back to what it previously was. By the way, the documented bug can be found here. This does not appear to be intentional behavior (on Unity's behalf), so we are assuming it will be fixed at some point.

This does not appear to be a microphone hardware or operating system issue and is strictly limited to Unity 5.2. MicInput and SALSA operate correctly on Unity 5.1 and lower. Read more in our blog post detailing how to work around this Unity bug.

 

[CORRECTION] Unity 5.2 AudioSource.GetOutputData behavior change (bug)

It appears as though this is the intended behavior that Unity has implemented. The 5.2 release notes do not mention this change, but the issue-tracker is pretty clear that they are playing catch-up on correcting something they feel is illogical in implementation.

For micInput (and any other operation where you need to process audio and not hear the audio itself), use the Mixer Group method to pipe audio into a group with -80dB attenuation.

 

SALSA with RandomEyes now supports MCS models from Morph 3D.

If you haven't heard, Morph 3D has released their new Morph Character System (MCS). This awesome system starts with a male or female base mesh, and includes 350 individually configurable and combinable morphs allowing you to customize your characters to just about any configuration you could want. You can read more about it using the link below.

https://www.morph3d.com/mcs-morph-character-system.php

Since MCS models are based on DAZ3D Genesis 2 models, we added MCS support to the DAZ3D 1-Click workflow in our latest 3.0 add-on release.

**SALSA with RandomEyes will not work with the Lite versions of the MCS models because the Lite version does not include any of the mouth or blink shapes required.

Check the link below for more information.

http://crazyminnowstudio.com/posts/using-daz3d-characters-with-salsa-with-randomeyes/

 

We posted new minor releases for the following character system add-on's:

• Autodesk Character Generator
• Fuse Character Creator
• Daz3D
  
• MCS by Morph 3D