SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Hello, I believe this was solved in an email chain a little earlier. But for reference in the thread, the problem was caused by duplicating characters and the component references remained linked to the original. The solution is to link the correct AudioSources to the correct instances of SALSA on the appropriate characters/models.

 

From this previous post I had assumed you would be releasing the 2.0 update soon, but are you waiting for the 2D rewrite to be done as well?

 

Hello magique,

That is correct. The current version of SALSA supports 2D & 3D characters, and we plan to continue to offer the same for SALSA 2.0. Since Unity has acquired some useful 2D assets, we're working to make sure our SALSA 2.0 2D implementation works well with current tool availability and best practices.

Michael

 

OK, thanks for the information.

 

We're excited to announce Amplitude for WebGL, our upcoming new asset (Submitted to the Unity Asset Store).

Amplitude is a Unity3D asset that provides beat detection and amplitude data on the WebGL platform. Unity uses the FMOD audio framework on all of their distribution platforms except WebGL, where they use a wrapper for the browser-based Web Audio API. Unfortunately, this means many of the more sophisticated tools for audio analysis simply are not available on the WebGL platform. Amplitude uses a native JavaScript library to communicate directly with the underlying web browser to access Web Audio API capabilities that have not been exposed by the Unity API wrapper.

Amplitude is easy to use, simply add the component, link your Unity AudioSource to the Amplitude AudioSource field, set the Sample Size as a multiple of 2 (16, 32, 64, etc.), an amount of boost if desired, and whether or not you want your output to use absolute values. Play your audio using the normal Unity AudioSource API by accessing the AudioSource directly or through Amplitude's AudioSource reference. While your audio is playing, Amplitude exposes a float[] array of the size you specified, and a float average amplitude. The values range from -1 to 1, or 0 to 1 with Absolute Values enabled.

It comes with a clean and simple custom inspector, and of course we created a SALSA lip-sync add-on that allows SALSA to leverage Amplitude for WebGL-based character lip-sync. The SALSA add-on (AmplitudeSALSA) will be available for download from our downloads page, free for SALSA customers, once the Amplitude asset goes live on the Unity Asset Store. You must have SALSA, Amplitude, and the free AmplitudeSALSA add-on to use SALSA on the WebGL platform.

Amplitude Inspector (New Asset)


AmplitudeSALSA (Free Add-on)


Included demo scene


We don't want to clutter this thread, so if you're interested or have questions about Amplitude for WebGL, head over to the Amplitude forum, or check our website for more information.

Amplitude Unity forum
https://forum.unity.com/threads/amplitude-for-webgl.507835/

Amplitude website
http://crazyminnowstudio.com/unity-3d/amplitude-webgl/

AmplitudeSALSA (add-on to link SALSA & Amplitude together on the WebGL platform)
http://crazyminnowstudio.com/posts/salsa-lipsync-in-webgl-with-amplitude/

The Crazy Minnow Team

 

With the pending release of our two new assets, we will be temporarily offering cross product upgrades for Amplitude and MorphMixer to existing SALSA customers. If you own SALSA, you will be able to purchase Amplitude and/or MorphMixer for 25% off the regular price. Both assets have been submitted to the Asset Store and we're currently waiting on approval.

Amplitude for WebGL
Regular price: $25.00
SALSA cross product upgrade discount: -25%
Upgrade price: $18.75

MorphMixer
Regular price: $20.00
SALSA cross product upgrade discount: -25%
Upgrade price: $14.00

The Crazy Minnow Team

 

Salsa under WebGL? Sounds amazing!

 

Hi guys, wondering if is any option to let you save the animation, im not looking use it in real time.
Just play ones and save the animations for use it without all the Salsa system added to my characters.

 

Hello, there is no option built into the product, but you might find some success with this post: https://forum.unity.com/threads/sal...expression-system.242135/page-14#post-3204395

It should also be completely possible to roll-your-own and capture the index output of SALSA or the blendshape values of your model over time.

 

Hi - Will be "1 click" module for genesis 8 (Daz 3d)?

 

Hello denebhyde,

We have just completed our first bit of testing with Genesis 8 characters and they seem to work well, so we should be able to include them in the 1-click setup. Until that update is available, you can also use our SalsaSync add-on to manually map SALSA to Genesis 8. It's not as simple as our 1-click setups, but it is more flexible.

https://crazyminnowstudio.com/posts/salsasync-flexible-salsa-map-for-custom-characters/

Michael

 

Yeah - Thanks

 

waiting the v2.0

 

Being able to programatically set a blend speed for an entire shape-group would be awesome. I want to trigger a group relatively fast, but then let the expression slowly die out.

I made a workaround by triggering the group, setting the blends of all the shapes in the group and then triggering the group again at the end of the previous one, but it's still kind of clunky.

 

Hi JonassAA,

That's a great idea, we'll look into it. Thanks for the suggestion.

Michael

 

I am working on a game in 2D, and had a question about Salsa. Is it possible to use animations instead of sprites for phonemes in Salsa?

 

Hello onefiftyone,

This is possible using our exposed [say] or [sayIndex] properties. You can leave the sprite fields empty, then use the [say] or [sayIndex] properties to determine which animation should be active in an external script. When one is active, the rest should be inactive.

See the code example section of our documentation for more information on access a salsa instance and it's properties and methods.
http://crazyminnowstudio.com/unity-3d/lip-sync-salsa/manuals/salsa-manual/code-examples/

See our documentation for more information about Classes, Methods & Properties.
http://crazyminnowstudio.com/unity-...uals/salsa-manual/classes-methods-properties/

Public properties

Salsa2D.say (string)

String value that is always updated with the current shape (Rest, Small, Medium, Large). This is useful for asset integrators that want to use Salsa2D as a lip sync engine with their own integration.

Salsa2D.sayIndex (int)

Int value that is always updated with the current shape (0=Rest, 1=Small, 2=Medium, 3=Large). This is useful for asset integrators that want to use Salsa2D as a lip sync engine with their own integration.

Michael

 

I have run into a small problem. I had something similar before but I can't seem to remember how to fix it.

If I have an NPC in the scene, as in the attached pictures I have the playermate and playerboychild in the scene, and the look target works fine.

But if I want to spawn in the characters instead of having them stay in the scene, then the look target doesn't stay, the field goes empty when I hit the play button.

I remember having to do something with global variables before, but I can remember what it was for to check, and I can't remember how to do it.

Any advice?

 

Hil helgarddebarros,

You can use the RandomEyes.SetLookTarget API method to set the look target.

https://crazyminnowstudio.com/unity...randomeyes-manual/classes-methods-properties/

RandomEyes3D.SetLookTarget(GameObject target)

Something like...

Code (CSharp):

1.  
2. GameObject target;
3. GameObject prefab;
4. Transform npc = Instantiate(prefab);
5. if (npc)
6. {
7.     npc.gameObject.GetComponent<RandomEyes3D>().SetLookTarget(target);
8. }

Michael

 

Hi,
I am having a new issue using the Random Eyes 3D component on a MakeHuman character. I am following the instruction on this tutorial : http://crazyminnowstudio.com/posts/using-makehuman-characters-with-salsa-lipsync/
and I am now stuck on adding the second RandomEyes3d (for facial expressions ) component, Iink the character body , check the "Use Custom Shapes Only box BUT when I click "Auto-Link", nothing happens.

I am saying it is a" new " issue, because I did the same procedures about 6 months ago and everything worked fine and I know I am supposed to have a list under Shapes when clicking Auto-Link. I have just downloaded all the possible updates of Salsa and SalsaSync and I am using Unity5.6.3f1 .


I am also seeing some weird errors in my console each time I click on the RandomEyes 3D Component :

ArgumentException: Getting control 0's position in a group with only 0 controls when doing Repaint
Aborting

Unexpected top level layout group! Missing GUILayout.EndScrollView/EndVertical/EndHorizontal?
UnityEditor.DockArea:OnGUI()

IndexOutOfRangeException: Array index is out of range.
CrazyMinnow.SALSA.RandomEyes3DInspector.OnInspectorGUI ()
UnityEditor.InspectorWindow.DrawEditor (UnityEditor.Editor editor, Int32 editorIndex, Boolean rebuildOptimizedGUIBlock, System.Boolean& showImportedObjectBarNext, UnityEngine.Rect& importedObjectBarRect) (at C:/buildslave/unity/build/Editor/Mono/Inspector/InspectorWindow.cs:1240)
UnityEditor.DockArea:OnGUI()

 

Hello quiome66,

It sounds like the exported MakeHuman character doesn't have any BlendShapes. Please check the characters body SkinnedMeshRenderer component for a section called BlendShapes. If it's there, it can be unfolded to expose all the BlendShapes, otherwise the BlendShape section will be missing.

It sound like you may have missed step 2 in the following section.

Import MakeHuman export into Blender for FBX file conversion
...
2. Use the following MHX2 import options:

• Override Exported Data: Checked
• Import Human Type: Base, Offset, Face Shapes
• Masking: Apply, Conservation Masks
• Rigging: Add Rig, Rig Type - Exported, Custom Shapes

Also, since the MakeHuman workflow relies on many different software packages, it's also important to test in the versions documented before trying to move to another version to make sure it's not a version issue (Unity v2017.1.0f3 vs Unity v5.6.3f1). Unity v5.6.3f1 should be fine, but it hasn't been tested.

• SALSA with RandomEyes v1.5.1
• SalsaSync v2.4.0
• MakeHuman v1.1.1
• Thomas@MakeHuman's MHX2 file exchange format v0.27
• Unity Rig export configuration for MakeHuman (05/12/2016)
• Blender v2.7.9
• Unity v2017.1.0f3

Thanks,
Michael

 

Hi does Salsa Eyes still support UMA 2?

 

Hello markashburner,

We have moved our support add-on to UMA 2 Dynamic Character System (DCS) since it's the current version, and we believe it's easier to get started with.

https://crazyminnowstudio.com/posts/salsa-add-on-for-uma-dcs/

Michael

 

How's SALSA 2 coming along?

 

Hi adamz,

We wanted it to be ready before the end of 2017, but unfortunately that didn't happen. However, it is progressing well. We're testing out a refinement to the new SalsaQueue (centralized animation processor) that, if it works, will provide precise timing control for all animations. This will make timeline-based animation editing much easier to manage. We also expanded and unified our control hooks that now allow you to hook into BlendShapes or bones on a per-shape basis. This greatly expands flexibility and actually reduced code by unifying the approach everywhere. The end result should be that you can use SALSA for most any character setup without the need for add-on's. We will likely still offer downloads, but they will be configuration scripts that only execute once to setup the system for common usage scenarios...to maintain our 1-click simplicity.

We recently released two new assets that slightly pushed back SALSA development. However, both SALSA 1.x and 2.x can benefit from those assets so we wanted to get them published. Now we're back on SALSA 2.0 100%.

Michael

 

Hi,


Any update about Genesis 8 in the 1-click setups?



Regards.-

 

Hi Onsterion,

Not yet, but I might have some time this weekend to try to get this ready.

Michael

 

Our Daz 1-click setup has been updated to v3.7.0. It now supports Genesis, Genesis 2, Genesis 3, Genesis 8, Dragon, & Emotiguy.

https://crazyminnowstudio.com/posts/using-daz3d-characters-with-salsa-with-randomeyes/

 

Our MCS 1-click setup has been updated to v2.0.0. It includes BlendShape sync for body morphs. There is potentially a lot going on here depending on model accessories, so we added an accuracy selector (Efficient, Accurate).

https://crazyminnowstudio.com/posts/using-mcs-characters-with-salsa-lipsync/

 

Nice!, Thanks!





Regards.-

 

We just posted SALSA 1.5.2. FYI, this update only removes the UnityScript (js) files and example scenes, so no need to update. Unity is depreciating UnityScript and we are required to remove them from the asset.

 

Hi, I'm using the 1-click setup for Fuse characters and I can't locate the eye shape indices (LookUp, LookDown, LookLeft, LookRight, Blink) to reference in the RandomEyes3D plugin. They don't appear in the list of blendshapes, so the plugin just auto links with the first shape it sees (BlinkLeft) which leads to odd faces, and also disables the ability to use LookTarget. All the other facial blendshapes are coming in, but just not the eye shapes, so I'm wondering if Fuse changed how they export their shapes and the eye movement stuff hasn't been updated in the plugin, or if I'm just missing something either in my character import or the plugin setup.

Thanks for any suggestions you might have.

 

Hello, if you are a MorphMixer or Amplitude customer, Unity just deprecated our packages without warning. We have been trying to work with them to clean up the issue with the SALSA reviews appearing under the MorphMixer and Amplitude assets and they executed our proposal without communicating the feasibility with us and letting us contact current and potential customers.

Rest assured, if you already purchased MorphMixer or Amplitude, you did not purchase a dead-end product and will have a product page and asset to work with. We are trying to contact someone at Unity to find out what is going on and when/how it will be fixed. Please stand by, we appreciate your patience.

Ultimately what we believe will happen (as far as what we proposed to them [Unity]):

• The assets will be removed (which is what already happened).
• They will be recreated as stand-alone assets without the cross-product upgrade (Unity is unable to support this mechanism).
• Existing customers will be transferred to the new products (and notified).
• The cross-product promotional pricing will no longer be available.

They completely bypassed the "communicate with us" bullet point we proposed -- so we cannot confirm how the rest of this is going to play out. We apologize for this situation and humbly ask for and appreciate your patience.

 

Going forward, we will be posting information concerning the Amplitude and MorphMixer assets on their respective threads -- unless something specifically effects SALSA itself. If you have not already subscribed to those threads, please do so if you want to be updated.

 

Hello,
It's a good news that we can now use Salsa on the WebGL! Since I missed this moment,, I'll wait until it appears on the Asset Store again.

But now I write to report some issues.
I'm using MCS with Salsa and found:
1. OpenMax/CloseMax values don't make effect until I change BlinkSpeed value manually at run time.
2. Eyes track LookTarget by strange way. Imagine that the character twists his head and torso (a little) during idle animation. But the eyes do not turn with respect to the head, in order to follow the target. If to describe formally, then the eye's Z axis is not aimed at the target each tick.

And one more note. Salsa system's components intercept all eye movement control, even if RandomEyes3D.RandomEyes property is unchecked. The only way to manage this movement is to set LookTarget. I use Opsive's Third Person Controller for player character and this TPC can control look direction properly depending of mouse movement. I want to exclude only random eyes movement, and remain all other abilities of Salsa.

Versions:
MCS 1.6.4
Salsa 1.5.1
Unity 2017.3.0p4

Thanks
Alex

 

Hello! Thank you for helping developers while we wait for SALSA 2.

 

Hello Alex-3D,

Sorry for the delay, we are looking into the issues you mentioned and will report back.

 

Hello Alex-3D,

1. OpenMax/CloseMax had a bug that was preventing the clamped values from being written to the [RandomEyes3D.lookAmount.blink] property. This fix was added to the SALSA 1.5.3 release, and submitted to the asset store.
2. We have tested LookTarget while the head, neck, etc. are animating, and could not reproduce the symptom you described. During our tests, the eyes continue to track correctly. If you can describe in detail how we can reproduce the symptom, we're happy to test again. Maybe your target is behind the character, make sure the [Track Behind] option is checked if you want the targets tracked even when they are behind the character.
3. If you want another system to control eyes, you have a couple of option depending on the type of character you're using and what RandomEyes3D features you are taking advantage of. Many of our 1-click setup's use two instances of RandomEyes3D, one for eye control and the other for expressions. You can identify each instance by the check box option at the top of the inspector [Use Custom Shapes Only]. If this is checked, the instance will not apply any values to eye control. If you are not using expressions, you can remove all instances of RandomEyes3D and only leave Salsa3D in place.

 

SALSA 1.5.3 is live on the Asset Store. This update should resolve the issue with RandomEyes3D open/close max values not being applied correctly to the RandomEyes3D.lookAmount.blink property.

 

Hello,
1. It works.
2. I sent you a project proving that tracking is not carried out as it should.
3. I am using Salsa for a long time and know this. But it would be very nice if it was possible in Salsa to switch on only blinking, without eye movement.
Thanks
Alex

 

1. Great!
2. I replied in email, thanks.
3. This is a good idea, we'll look into implementing this, thanks!

 

Ooh! If you are looking into eye movement, can we also get a little jitter in the eyes while maintaining a point of focus? Like a saccades control where we can adjust speed and variance…? For instance, if I set the point of focus to a tabletop with low frequency (0.5 - 1.5 sec) and a wide variance for the saccade (a few degrees) the character appears to look at various objects on the tabletop in turn.... A fast frequency with very small variance would work for thinking or worry or just to add intensity to expressions….

What I do now is make the eyeballs point at a "bouncing ball" that is parented to a game object, which I send to places I need them to look. (I also use another asset that specializes in just this random look-at and saccade eye movements when I don't really need to control it).

 

Hi , i buy your asset and i want to make 2d lipsync (with images) on 3d model

like this :

how can achieve this ?

 

you have 2 options:

Use Salsa2D and swap the sprites (looks like the mouth in your video), or create a model with blendshapes and use Salsa3D:

 

i make it work in 2D , but i dont want update a sprite i want update a texture or use texture atlas

I notice that Salsa2D has a "say" and "sayIndex" property.
i can use this for update my texture , in every update.
but i d prefere an event who ll be sent on every change of sayindex.

Does it exist ?

What is the best way ?

thanks.

 

Saccade movement around a focal point during tracking is a great idea, we'll definitely look into that. Thanks!

 

If you want to swap textures, you can reference the Salsa2D/Salsa3D.sayIndex in update, and only update when the AudioSource.isPlaying property is true.

 

Thanks i made it work like this ,
is possible to disable sprite update of salsa and just keep the minimum (i just need sayindex)

thanks

 

Salsa2D requires a SpriteRenderer reference, but if you are manually linking using the sayIndex values, you can just ignore that SpriteRenderer or just use Salsa3D and leave the SkinnedMeshRenderer null. Since you aren't using the inspector for linking, Salsa2D and Salsa3D are functionally equivalent. I'm not sure what you mean by "keep the minimum".

 

Fuse, eyelashes not blinking with 'body'. How I can fix it?