SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

I installed the OnceClickDazEyes and OneClickBase and now I have tons of errors :

Assets\Plugins\Crazy Minnow Studio\SALSA LipSync\Plugins\OneClickRuntimes\OneClickDAZEyes.cs(107,13): error CS1061: 'Eyes' does not contain a definition for 'blinklids' and no accessible extension method 'blinklids' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)"

 

Hello,
You are using the wrong version for your version of SALSA. We are in the process of submitting a new version of SALSA which requires some changes to the API and subsequently the OneClicks..

Until the new version of SALSA is ready, you will need to use v2.1.3 of the DAZ OneClick marked (Legacy).

 

Hello everyone! SALSA LipSync Suite v2.3.0 has been submitted. This version has a huge upgrade to the QueueProcessor and changes things up a lot. There is now a hierarchical set of queues that allows us to do some things we were not able to previously. The shape controller was also upgraded to take advantage of these separate queues allowing the hierarchies to merge back to lower-level queues. For example, a character can be set with droopy eyelids using a "blink" shape in EmoteR and blinks fired from the Eyes module (using the same shapes) will operate correctly and not cancel the 'droopiness', smoothly merging back to the configured EmoteR emote.

There are also quite a few fixes to the shape and bone controller logic, which should smooth out animation transitions a lot, especially when using Advanced Dynamics. This really increases the quality of emphasizer emotes as well.

Those of you chomping at the bit to over/underdrive your blendshapes will enjoy the ability to do so in this new version. Originally we had planned to introduce a new controller capable of leveraging the unclamped blendshape values available in 2018.3+. However, through the miracles available in today's advanced quantum and rocket sciences, we were able to implement this in the standard shape controller and make it backwards compatible with existing projects. Yea science! OK, maybe it had less to do with dark matter and ultra-tiny particles and was actually due to some out-of-the-box thinking...but nonetheless, over/underdriving is now available in v2.3.0.

Also of note, the Unity Microphone class has been removed from SALSA Core files, thus removing the dependency and unnecessary permission requirement for some mobile platforms when the microphone was not used. This changes the way SALSA works slightly and required an update to the SalsaMicInput addon. A new version (v2.3.0) of micInput has been created to work with the new SALSA Suite v2.3.0.

Likewise, since the queue hierarchy changed, introducing more ExpressionComponent types, the Eyes module required some changes, subsequently requiring changes in the OneClicks. If you use the Eyes API, you will definitely want to take a peek at the documentation. If you use OneClicks, you will need to grab the latest like-versioned OneClick to use with this update (versions 2.3.0). The old OneClick (and micInput) versions will still be available for a short time for those of you who do not want to update to the new SALSA Suite v2.3.0.

As always, it is recommended to read the release notes and back up your project if you are updating to this new release. If you are starting a fresh project, it is definitely recommended to use this latest version.

Thanks and enjoy -- we will post back when the package is live!
D.

 

Sounds exciting!
Will stuff like pre-existing Salsa-Slate actions in a project still work after an upgrade, or will they need to be redone?

 

Only character configs that use the Eyes blink configurations will need to be reconfigured and only the blink/tracking section. If you are specifically leveraging the Eyes configuration API then changes will be necessary. Otherwise, I would think your Slate actions should be fine. As always, backup your existing projects before updating.

 

Hello varan941,
Unsure what you are trying to accomplish here. If you are trying to play configured emotes using keycodes, you will want to view https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/emoter/api/#manually-triggering-an-emote

This of course, requires you have an emote configured in EmoteR.

As far as your lip sync animation triggers, I'm not sure what you are trying to accomplish there either. It's not optimal, to use animations for lipsync and this would be better served with a blendshape setup. At this time, SALSA does not support animator triggers as controllers. You can use animator triggers on EmoteR controllers though.

Thanks,
D.

 

You will need the AmplitudeSalsa add-on, download it from the Amplitude downloads site. Then you should be able to continue following the instructions.

 

Thanks

 

Hello!

SALSA Suite v2.3.0 is now available on the Asset Store. Ensure you read this post and the release notes prior to upgrading your project (after you back it up, of course).

Enjoy!
D.

 

OK, after doing the update to 2.3.0 along with the integrations, I'm getting these:

Assets\Crazy Minnow Studio\SALSA LipSync\Plugins\OneClickRuntimes\OneClickDAZEyes.cs(103,13): error CS1061: 'Eyes' does not contain a definition for 'eyelidTracking' and no accessible extension method 'eyelidTracking' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)


Assets\AdventureCreator\Scripts\Actions\Eyes_Blink.cs(58,43): error CS1061: 'Eyes' does not contain a definition for 'eyelids' and no accessible extension method 'eyelids' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)

 

Hi Gregorik,
Sorry about that, I've uploaded DAZ OneClick v2.3.1 (now available) and should fix that reference error.

Will take a look at the AC Action as well. Thanks for the report. Will post back on that one as soon as possible.

Thanks,
D.

 

Another one appeared:


Assets\Crazy Minnow Studio\Addons\SLATE Cinematic Sequencer\ActionClips\Editor\Eyes_BlinkEditor.cs(76,58): error CS1061: 'Eyes' does not contain a definition for 'eyelids' and no accessible extension method 'eyelids' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)

 

Hi Gregorik,

We'll get the visual editor add-ons updated as soon as possible. Thanks for your feedback and sorry about the delay.

Michael

 

So the Blink/Eyelids part is shot with every single add-on (I'm just using 3 of the add-ons), but you already know that. Thanks for looking into it.

 

Thanks Gregorik!
While fixing the visual coding add-on, Mike discovered a bug in some interface code. Anyone using visual code add-ons, please wait to upgrade. A point release will be created to fix the issue.

Thanks,
D.

 

Any luck in progress? We're waiting to re-implement the 2.3 add-ons to our project.

 

Howdy, I installed the Behavior Designer plugin and i got the same eyelid error Gregorik got.

Assets\Crazy Minnow Studio\Addons\Behavior Designer\Actions\Eyes_Blink.cs(45,58): error CS1061: 'Eyes' does not contain a definition for 'eyelids' and no accessible extension method 'eyelids' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)

Just letting you know that one needs attention as well. Thanks for the hard work!

 

Yes, we are in the process of getting a new package submitted (v2.3.1). Discovered a few small issues and believe we have them corrected. There will be new packages for the visual-code/design add-ons as well and they will all be like-versioned with v2.3.1. We are going to get the new version 2.3.1 published and then update the visual-code/design add-ons in the downloads -- that way we don't get new customers downloading the wrong versions by mistake..

A point of clarification for other existing and potential customers, SALSA Suite v2.3.0 is not broken in and of itself. These issues only apply to the visual-code/design add-ons. If you are not using our add-ons for: Adv Creator, Playmaker, Behavior-Designer, Slate, etc. there are no known issues and SALSA Suite should function as expected.

This was a pretty big update, sorry about the errors and confusion.
D.

 

Howdy again, I def appreciate the quick respond and I can wait on the behavior designer stuff no prob. Though I do have a new issue this time. I got SALSA largely to mix with UMA frameworks, and I've ran into a pretty big issue right out of the gate. When i follow the instructions and hit play, i get a host of Index errors and if i push past them i get... this.

It's still lipsyncing, and the audio is playhing, so it works, but something's very wrong here and I can't find anything about it. The video on the youtubepage seems a bit out of date as well as the formatting didn't really look the same. Any idea on what's causing this, or does this seem like something that's gonna need to be patched down the line?

If you need to me uploade more shots or upload my scene or something that'd be no prob as well.


EDIT: Okay it stopped doing it?? I dunno what i changed but it cut it out. I think it has soemthing to do with the UMAs sometimes losing the head tracking via maybe animator changes or what, it's a bit inconsistent but i think most of it's on my end?

 

Hello!
We have the add-ons updated, complete, and ready to publish; however, the updated SALSA Suite package (v2.3.1) didn't get approved on the store yet. Once that's released, we will make the add-ons available. Had hoped it would have gone through yesterday, but likely not till Monday now (when Unity gets back to work).

As for UMA it is really hard to tell what caused it (glad it worked itself out for you). UMA is such a beast and everyone seems to do things differently with it -- which is part of its appeal, but also what makes it so difficult to help with and troubleshoot. If it happens again and you can't figure the cause, email us with screenshots of your SALSA Suite component configuration (completely expanded out) and we'll see if we can help out. Originally, I would say there was some axis nonsense going on, but that doesn't ordinarily fix itself.

Good luck on your project!
D.

 

How to Create Custom OneClicks (it's easy):

Hey everyone! We have had some customers dig in and create their own OneClick mods to apply their preferred settings to their models. This is fantastic and is currently the recommended method to apply your desired settings to your model. While this method is a bit involved and not necessarily for the feint of heart, it is relatively simple once you get under the hood and look at the operation of the OneClick add-ons provided by The Minnow.

If all you want to do is change some of the settings applied to the visemes and emotes, it's pretty easy to just copy the files to your own definitions and make the appropriate adjustments. If you want to go any further than that, there is some nuance in the underlying structures.

We have a LOT of documentation available that discusses how the Suite works on the surface and under the hood, but it may not be readily apparent how to take advantage of all of this in a OneClick with your own code. Our docs are constantly evolving and trying to provide you with more information to assist you with leveraging the power of the SALSA Suite. To this end, I have just added a new section in the OneClick Add-On documentation for creating custom OneClicks that will hopefully assist you with the task and maybe give you some new ideas on things you can do with custom configurations.

Check it out here: https://crazyminnowstudio.com/docs/salsa-lip-sync/addons/one-clicks/#creating-a-custom-oneclick

Enjoy!
D.

 

I had to recreate my project and now had to install the new Salsa 2.3.0, I went to the site and downloaded the one click base 2.1.6, then now installed the Oneclick DAZ 2.3.1. I now get errors

Assets\Plugins\Crazy Minnow Studio\SALSA LipSync\Plugins\OneClickRuntimes\OneClickDAZEyes.cs(103,13): error CS1061: 'Eyes' does not contain a definition for 'eyelidTracking' and no accessible extension method 'eyelidTracking' accepting a first argument of type 'Eyes' could be found (are you missing a using directive or an assembly reference?)


What is the correct install now for this? I dont see a new OneClickBase. I tried installing the OneClickDaz 2.1.3 but that is even more broken... Please help

 

Hi Ctttdaddy,

EDIT: The OneClickDaz_2.3.1 works with SALSA v2.3.0+. Looks like you might have had some initial issues when you loaded the OneClicks or maybe moved some files. The forum notification we got for the original post included some error information about missing files. Try removing the DAZ OneClick files you have -- the Editor script and the two run-time scripts and then reload the DAZ OneClick v2.3.1. The "eyelidTracking" error above indicates v2.3.0 of DAZ OneClick. If you still have that OneClick in your downloads, delete it.

I've just created a new Unity 2019.2 project with zero errors:

• SALSA v2.3.0
• OneClickBase 2.1.6
• OneClickDAZ 2.3.1

As mentioned in the downloads information/requirements, the OneClickBase requirement has not changed from previous OneClick versions. v2.1.6 is the correct one to use. DAZ OneClick v2.1.3 is not broken, it is simply not to be used with SALSA Core 2.3.0+. Ensure the file hierarchy is not changed, otherwise the Editor script may not be able to see the runtimes.

Thanks,
D

 

Why not 2019.3 though? That should be most folks' choice for a new project now. Hopefully it's also flawless on 2019.3.

 

Yes works just fine on 2019.3. I noted on the store page that 2019.3 was tested when 2.3.0 was submitted. Using 2019.2 was just an arbitrary decision based on nothing really. The version used wasn't really important, I was merely stating that there were no known issues with the packages ctttdaddy was trying to use together.

Hope that clarifies,
D.

 

Hi - this looks to be a great asset! I am having trouble integrating with the Photon Multiplayer voice demo - Everything works great in first person, in multiplayer mode, if I create the room, I can see lip sync on the other player, but it stops working on my machine.

I feel like I should be adding an entry to PhotonView perhaps? Any thoughts much appreciated.

 

Hi MonkeyPuzzle,
We have only experimented with Photon Voice and are far from Photon experts. In PhotonVoice, the prefab for the spawned character needs its SALSA instance to be set to wait for the an AudioSource that PV attaches to the player once spawned. If you are seeing lipsync it sounds like you have most likely already configured this sort of setup. If the lipsync is visible, but stops after some time, my guess would be the serialized audio stream is somehow being disrupted, or the AudioSource is being lost on the spawned character. Confirm your spawned character has an appropriate AudioSource and the audio is playing through that source. You can start your troubleshooting at the SALSA component on your spawned-character and see what AudioSource it is attached to. It might be of benefit to post your question to the ExitGames forum for assistance, unless someone else here has more information.

D.

 

SALSA Suite v2.3.1 is now live on the Asset Store!

That took longer than expected, but nonetheless it is now ready. The primary reason for this update was to correct a couple of issues occurring with the visual-code/design add-ons. We have also published the corresponding visual-code/design add-ons for this new version of SALSA Suite. The appropriate add-ons are like-versioned (v2.3.1) and ONLY work with SALSA Suite v2.3.1+.

We will leave the old (pre-v2.3.0) add-ons and OneClicks on the downloads site for a short time. Please read the downloads descriptions to ensure you grab the correct versions for your version of SALSA Suite.

SALSA Suite v2.3 produces much better quality results and supports more advanced implementations than previous versions. We highly recommend using this version.

Always back up your existing projects before updating any Asset or Add-on!

Thanks and enjoy!
D.

 

Hi, the 2.3.1 Slate add-on works great. The new Eyes_EnableAll action however seems to be mostly identical to the old Eyes_Set action. Is the new action the recommended one for now?

 

Hi, that's great! The new Eyes_EnableAll action allows you to independently enable or disable processing in each of the modules of the Eyes component (head, eyes, eyelid blink, eyelid tracking).

This action exposes the EnableHead(bool status), EnableEye(bool status), EnableEyelidBlink(bool status), and EnableEyelidTrack(bool status) methods. See theEyes API section of the documentation for details.

Michael

 

Thanks. Do we need to delete Salsa 2.3.0 components from existing characters, and reapply 2.3.1 ones?

 

I wouldn't think so. The differences between the two versions should mainly be with Inspector display and not underlying data as far as components on models go.

D.

 

Hi, Salsa works great for my use case. I have few questions

1. What the best way of sharing Viseme and Emote configuration between project? I tried with prefab it worked once but not anymore not sure why keep losing viseme and emote configuration. I have complex configuration so it will be take hour do it manually and prone to mistakes.

2. Is there any way to make it update at LateUpdate?

 

Same here actually, they said I need to write a custom Oneclick add-on based on the existing templates. It's a pretty involved DIY solution to a common problem: it takes an hour to set up a Salsa-enabled character up w/ sufficiently complex visemes+emotes, and they said portability is not implemented. Salsa is great, but this is a major pain in the butt for obvious reasons. Previously I tried to solve it with Presets (introduced in Unity 2018.1), but no dice, it's not made for that.

 

Having a problem with UMA One Click.

-Im using UMA 2.9
-Unity 2019.3

Downloaded the newest Salsa from the Asset store. Right when I import the One Click I get this error.

Assets\Crazy Minnow Studio\Addons\OneClickUMA\OneClickUmaDcs.cs(5,32): error CS0246: The type or namespace name 'OneClickBase' could not be found (are you missing a using directive or an assembly reference?)

I saw other people had this problem and looked at the documentation but I didnt see anything that would help with this problem since its breaks right when I import the asset. I cant start Unity with this error either. I also have the expression folder installed. Image below. Thank you!

 

Hi Aldo-V,

The recommended approach is to create your own one-click setup. This is exactly what we do for all of the character systems we support with a one-click setup. It's really not that involved if you start with one of the existing one-clicks as a point of reference, and use the API documentation to help guide you through what various parameters do.

Keep in mind, unlike SALSA v1, where one-click setups served to tie the system and the character together, one-click setups in v2 are just configurations of what is already available within the core system.

Michael

 

Hi mfitzpatrick-1,

As the documentation states, the OneClick Base is required for all OneClicks.

https://crazyminnowstudio.com/docs/salsa-lip-sync/addons/one-clicks/
OneClick Installation

• Make sure SALSA LipSync Suite is installed.
• If upgrading from previously-installed OneClicks prior to v2.1.0, first remove the existing OneClick directory located here:
  Assets > Plugins > Crazy Minnow Studio > SALSA LipSync > Editor > OneClicks
• Download and install the latest OneClick Base (REQUIRED for all OneClicks). Also ensure the appropriate version of the SALSA Suite is installed. Release notes for OneClickBase are located in the script itself.
• Download the appropriate model-specific OneClick unitypackage and install it. Release notes for all OneClicks are located in the OneClick Editor scripts.

 

Oh I see. my mistake.

High Level Question.

With UMA and Salsa. Can I get UMA to work with Salsa if i'm not using bone pose's for the Visemes but blendshapes? I know the expression player is all bone based but are there other work around for this that you know of?

Thank you agian.

 

Hi Gregorik,

Portability is implemented in the form of our one-click setup scripts. If you have several characters to setup, it's well worth the time investment to write a one-click setup to fit your needs specifically. For obvious reasons, we can only create one-click setups for common character systems that have a large user base, and a stable design. This is why we encourage implementing your own for your own custom character needs, or even just your own tweaked settings on one of the setup we wrote.

Michael

 

Hi mfitzpatrick-1,

Our UMA implementation is designed as a fast track to get up and running with UMA and leverages the expression player to achieve this. That being said, SALSA inherently supports many different setups, including bones, blendshapes, sprites, materials, etc. UMA is a complex run-time system, so you would likely need to dig into how our one-click setups work and implement your own hook into the UMA OnCharacterCreated event as a trigger to get things kicked off.

Michael

 

Thank you Michael. up and running with Salsa now.

Cant figure something out. What would cause their blink to get stuck? I keep messing with the controls and cant get them to blink normal.

 

Disable "Merge with Influencer" in the QueueProcessor component. This will stop the system from trying to merge back to whatever the animation controller is doing.

 

Creating Custom OneClicks:

As a reminder/re-post, a few days ago I posted this notice about new documentation for creating Custom OneClicks.

PS - The SALSA Core modules already operate in LateUpdate().

D.

 

Perfect, Thank you!

 

Is it possible that there's now an eyelid issue with the Reallusion CC One-click? I just upgraded to 2.3.1 but found no updated one-click. I took a quick stab at using the one from 2.3.0 but it didn't seem to work... Happy to investigate further tomorrow and post the error messages.

 

Hello, the CC3 OneClick v2.3.0 should be the correct one for SALSA Core 2.3.1. If you find an issue or error, please let us know. We've seen so many variants of CC and iClone models...it is possible you've got one we need to adjust for. You shouldn't be getting errors though, other than maybe some debug outputs about not finding shapes or meshes. Let us know. In fact, if you do have a problem, go ahead and send your info (invoice number/software versions/errors/issues) and model to us via email and reference this forum post.

Thanks!
D.

 

I am sorry, I misunderstood your problem previously. I thought you were having an intermittent issue with the SALSA instances that were working. OK, so again - I'm definitely not a Photon expert, but I do believe that is a limitation of the PhotonVoice integration as well. We can make the local player lipsync work in Dissonance because the Dissonance guys worked with us and feed us audio calculations that we then feed back into SALSA as an "external" calculation. IF Photon has some sort of audio calculation value that can be read for the local player stream, it could be funneled back into SALSA in the same manner.

At this point, SALSA doesn't have a way to monitor an audio stream that is happening down the fmod pipeline (if that's where it is going) and would know nothing about a custom stream solution. Using more than one microphone script (micInput + Photon) likely won't work. One or the other will win out. If you know where the audio stream is going it is possible you could tap into it and analyze the data, feeding it back to SALSA via the "external analysis" field. If they are using the standard Microphone class, there is likely an AudioClip buffer somewhere. But there are a lot of "ifs" in those caveats.

D.

 

We just verified that there is in fact an error related to the eyelids. I'll forward a screencap of the detailed error message shortly.

 

Any idea why the Eye script would be turning off at runtime?

 

No idea. Ensure there are no other scripts disabling the component.