Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Discussion in 'Assets and Asset Store' started by Crazy-Minnow-Studio, Apr 23, 2014.

  1. Hi.

    Recently I have started to use Fmod. The problem is that it's very hard to drive any lip sync (Salsa included) from another thread with data. Since the audio files aren't in Unity's audio system anymore but in sound banks coming from the middleware.
    Now, I have found a somewhat solution a couple of pages back (thanks @xibanya ). I grabbed it and ran with it.

    Here is what I came up with (for other fmod users):
    Code (CSharp):
    1. using CrazyMinnow.SALSA;
    2. using FMODUnity;
    3. using System;
    4. using System.Linq;
    5. using System.Runtime.InteropServices;
    6. using UnityEngine;
    7.  
    8. public class SalsaFmodBridge : StudioEventEmitter
    9. {
    10.     [SerializeField]
    11.     private Salsa3D salsa;
    12.  
    13.     [SerializeField]
    14.     private float averageModifier = 8f;
    15.  
    16.     private FMOD.DSP dsp;
    17.     private FMOD.ChannelGroup channelGroup;
    18.     private FMOD.Studio.PLAYBACK_STATE state;
    19.  
    20.     private void Awake()
    21.     {
    22.         if (PlayEvent == EmitterGameEvent.None)
    23.         {
    24.             instance = RuntimeManager.CreateInstance(Event);
    25.         } else
    26.         {
    27.             instance = EventInstance;
    28.         }
    29.         RuntimeManager.LowlevelSystem.createDSPByType(FMOD.DSP_TYPE.FFT, out dsp);
    30.         RuntimeManager.LowlevelSystem.getMasterChannelGroup(out channelGroup);
    31.         channelGroup.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, dsp);
    32.     }
    33.  
    34.     private void Update()
    35.     {
    36.         if(IsPlaying())
    37.         {
    38.             IntPtr unmanagedData;
    39.             uint length;
    40.             dsp.getParameterData((int)FMOD.DSP_FFT.SPECTRUMDATA, out unmanagedData, out length);
    41.             FMOD.DSP_PARAMETER_FFT fftData = (FMOD.DSP_PARAMETER_FFT)Marshal.PtrToStructure(unmanagedData, typeof(FMOD.DSP_PARAMETER_FFT));
    42.             var spectrum = fftData.spectrum;
    43.  
    44.             if(fftData.numchannels > 0)
    45.             {
    46.                 salsa.average = spectrum[0].Average() * averageModifier;
    47.             }
    48.         }
    49.     }
    50. }
    51.  
    The catch is, you need to open up the StudioEventEmitter.cs as well and change line 26 too.
    Code (CSharp):
    1. private FMOD.Studio.EventInstance instance;
    to
    Code (CSharp):
    1. protected FMOD.Studio.EventInstance instance;
    In my experience the averageModifier works in the region of 6 to 12-ish... and make the lip movement a bit better.

    Also it's not the best quality drive for a lip sync. The problem is, I have no clue, what I'm driving and with what data. (Yeah... I know) Audio and lip syncing aren't my strong suits to put it mildly.
    @Crazy-Minnow-Studio do you have any better idea to drive Salsa with any data from any middle ware (either Wwise or Fmod) in any way? Have you made any tests in the subject?
     
    Last edited by a moderator: Feb 4, 2019
    luguoshaonian, xibanya and HamFar like this.
  2. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    @Lurking-Ninja we have not, not in any serious capacity. Looks like you've made some good progress!
     
  3. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I'm going off topic here, but is it really worth it to use expensive external audio libraries like fmod or wwise with Unity?
    I know they are good for automatic music branching, but isn't it just easier and cheaper to implement this yourself in Unity? (or buy a specific plugin from the assetstore for this)
    Am I missing something?
    I'm really curious about this.
     
  4. valentinwinkelmann

    valentinwinkelmann

    Joined:
    Nov 3, 2014
    Posts:
    188
    Your website is down, so I can't download the uma plugin, can you upload it somewhere else ? I'm currently experimenting with uma and would like to test it.
     
  5. SickaGames1

    SickaGames1

    Joined:
    Jan 15, 2018
    Posts:
    1,269
    2.0 done yet?
     
  6. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi Diego_Graphics,

    Our website is up and available. Please send your invoice number to assetsupport@crazyminnow.com along with the link you're trying to view.

    Thanks
     
  7. It depends. Middleware like Fmod or Wwise can help you a lot. First of all, it keeps audio separated from your code and scenes. You can easily test ideas and edit both sfx and music. They can help you to build layered music as well, easily changing it on events.
    Also if you're planning to work with other people like musician/composer and/or sound designer or audio engineer, it's almost mandatory.
    Not to mention that it runs separate thread so you aren't tied into Unity code that much, unless you need things like this (lip sync). I like fmod studio far more than unity's audio tools in general.
    Of course, you can use things like Master Audio and such, although I have never tried it, I'm trying to see if Sectr Audio can have any use for me. I'm evaluating and experimenting.
     
    Last edited by a moderator: Feb 7, 2019
  8. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
  9. Last edited by a moderator: Feb 7, 2019
  10. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Thanks for the details Lurking-Ninja. We're looking into it, but have not been able to reproduce the issue on four different connections so far.
     
  11. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    We have found and resolved the issue on our website that was effecting some Chrome and Safari users. Thanks for letting us know, and providing great details.
     
    Lurking-Ninja likes this.
  12. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Yeah, maybe that's the main reason after all: it has become an industry standard like Photoshop or Maya and professionals are used to it. Granted, there is a reason why this software became a standard in the first place: it was very good at its job. But as time goes by, I find the premium price or subscription they ask for their solutions less and less justified. But maybe that's because I switch tools very frequently and I don't have to deal with a team of designers :p
     
    Last edited: Feb 7, 2019
  13. valentinwinkelmann

    valentinwinkelmann

    Joined:
    Nov 3, 2014
    Posts:
    188
    I was hoping it was just a temporary downtime. The website is now available. Thanks anyway for the quick answer.
     
  14. HamFar

    HamFar

    Joined:
    Nov 16, 2014
    Posts:
    89
    Many thanks :) I will go through the pages and learn about the requirements. In the meantime, can you (anybody on the forum) tell me please if you would know of a highly-realistic character (a human-like avatar) that you have used with "SALSA & RandomEyes" with results that have impressed you in terms of realism? I am tasked with buying one that can be used with SALSA and RandomEyes in Unity, but there are lots of options, so I thought the best start would be asking the members of this forum to see if anyone could recommend something extraordinary. Thanks in advance,
     
  15. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Adobe Fuse (my avatar).
     
    HamFar likes this.
  16. cygnusprojects

    cygnusprojects

    Joined:
    Mar 13, 2011
    Posts:
    766
    Or Character Creator 3, works fine for me.
     
    HamFar likes this.
  17. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Since you seem to have experience with both, how does it compares to Daz3D models for Unity use?
     
    HamFar likes this.
  18. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Royalty-free is a plus. ;)
     
    HamFar and atomicjoe like this.
  19. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Do you use it with Daz models or iClone models?
    How is the export to Unity working? Are the materials correctly exported or do you have to edit them by hand in Unity?

    I have trust issues with Reallusion since I bought a Pro Developer license of CrazyTalk for Unity and they never updated it and completely deprecated CrazyTalk... That was a lot of money directly to the trash can.
     
  20. magique

    magique

    Joined:
    May 2, 2014
    Posts:
    4,030
    I have CC3 as well. I used a Daz model with a Daz outfit and exported to CC3 then did the final modifications and exported to Unity. I did have some issues with materials and LODs though. Most LOD levels looked horrible and the textures were all messed up. I finally found a way to just get a single mesh exported that was not too high poly and was textured correctly. I haven't reported any of my issues yet and not sure if I have the latest version anymore, but it definitely needs some bug fixes.
     
    atomicjoe likes this.
  21. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Same! I even bought the stupid Unity plugin for CT… I will not be throwing any more money at Reallusion.

    I love Fuse. I'm of course sad that Adobe bought it just to nab the design team (so mad that to "spite" Adobe I spent a year and dumb money looking for a replacement before coming back to Fuse).

    Import directly to Unity is fine. Some other programs have issues with the FBX changes over the last 3yrs (Blender) and Unreal has some bizarre need for a "ground bone" or something which Adobe decided to stop supporting (probably because no one uses Unreal), but Fuse was designed for Unity and will atlas its own textures if you tell it to.

    It seems to run better on Mac (or maybe some flavors of Windows have launcher problems). It's in a weird limbo where Adobe hasn't mothballed it, but they stopped supporting 3rd-party bells-and-whistles. They streamlined the Auto-rigger to get rid of extras (imo this improved things, but if you used the extras it was a dealbreaker). Unlikely to see any changes anytime soon, so it is "stable", –– unlike Morph3D who lied from the day of release, then stopped returning emails when they realized they actually had to support Unity's upgrade-and-break-everything cycle.
    TBH, maybe I'm getting old, but knowing something won't be broken in the next update is sort of a good thing… I mean it won't break because there is no next update, but still… one less headache to have a consistent system.

    This is Fuse textures with Unity Standard Shader:
    JulianS-webbanner06.png

    I have an itch.io store with some Fuse freebies, and a few tutorials on Youtube.
    I'm over on the Fuse forums: here's a thread about creating a custom skin shader with pics. There was a strong community before the Adobe purchase. Based on my youtube numbers this year I'd say there's still a sizable userbase, but there's not a lot of chatter. Fuse has a candy interface, but they were the best semi-pro tool in 2015. It's not difficult to bridge to other tools (Wrap3, MarvelousDesigner) making your own clothing/original figures. etc.

    SALSA 1-click is also a plus, but almost any figure can use SALSA.
     
  22. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    That's not getting old, that's getting the job done! :p
    Thank you very much for the deep insight.
     
    wetcircuit likes this.
  23. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    That's EXACTLY what I supposed would happen just looking at the videos, but I had a little hope I could be wrong.
    I don't have much confidence in Reallusion.
    Not that Daz3D is much better software wise, but at least they don't sell you false hopes.
     
  24. magique

    magique

    Joined:
    May 2, 2014
    Posts:
    4,030
    @atomicjoe CC3 can still be valuable though. I am using it for my main character and it looks good. I will eventually report my issues and hopefully they will fix it. Here is my character imported from CC3:
     

    Attached Files:

    wetcircuit and atomicjoe like this.
  25. cygnusprojects

    cygnusprojects

    Joined:
    Mar 13, 2011
    Posts:
    766
    I followed along with their latest tutorials (targeting Unity) and the import tool they provide and don't have any issue with LODs and the materials as thus far.
     
  26. HamFar

    HamFar

    Joined:
    Nov 16, 2014
    Posts:
    89
    Hi all, Many thanks for everybody's insightful answers, particularly wetcircuit's detailed posts. From all this, I get that it is best to go for an Adobe Fuse character (such as @wetcircuit avatar) if you need something really human-like and would like to import it directly into Unity without texture problems, and would like "SALSA & RandomEyes" to work on it, especially SALSA 1-click. Is this an accurate summary?

    Also, @wetcircuit You mentioned you have an itch.io store with some Fuse freebies, and a few tutorials on Youtube, forums presence, etc... As I am going along and learning about this stuff and developing a conversational avatar with TTS, STT, etc... I am putting together a little readme doc. Is it OK if I mention your links in them, so other students will be able to check out the store, forums, etc... directly, rather than like me searching around for all this useful info?
     
  27. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Sure, sure. Info is free. ;)

    My advice to students is don't pay for bells and whistles until you need them. You can't "throw money" at a problem by buying an asset that promises to do everything – eventually it will break, Unity will re-do their entire process, or the support will go belly up. It's a Catch-22, the more an asset does for you, the more vulnerable to software creep and Unity updates.... Morph3D tried to do too much, AND they wanted to control purse strings so you had to use their tools to create new content…. Most figure systems are a carrot within a suite of 3D software or a content store, or (like Reallusion) you pay to export the figure out of their pipeline. Adobe has no idea what they are doing in 3D and they aren't even trying to monetize Fuse. :rolleyes: At the same time they have more money than most nations, so there is zero chance they will disappear.

    Whatever system you go with, SALSA 1-click is fabulous. You can have a new talking figure in – well it is more than one-click, but within a few minutes of dropping a figure into the Unity project. It's not a set-up wizard, it's a preset, so you'll tweak the settings to suit the character but it's already working from the preset…. Figures get loaded with other systems (IK, navmesh agent, look controllers, custom shaders) so in reality there is no "1-click" anything in Unity, haha.

    Since you are using TTS voices, I'm sure you've seen RT-Voice in the asset store.… With SALSA you could develop using TTS voices, and later switch to pre-recorded voice actors. You won't have to replace SALSA just tweak volume sensitivity, and the scenes still work.

    :cool: Good luck!
     
    nirvanajie, atomicjoe and HamFar like this.
  28. SickaGames1

    SickaGames1

    Joined:
    Jan 15, 2018
    Posts:
    1,269
  29. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Yeah! We want 2.0!
    Lets grab our torches and pitchforks! :p
     
  30. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    NOOO, not the torches and pitchforks! ;) OK, we're not quite finished, still cleaning up some Eyes integration. But I do have a question for you all. One thing holding us up at the moment is converting all of the add-ons (the add-ons that aren't being deprecated). We can do one of two things: A) continue to convert the add-ons and hold release until they are finished. B) submit the package (when complete -- again, almost there, but not quite) and then complete the add-ons after submission. We have quite a few that are complete (see the devBlog for details) and the 1-clicks should be wrapped pretty quickly -- all but UMA DCS which is a bit more challenging. The code-less systems like Playmaker, NodeCanvas, etc. are still in the queue to convert.

    I don't have a time-frame on any of it at the moment (hopefully another week or two). I can say this...if we submit as soon as Eyes is finished, regardless of the add-ons, we can start the Unity submission process timer (which was quite long the last time we submitted a new product). And then use the approval time to wrap up the add-ons and administrative tasks, understanding that some of the add-ons may not be ready when the product is first available on the Asset Store. Option B would be my preference, but with thoughts of pitchforks...let's hear what you all think...

    Thanks,
    Darrin
     
    atomicjoe and wetcircuit like this.
  31. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    I'm for option B all the way. (since the Unity store takes some time)
     
    Crazy-Minnow-Studio likes this.
  32. HamFar

    HamFar

    Joined:
    Nov 16, 2014
    Posts:
    89
    @wetcircuit Awesome, thank you!

    One other thing: I downloaded the Adobe Fuse and realized that there are only 16 characters available. Is that right or am I missing something?

    I have been told by my superior that I should buy something like this, though I am not sure if it will be easy/possible to integrate Salsa and Random Eyes in this, or manipulate its blendshapes. In the description, it says it is ready for you to "rig" and thus the perfect place to start if you are interested in creating a facial rig. Does this mean I have to ask someone to do the rigging for me and then it will be ready for animating the lips, face, eyes, etc...?

    If money was not an issue, would anyone go for one of these to use with Salsa and Random Eyes and plug-ins that manipulate blendshapes, such as this Blendshape Preset Tool?

    Many thanks in advance
     
  33. Firlefanz73

    Firlefanz73

    Joined:
    Apr 2, 2015
    Posts:
    1,314
    I also vote for B. :)
     
  34. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    1,409
    Fuse comes with 10 bodyscan figures, plus a few "generic" starter figures (Male, female, toon). In each case you can load the figure and make adjustments under the Customize tab to the face and body. You can download the Content Creator Package under the File menu, for base OBJ figures that you can import in your 3D app and resculpt. The professional workflow is to use wrapping software like Wrap3 or the new Z-wrap plugin, which allows you to take any figure sculpt and wrap the game-ready basemesh to it. This is how the bodyscan figures were made.

    I regret to inform you that your "superior" does not know what he is talking about :rolleyes: , but invite him to purchase the non-rigged head without any blendshapes for you. Do NOT spend your own money on that, it is not a beginner project. Here's the standard Fuse figure with fancy lighting and shaders in Unity, to compare that $40 retopo head. My figure came from Fuse, is already rigged and has blendshapes (the shader is a custom I made with Amplify Shader Editor). Fuse is still free. The $40 head has no rigging and no blendshapes, you are meant to create them yourself.



    Rigging is bones. Blendshapes are "alternate" mesh data (called "morphs" in other software). They are not the same thing, but both could be used for facial animation. SALSA works with mouths that are blendshapes (like Fuse) or mouths that are bone-rigged (hinge jaw). SALSA 2 should be able to handle more complex face rigs, but for right now SALSA 1 is either hinge jaw or blendshapes, but not complex facial bone rigs.

    I haven't used the Blendshape Tool. Crazy Minnow has a similar tool that can combine blendshapes into a new blendshape. (I also use an asset called SKINN:VM, put it in your wishlist).

    SALSA/RANDOMEYES 1 has "groups" which is a similar concept. I can create face expression presets that (for instance) include a smile left, smile right, mouth open, eye squints, etc. Name the group in RandomEyes and it can be called by code. SALSA 2 will move this function to a new dedicated component, so same idea but better, and the expressions you make still can be called by code or triggered intermittently the way RandomEyes now works on single blendshapes, so the figure can cycle through a random selection of pre-made expressions as they idle, and another set of expressions that trigger by SALSA2 when the figure is speaking. I have coded this for SALSA1 in the past, so it's possible now, but SALSA2 will make this easier to set up and code-free.

    Since we've hijacked this thread :p feel free to PM me or ask questions over on the Fuse forum. This stuff took a while for me to get my head around – not that it is so complicated, but I didn't know what questions to ask and the whole thing seemed very unclear and mysterious…. Some of it is a bit mysterious because pros don't always share all their tricks. Play around with Fuse and see what you can do with it. It is designed to make creating and importing your own content easier, but there is not a big marketplace for ready-made figures. Instead Mixamo wanted users to have a semi-professional path to creating their own figures.
     
    Crazy-Minnow-Studio and HamFar like this.
  35. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    Yeah, let's get back to our torches and pitchforks please. :p
     
  36. SickaGames1

    SickaGames1

    Joined:
    Jan 15, 2018
    Posts:
    1,269
    Lol i kid! As long as it works with Daz and Dialogue system, then make it so!
     
    TonyLi and Crazy-Minnow-Studio like this.
  37. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    @Crazy-Minnow-Studio I have a couple of presale questions. I'm doing 2D animation (not for games but videos) and would like to control lipsync and expressions on the timeline.

    1: The manual page suggest 2-4 sprite shapes. Does this mean SALSA is not identifying phonemes but using amplitude for approximation?

    2: Can I use SALSA with the mic input and record the audio at the same time? Once recorded can I use that audio on the timeline?

    3: I see most solutions have a degree of lag eg voice audio starts and then shapekeys start moving. Is there a way to split the audio from the shapekey info so we could use some kind of offset to make the shapekey response time faster?

    4: Anyway to have a number of emotional expressions we can switch between? I can imagine my characters speaking with varying emotions and would like to control (via timeline) eyes, eyebrows, and mouthshape - eg anger, disgust, fear, happiness, sadness, surprise, amusement, contempt, contentment, embarrassment, excitement, guilt, pride in achievement, relief, satisfaction, sensory pleasure, and shame

    Thanks
     
  38. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hello!
    We have a couple of Timeline add-ons available for SALSA, allowing audio switching that SALSA can process and emote firing. SALSA cannot work with the default Unity AudioSource Timeline playable -- Unity doesn't expose the AudioSource for linkage. So, the answer to your Timeline question is 'yes'.

    1) That is correct. It uses amplitude analysis for the basis of its processing -- i.e. *not* a phoneme mapping solution.
    2) We have a micInput add-on/example that links SALSA to a microphone for real-time input and lipsync. The add-on does not record audio, you would need to implement that yourself or find another asset that does it for you. Pre-recorded audio is certainly usable and may be preferred based on your design needs.
    3) Most solutions? SALSA v1 does have a slight lag due to the audio pipeline involved. SALSA v2 has the ability to specify a look-ahead bias, greatly reducing/eliminating the perceived lag. This method is limited when using the microphone however, and must be used with caution or you can easily get ahead of the microphone record buffer and suffer some very strange effects as you process the buffer history.
    4) Yes -- see the Timeline answer above. However, with 2D and sprites, you will need to design your sprite layouts to accomodate. Obviously, it wouldn't be possible to blend speaking with an emote that uses the mouth -- like a big smile. However, you could have a mouth shape with a big smile and then animate only the mouth opening (as an example).

    Hope that helps. Be sure to check our site for more information and our SALSA v2 devBlog for the skinny on what SALSA v2 is bringing to the table.

    Thanks for the inquiry!
    Darrin
     
  39. ghtx1138

    ghtx1138

    Joined:
    Dec 11, 2017
    Posts:
    114
    Hi Darrin

    thanks for your quick and comprehensive reply.

    Cheers
     
    Crazy-Minnow-Studio likes this.
  40. vibedev

    vibedev

    Joined:
    Nov 1, 2018
    Posts:
    13
    I have a script that uses the public void SetLookTarget(GameObject obj) method, but since I could not see any effects, I right-clicked on it and selected Go to Implementation, but I get a message that says The symbol has no implementations. When I click on Go to Definition though, I am taken to the signature of the method... Do I have to implement this myself? Where do I go to see how this is done?

    I am trying to get some control over the eyes, so that instead of random movements, I can do things myself; e.g. lookUp, lookDown, lookLeft, lookRight, blink, etc... Where do I find the code for these movements? Thanks in advance
     
    HamFar likes this.
  41. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi vibedev,

    The RandomEyes API is documented in the manual: https://crazyminnowstudio.com/unity...randomeyes-manual/classes-methods-properties/

    My first thought is that you may be using one of our 1-click setups. Several of them use two instances of RandomEyes3D, one for eye control (use custom shapes only = false) and the other for expressions (use custom shapes only = true). If you aren't calling the SetLookTarget method against the eye control instance, it would appear as though it's not working.

    If you tell us more about your setup, Unity version, SALSA version, add-on's used and versions, we can better assist you.

    Michael
     
    vibedev likes this.
  42. vibedev

    vibedev

    Joined:
    Nov 1, 2018
    Posts:
    13
    Hi Michael, Thanks for your reply. I am using Unity version 2018.2.17f1 with SALSA & RandomEyes version 1.5.5 with Salsa 1-click AutoDesk setup, purchased on 22 October 2018. So, your first thought is actually right.

    My AutoDesk character has random eye movements, but never blinks, so I am trying to see how I can control eye movements and blinking. I am specifically trying to get "this" working, but as an exercise to learn about controlling eyes movements, so I would appreciate your further help.
     
  43. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi vibedev,

    Was your AutoDesk character created with a current version of the AutoDesk Character Generator? If so, please send your character model to assetsupport@crazyminnow.com so we can inspect the model to see if AutoDesk has made any base-level configuration changes.

    Please reference your forum name in the email so we have context and know who the email came from.

    Thanks,
    Michael
     
    vibedev likes this.
  44. vibedev

    vibedev

    Joined:
    Nov 1, 2018
    Posts:
    13
    Hi Michael,
    I created it in September 2018 and I don't know what might have changed with the AutoDesk Character Generator behind the scenes, so may I email it to you anyway?
    Cheers
     
  45. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Yes, please email the model to assetsupport@crazyminnow.com and reference your forum name so we have context and know what the email is in regards to.

    Thanks,
    Michael
     
    HamFar likes this.
  46. therewillbebrad

    therewillbebrad

    Joined:
    Mar 2, 2018
    Posts:
    151
    Does version 2.0 integrate better with master audio? I'm wanting an easy method of playing master audio files and am hitting dead ends with the current version.

    IE play group audio and not have to drag around audio clips.

    If so I will hold off on hand animating these mouths. Look forward to 2.0!
     
  47. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hello Brad. I just responded to your email. They (v1.x and v2.x) integrate more or less the same way with Master Audio. I just recently converted the code to work with SALSA v2 but that only required some refactoring to remove the Salsa2D/Salsa3D logic -- it is now simply Salsa. I just tested SALSA v1.x and 2.x with MA groups integration (as your email suggested you were using) and it all worked just fine.

    To utilize MA's groups:
    1. Setup a group according to MA's documentation.
    2. Configure a character with SALSA and confirm it is working.
    3. Add the groups integration addon from the SALSA downloads section (or roll your own linkage code) to the SALSA-enabled model/character.
    4. Configure a way to call the group playback (i.e. a gui button as in the demo scene).
    5. Play the scene and invoke the playback.

    Hope that helps,
    Darrin
     
    SickaGames1 likes this.
  48. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    does this work in runtime? where working in Google Cloud Text To Speech then in Speech will be added in the character side.. which we want to talk..
     
  49. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,398
    Hi domdev,

    SALSA is a run-time system. It requires a Unity AudioClip to be played through a Unity AudioSource component. There are several TTS systems that generate audio outside of the Unity API, these systems will not work with SALSA unless you can get the generated audio into Unity. I don't have experience with Google's TTS so I can't speak on it. We typically recommend the RT-Voice asset, and have had some success IBM Watson's TTS.

    A few links you might find useful.
    https://crazyminnowstudio.com/posts/salsa-and-rt-voice-for-runtime-lipsynced-text-to-speech/
    https://crazyminnowstudio.com/posts/ibm-watson-and-salsa-lipsync/
    https://crazyminnowstudio.com/posts/accessing-web-based-audio-resources-in-amplitude/
     
    vibedev likes this.
  50. Firlefanz73

    Firlefanz73

    Joined:
    Apr 2, 2015
    Posts:
    1,314
    Hello,

    Until now I was using Salsa together with UMA only. Works very well.

    Now I have got some (non UMA) monsters, that have a mouth, but no blendshapes(yet). So I need to add blendshapes to the Monsters meshes first to get them moving their lips, right?

    Thanks.