Search Unity

SALSA Lipsync Suite - lip-sync, emote, head, eye, and eyelid control system.

Discussion in 'Assets and Asset Store' started by Crazy-Minnow-Studio, Apr 23, 2014.

  1. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hi lazygunn. no offense intended with the italics, they were purely used as stylized elements--just trying to help. We've already been talking about adding some elements of your latest suggestions to improve the product. And looks like I misread your question concerning the curves implementation. Unsure how difficult that would be at this point. Likely we would try to leverage the existing curves system, but we'll have to take a longer look at it.

    The reason the shape currently snaps back is due to the error clamping that is in v1.0.0, to prevent additive blendshapes from distorting the morphs. We're discussing whether or not that "feature" should be removed or modified. It's a fine line between error trapping Inspector properties and allowing greater programmatic control.

    What you are describing is possible using the existing Animations curve editor. Caveat, the rangeOfMotion properties for shapes are currently defined as type int. The Animation property browser doesn't support int types. A work-around at this point might be to use a helper-script that declares a float value and assigns that value to the rangeOfMotion property as an int conversion. The helper-script can then be driven from the Animation Curve, in turn driving the range property. I've quickly tested this and it seemed to work just fine.

    Additional v1.1.0 Information:
    CustomShapes will have a couple of options for activation. First, if there are set expression shapes that includes the mouth and eyebrow shapes (whole face type stuff) that should only be active one at a time. By setting the active shape, other shapes are reset automatically. Think of it as a radio-button sort of functionality. If smile expression is active and frown expression is activated, the smile shape will animate down to zero while the frown animates to the max range property. This works great if you've got expressions that are complete and exist as a complete shape.

    The second method can be used when there are components to shapes. i.e. mouth-only, brow-only, eye-only, ear, hair, etc. shapes. So, as a designer, if you want to enable the smile mouth with the raised brow. You set the range and speed and enable both shapes. This method has the capability of having more than one shape active and controllable.

    Hope this helps clarify, please let us know if not.
     
  2. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    FYI - our server host will be performing maintenance tonight, btwn 8PM - 4AM MST. During this time, there were will be a small time slot (approx 30 mins) where our web, forums, and email will be unavailable.
     
  3. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    The Cradle game looks very interesting, except the landscape looks mismatched to the human made thing but maybe that was intentional.
     
  4. lazygunn

    lazygunn

    Joined:
    Jul 24, 2011
    Posts:
    2,749
    Ahh I wasnt expecting you to write your own curves at all! no problem. I definitely meant the curves from the animation window. To overcome the int restriction on blendshapes a Mathf.FloorToInt would fix that before the value is given to whatever drives the blendshape. Combining blandshapes is exactly what I had in mind for this system, as, for example, on my models the visemes are pretty much mouth shapes, but the expressions can be anything from full facial like rage, to single eyelid blinks, raised eyebrow and so on. There are blendshapes that if used alongside others can completely set the mood of the animation, leaving the 4 position system of salsa working fine as it is and nuance added by combining these blendshapes - example being a snarl, how open the mouth is or even jaw movement. Salsa then represents a really nice quick way to get the facial animation off the ground then as intricate modifications made as you want with the curves provided with Unity's own animation window


    I'm not referencing the documentation here so i might be a bit off but this is a quick idea that might be worked around

    Even more interesting is it gives your object an Animator component and a state in a mecanim node tree, you could completely ignore this, or use the events you say randomeyes and salsa have or are getting, to transition to blendshape states inside the state machine. The average user could just want the basic functionality and that would not even remotely change, with the additional control there for those that want it, the tools already be there just need an elegant workflow for being able to give the system the capacity for a great level of nuance.

    I'd basically imagine that for a typical lip sync situation you'd have a randomeyes, or salsa component that will add animclips for each audio file involved in the system. You might even trigger the audio from within the animclip. If it was triggered by the animclip, it would automatically play when you played the animclip - put the clip on loop and then its very easy to create the curves in the animation window to match the blendshapes, the curves giving absolutely comprehensive control over blendshape's weight they are matched to, completely on time

    The fact an animator is added to the gameobject has me thinking the facial animation could be added as a layer to the models main animator (For it's skeleton animation) or attached to the gameobject in the model with the skinnedmeshrenderer that will automatically find the renderer and set everything up. I'm kind of just talking as things occur to me but basically a very elegant solution could involve a component that creates the animator if needed, if theres already one ask if they'd like to add the control to a new layer (if this is possible) or just attach an animator to a suitabe gameobject and add any audio clips specified, as this would create clips for each audioclip and match them.

    This audioclip/animclip pairing could then have you specify the blendshapes you wish included in it, and if you click 'edit' you have the animation window open with the curves for each blendshape present, press play and the audio plays, the character in, ideally, the editor window will animate with salsa and as you adjust the curves in the animation window you'll see the adjustments you make will be seen in real time. Simplicity of salsa, flexibility of typical lip syncing process without nearly being such a pain in the backside and probably very fun

    You could, if keeping randomeyes as the container for all these blendshapes, let people create animclips without audio and just let them animate a period of time, perfect for non vocal expressions and ambient facial movement and with each animclip appearing in the Animator for that gameobject, you could use it to show complex expressions based on events, with and without audio.

    I really like the idea of randomeyes' procedural nature, somewhat mimicking the way real eyes actually move, the quadrant idea being a good idea. Programatically you can obviously add bias to quadrants programatically or in-editor, being able to weight the bias both these ways further with floats would be nice. Using curves, again with a curve for each quadrant (All flat meaning straight ahead) would be great for even more control over how the eyes move, still mainly procedurally as it has a random element - you could get incredibly realistic mood communicating eye movement this way. A way to fully override or add programatically alterable modifiers (Again with weighting) would be great. The weighting really just accounts to multiplying the max clamped value with a 0 to 1 float, using FloorToInt for the result. A typical modifier would be to look at something specific, for example.

    Then all of this, being nodes in an Animator, can automatically make use of logic and values in there to make a facial animation state machine that you as developers dont really have to do that much thinking about, it's already designed to have the functionality for adding special behaviours there

    These all work around in editor and api controls that dont need to be initially obvious, just like click 'advanced mode' to expand the inspector, for example, and i'm not nearly as well aquainted with your system as you are so i wouldnt make these changes myself, but i do feel i havent said anything groundbreaking, but have described how you can make this a F***ing awesome tool, not just for people wanting something very simple, but also people wanting something capable of really nicely done facial animation in a package that makes it all so simple. I'm mostly motivated by the fact that I think the addition of blendshapes is a very recent feature and a great idea and you're one step ahead of the rest to make a very attractive full solution using stuff that Unity mostly already provides

    The only other thing I can think of is, firstly, and most importantly for me, let the eyes be controllable via a transform as well as blendshapes (Just rotation in two euler axis), because of the shader I use I have to replace the original eyes in the model with blendshapes with my own objects and I don't want to break the blendshapes. And if you havent seen UMA yet, you should go take a look, it will, hopefully oneday, be extremely significant to Unity - it has no current lipsync implementationand the one i know of has no automation. It is however driven by facial bones rather than blendshapes - this is probably a much more involved thing to support than my suggestions in this long post (sorry about that) and i dont even use UMA, but supporting facial bone animation as well as blendshapes eventually would be a real boon to the community, very appreciated, even if it's at cost (Or you make a free version that just has the basic lipsync and randomeyes as it stands, but for UMA or bone rigs only

    Big post, lots to say on the subject but im glad you understand me on the curves thing and I wrote a lot but i'm not worried about it being particularly complex to implement, its more that doing it elegantly and automated as possible could take some work. It would have, imo, real power there and to very little complexity. If you're still not sure about what i mean with the range of facial blendshapes available i can pm you with a model that will show you exactly what i mean, and its a model from a system which is really relatively affordable to license, has an enormous amount of existing content and would look absolutely the bomb with pbr shader treatment and strong facial expressions (And its not mixamo)

    Cheers!
     
  5. lazygunn

    lazygunn

    Joined:
    Jul 24, 2011
    Posts:
    2,749
    I think Cradle is going to be a very surreal, and beautiful, and bizzare bit of Banksian sci-fi, and is a massive influence to me. My own work is very much of that inclination although i dont think i can approach their handle on all those aspects. Such a beautiful and unique looking game, genuine game as art right there. Only other recent games ive seen with such beautiful bizarre-yet-awesome visual integrity are No Man's Sky (You're an old timer, goat, go look up videos of No Man's Sky if you havent already - be taken back to the seventies and 80s of a beautiful age of scifi artwork and feel very happy)and that new-coming Rogue-like which is apparently fantastic. Good to see leftfield artistically strong games are still there saving the world of gaming from complete embarrasment. I'm going to get this asset to help me help that too dammit
     
  6. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hey lazygunn! You've got a lot of great ideas here and questions.

    We've been talking about all sorts of ways to further improve RandomEyes, while (as you said) keeping the simple nature and allowing more advanced capabilities if someone wants them. I worked an example yesterday of driving eye-movement with the curves system, using the lookTarget. I was able to put some crude curves together to control an empty (target) in a somewhat circular patter to drive the eyes. It worked great! At first I created the circular movement too large and the eyes reached their limits and created more of a box pattern...narrowing the circle down quite a bit was the trick and the eyes moved perfectly with the target. Using this method, there is no end to the movements that can be driven on the eyes. It was really cool and no code required. That particular feature is available now in v1.0.0.

    In v1.1.0, the rangeOfMotion properties have been changed to floats, so there's no longer any problems accessing them with animation curves. Yea!!!! I think you'll be very happy with the new customShapes capabilities. It's an initial implementation, but opens up the door to so much flexibility. And it will be possible to drive their rangeOfMotion with the Animation curves. Nothing built into the Inspectors at this point.

    We have seen UMA and it does appear to be a really cool concept!

    We'll try to post a couple of tid-bit videos to demonstrate the stuff we've been talking about. Here's one vid demonstrating the lookTarget controlled with curves.
     
  7. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Here's a quick video to demonstrate some of the new features we're working on for the upcoming 1.1 release. This example is using the event parameter passed in the custom shape event to iterate through the list of available custom shapes. The Console shows these parameter details and parameter details contained in the RandomEyes look event parameter. The events and event recipients are controlled independently for maximum flexibility.

    A code example that demonstrate catching the events and reading parameter properties
    Code (csharp):
    1. void RandomEyes_OnLookStatusChanged(RandomEyesLookStatus randomEyesStatus)
    2. {
    3.     Debug.Log("RandomEyes Look Event: " +
    4.               "lookPosition=" + randomEyesStatus.lookPosition.ToString() + ", " +
    5.               "blendSpeed=" + randomEyesStatus.blendSpeed + ", " +
    6.               "rangeOfMotion=" + randomEyesStatus.rangeOfMotion);
    7.  
    8. }
    9.  
    10. void RandomEyes_OnCustomShapeChanged(RandomEyesCustomShape customShape)
    11. {
    12.     Debug.Log("RandomEyes Custom Shape: " +
    13.               "index=" + customShape.shapeIndex + ", " +
    14.               "shape=" + customShape.shapeName + ", " +
    15.               "overrideOn=" + customShape.overrideOn + ", " +
    16.               "isOn=" + customShape.isOn + ", " +
    17.               "blendSpeed=" + customShape.blendSpeed + ", " +
    18.               "rangeOfMotion=" + customShape.rangeOfMotion);
    19. }
     
  8. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    We've posted a QUICK-TIP for RandomEyes, using the Animation Curve Editor to drive eye-movement in RandomEyes3D. This method creates cool movement effects without requiring any code.

    Check it out here!
     
  9. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    $female.detailed.marketing.base.jpg
    This slide shows off the capabilities of a more detailed model with 3 very distinct emotions. Each of these 'looks', and an infinite variety more, will be programmatically accessible to work beautifully with SALSA and RandomEyes. Custom shapes will be available in the upcoming v1.1.0 release.
     
  10. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hey guys, the first two videos in the SALSA with RandomEyes Quick Start Series are online! These videos give a quick overview of what is included in the SALSA package and then move on to demonstrate how to set up SALSA for 2D and 3D use. The next set of videos, coming soon, will demonstrate the RandomEyes environment. Following the release of these intro videos will be more advanced topics. Enjoy!

    Also, code for the 1.1 update, that includes unlimited custom shapes, is complete. We have a few more details to work out on the updated manuals and new demo scenes before we submit the update, but it will be available soon.



     
  11. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hey guys, here is the third video in the SALSA with RandomEyes Quick Start Series. This video walks you through a quick setup with the RandomEyes system to provide random eye movement for your 2D and 3D characters.

     
  12. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    We've started another video series, Working With Shape Keys in Blender! The Intro video has been uploaded. Several videos are planned in the series. This initial video briefly discusses shape keys (blend shapes, morph targets, etc.) and the shape key interface in Blender. If you're curious about shape keys, this is a great place to start.

    [video=youtube_share;BhKCwbHv3oM]http://youtu.be/BhKCwbHv3oM
     
  13. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    We are very please to announce that our 1.1 update has been submitted to the Unity Asset Store and will be available soon. This update includes the ability to link an unlimited number of custom BlendShapes to the RandomEyes3D system. You can control these custom shapes automatically within a scope you define, or you can disable the random feature and enjoy full control over the custom shapes behavior using a comprehensive set of exposed public function. This system is great for use with emotions and any other BlendShape application. Each BlendShape has its own controls for activation, blend speed, and range of motion. We've also added a new RandomEyes event system that fires [RandomEyes_OnLookStatusChanged] events for 2D and 3D and [RandomEyes_OnCustomShapeChanged] events for 3D. Here is a screen shot of the custom shapes section of the RandomEyes3D inspector, and a video of a new demo scene we built and included to show off the new capabilities. See the first post in this forum for more details.

    $RandomEyes_CustomShapes.png

    [video=youtube;-g36P_2BtTw]https://www.youtube.com/watch?v=-g36P_2BtTw&list=PLcVmXGedVLuZ5dG2eEYVz7U8XK-NJ691K&index=4
     
  14. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Here is part 2 from our new Video Learning Series Working With Shape Keys in Blender.

    Part II - Face Keys

    [video=youtube;KEEeE6IcegM]https://www.youtube.com/watch?v=KEEeE6IcegM&index=3&list=PLcVmXGedVLuahpG6 bRniRryIc1QBz11Nk
     
  15. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Here is the forth video in the SALSA with RandomEyes Quick Start Series. This video goes through a detailed explanation of all the SALSA inspector properties for 2D and 3D.

    [video=youtube;fy5aa3syABk]https://www.youtube.com/watch?v=fy5aa3syABk&list=PLcVmXGedVLuYwkvHkqWYUu91 3q8YSsBzq&index=5
     
  16. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    The 3rd part of the Working With Shape Keys in Blender series has been posted. This tutorial covers some gotchas, there are a couple of tips that are specific to using Blender's shape keys (blendshapes) for Unity assets. Enjoy!

     
  17. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Last edited: May 31, 2014
  18. goat

    goat

    Joined:
    Aug 24, 2009
    Posts:
    5,182
    Thanks for that. I forgot. Bookmarked or rather 'Asset Store Heart'ed for when I have a bit of extra money. :)
     
  19. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    The Minnow has been hard at work on some great new features for the next 1.2.0 release. Here are some of the features we've been working on for RandomEyes3D.

    RandomEyes3D_1.2.0.png


    RandomEyes3D_CustomShapes_1.2.0.png
     
  20. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    A few of our customers have asked us about using SALSA with characters generated using the Autodesk Character Generator system. This system outputs tons of BlendShapes, but can be overwhelming when trying to figure out how to link SALSA to these characters. We thought it might help people to have a reference workflow to follow, so we dug into a couple of these characters, figured out what was necessary, wrote a couple of freebie helper scripts to assist with the linkage, and made a video tutorial that covers the process. We hope people using these generated characters find it useful.

     
  21. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Another quick update on the upcoming 1.2 release...

    While using SALSA with RandomEyes in another project, it became clear that we needed a way to use the random custom shape feature from RandomEyes3D, but we also wanted the ability to exclude certain custom shapes from the random shape selection process so that the excluded shapes could be activated from another process using the API.

    RandomEyes3D_notRandom.jpg
     
  22. eagleeyez

    eagleeyez

    Joined:
    Mar 21, 2013
    Posts:
    406
    I bought this yesterday and must say that I am very impressed. I am currently a Mixamo full access customer and messing about with Face Plus, but SALSA is opening new ideas for quick prototyping and development and perhaps even being used in my full projects.


    1. When using your examples the mesh render just shows Element 0, Element 1 and so on. In your Videos it shows the names of the blend shapes. It still works when referring to the Elements, but why do I not see names of Blend shapes like you have?


    2. PLEASE add Playmaker support. I use lots of assets that support PM.


    3. Keep up the dam fine work and thanks again for this
     
  23. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    eagleeyez discovered that another asset (Skele Animation Tools), unrelated to SALSA, was renaming the blendshapes.
     
  24. eagleeyez

    eagleeyez

    Joined:
    Mar 21, 2013
    Posts:
    406
    Yes, I have informed the developer of Skele, as I would like both of these very useful assets to work together.

    And yet another thought guys and girls.
    How about combining a smart phone recordig my head movements as I speak my text and then have that synct with SALSA? Was just a thought.
    Or even bring in the Myo and have have the hands/ arms and fingers also go into the SALSA.

    Now I have to watch the World Cup as it is starting now Germany/Brasil.
    all the best to everyone
    Chris from Germany.
     
  25. Xoduz

    Xoduz

    Joined:
    Apr 6, 2013
    Posts:
    135
    Hey, question about the sayRest shape... is this not the same as the absence of the other three mouth shapes, i.e. all of those set to 0, or does this have some specific use that I'm missing?
     
  26. eagleeyez

    eagleeyez

    Joined:
    Mar 21, 2013
    Posts:
    406
    Hi,

    Looking at SALSA and one of my Mixamo characters “Battery boy” I just can’t work out how to find the right shapes for the mouth and for eyes there is no up or down.

    The Mixamo rigging is their standard throughout their models. It would be great if you could show a work flow for these models
    AND what is also going to get very interesting is Mixamo Fuse will have automatic blend shapes around about August this year. This will be great as I can pick and choose as I go. Some stuff done with SALSA and some stuff done with Face Plus. For example indirect conversations further from the camera SALSA and direct head on conversations with Face Plus.

    Also will you be adding more automatic random actions like frowning, squinting, puff up cheeks and so on?

    I can imagine it working later like this:

    Automatic, programmable or puppet mode.

    Puppet is for example my audio clips plays and the Lipsync starts, I would then have keyboard control to the other blends that I set up in the inspector. Pressing 1 would squint left eye, 2 right eye, 3 frown and so on, you get the point.

    This data would then be saved to SALSA when finished with the playback. Playing back again would show / mix my puppeting as well.


    Christopher
     
  27. eagleeyez

    eagleeyez

    Joined:
    Mar 21, 2013
    Posts:
    406
    Just to let everyone know,

    The bug in Skele Animation Tools has been fixed by the developer of Skele and will be soon as update available as Skele version 1.5. This means Skele and SALSA both work together now, which is really great news. This Bug was not due to SALSA.
     
  28. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hi Xoduz! Originally, sayRest was a shape of its own. Some bugs were discovered and fixed in SALSA and the shape was retired. Now, as you stated, sayRest is no longer used and the closed position is simply the other positions set to 0.
     
  29. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Interesting idea. You're meaning some sort of motion capture from a smart phone video capture. That would certainly be cool, but unfortunately, I think a little out of the scope of the SALSA with RandomEyes goal. However, RandomEyes can drive any blendshape that is added to the model. So, if you do get some sort of motion that you want to drive, it will work as it is now. Additionally, if you have a head-bone that you'd like to have some subtle motion applied to without having to motion capture it, we have a small script that we've posted on our website that can do that.

    If you're interested, grab the files available in this post:
    http://crazyminnowstudio.com/posts/salsa-autodesk-character-generator-workflow/
     
  30. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hi again Chris!
    We intend, very soon, to create a video tutorial on how to create the basic shapes for SALSA and RandomEyes (mouth and eye movements). The last model that we saw didn't have any blendshapes attached, so there wasn't much we could do in the way of automatic workflow. Once Mixamo starts including canned blendshapes, it will indeed be very useful and interesting!

    Puppet mode recording sounds very interesting and is related to some development going on right now. It's a little different, but might ultimately give what you want to a degree. We're working on a timeline event system that allows adding events to a wav file that could then drive custom shapes or possibly be intercepted in your own custom code to perform other functions. We're ramping up a for a 1.2 release soon, but the timeline events probably won't be ready till version 1.3 or 1.4.

    Hope this helps!!!
     
  31. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Thanks for the update Chris!!!!
     
  32. EmeralLotus

    EmeralLotus

    Joined:
    Aug 10, 2012
    Posts:
    1,462
    Really great project.

    Can the scripts drive jaw bones and eye bones in addition to blend shapes. If not yet, how hard would it be to add these features ?

    Cheers.
     
  33. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hi *rocki*!
    The asset is pretty focused on blend shapes. However, we have a helper script posted on our web site that will manipulate eye-bones using a dummy object with blank blend shapes. This script was specifically designed to work with the Autodesk Character Generator models, so you might need to tweak it a little. The same principal could probably be used for a jaw bone, but you would only get chomping-styled movement out of it.

    The blog link with the helper scripts is here: http://crazyminnowstudio.com/posts/salsa-autodesk-character-generator-workflow/

    And, we've also posted a video that shows how to link these scripts up to an Autodesk Character. Might be beneficial for what you're wanting to do.


    Hope this helps!
     
  34. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Our new SALSA with RandomEyes version 1.2 is feature complete and nearly ready for release. We're going to run it through a few more implementation scenarios just to be sure we didn't introduce bugs along with the new features. Stay tuned for the official release details coming soon!

    We smell a release celebration sale in the air!


     
  35. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    We've updated our tutorial for using SALSA with RandomEyes v1.2 with the Autodesk Character Generator models. This tutorial also shows off a couple of the new updates that will be available in v1.2.

    We cover:
    1) Synchronizing the jaw object's blend shapes with SALSA.
    2) Driving eye-bone movement with RandomEyes.
    3) Targeting eye movement and the new Target Affinity option.
    4) Linking up blink shapes to RandomEyes.
    4) The new inspector shape pickers and Auto-Link.
    5) Custom shapes
    6) Adding some subtle head sway to your characters.

    Three helper scripts and a dummy translation mesh are available for use in this workflow. They are available on our website [http://crazyminnowstudio.com/posts/salsa-autodesk-character-generator-workflow/].

     
  36. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    SALSA with RandomEyes v1.2 has been submitted to the @UnityAssetStore. We expect it to be approved within the next couple of days.
     
  37. sxkx360

    sxkx360

    Joined:
    Aug 18, 2013
    Posts:
    7
    Hi,
    I´ve found your tool very very useful and I´ve sorted some problems of my own to get results with my project.

    I´m curious would it be possible to animate 2 blendshapes with one value other being the optional shape. I can give you an example to demonstrate the problem:
    I have a 3d characer and both eyes are blendshaped separate for more control ingame. With random eyes result is that only one eye opens and closes. I set my script to get the weight of blendshape used and then set that value over to the other blendshape. Also used it with Salsa3d values and got the separate teeth moving in-sync, overall pretty good results.

    So the questions is that could it be more accurate codewise and more efficient to add support to mirroring "gestures" aka blendshapes.

    Thanks again for your awesome asset I´m pretty sure to use it in most of my projects this on.
     
  38. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Hi sxkx360 and thanks for your purchase! Sounds like you're handling the situation the best way possible at this point. It might be slightly more efficient if it were handled in the asset code, since you'd eliminate an Update() loop. But that should be pretty minimal.

    But perhaps I can offer another solution for you. Assuming you've made your own blendshapes, perhaps consider adding an additional set of shapes for the eyes that affect both eyes/blinks. This would still give you the control you want for separate eyes, but also allow RandomEyes to have its own set that handle both eyes. The eye shapes are pretty easy to create for RandomEyes' requirements. And if you already have them separated, you could easily create a set of shapes that include both eyes with copy functions.

    In Blender, you'd use the "create shape from mix" option. Set both eyes in the correct position (or blinks), and use the specials option to "create shape from mix" and bam! you've got your new shape that can then be patched in to RandomEyes.

    Hope that helps and can't wait to see what you make with our asset! Thanks again!
     
  39. sxkx360

    sxkx360

    Joined:
    Aug 18, 2013
    Posts:
    7
    Thanks for the advice, I´ll practice with blender to get it solved, I´m pretty used with learning as I go. Actually you are totally right keeping Salsa and Random eyes simple, for it makes them easier to use and grasp.

    At the moment I´m using it to add some character life to my scene in a OculusDK2 car demo that makes it possible to enter the vehicle in a dealership (switching scenes) and then in-car drive out of the window or throught a crumbling wall. It´s with DK2 positional tracking, slomo and freeze scene functions. Basically modernized STUNTS :D It´s been fun to experiment with it.
     
  40. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Sounds like a fun project!!! BTW, I believe the second video in our "Working With Shape Keys in Blender" series shows how to perform the functions mentioned above. If you haven't watched the series, are using Blender, and aren't that familiar with shape keys (a.k.a. blend shapes), it might be a good place to start.
     
  41. Ony

    Ony

    Joined:
    Apr 26, 2009
    Posts:
    1,977
    I'm using MegaFiers Morph for all of the morphing on my characters. Is it possible to assign MegaFiers Morph channels to SALSA so it drives their percentages? If not, does anyone have experience with SALSA use alongside MegaFiers?
     
  42. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Great question Ony! We're looking into that. Will get back with you shortly.
     
  43. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    @Ony - A couple of questions to help us understand your scenario:

    1. Are you interested in using our system for lip sync, eye movement, object tracking, facial expressions, or all of these?
    2. What modifiers are you using on your characters eyes, mouth, expressions?
    3. The modifiers you're using, can you control their range with a single value?
    4. If a single value, what is the range of the value (0-100)?

    Thanks
     
  44. Ony

    Ony

    Joined:
    Apr 26, 2009
    Posts:
    1,977
    Thanks for looking into it. :)

    1. Are you interested in using our system for lip sync, eye movement, object tracking, facial expressions, or all of these?

    All of the above.

    2. What modifiers are you using on your characters eyes, mouth, expressions?

    The eyes, head, and object tracking are being controlled from the Unity Head Look Controller but I would just replace that with Salsa and I don't see any reason why that would be a problem. The mouth and expressions are driven by MegaFiers Morph. I have several morph target channels set up with that and just control them as I need them.

    3. The modifiers you're using, can you control their range with a single value?

    Yes, the MegaFiers Morph channels each use a simple percentage value which can be set via script or animation curves.

    4. If a single value, what is the range of the value (0-100)?

    The default range is 0-100 but that can be changed if desired, to any range needed.
     
  45. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Ony, we recently provided our customers who use Autodesk Character Generator models with a work around that can likely also be used for MegaFiers. We created a 1 pixel model with 10 empty blendshapes, you can use this, or make your own, and map SALSA and RandomEyes to this model and it's blank Blendshape indexes. Then all you need to do is copy these BlendShape values to your applicable modifier values.

    To get BlendShape values from a SkinnedMeshRenderer component:
    http://docs.unity3d.com/ScriptReference/SkinnedMeshRenderer.GetBlendShapeWeight.html

    You can download our model using the link at the bottom of this page:
    http://crazyminnowstudio.com/posts/salsa-autodesk-character-generator-workflow/
     
  46. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    Ony, I just want to make sure you're aware that Unity handles these BlendShapes natively as of Unity 4.3 if your model already has them. I'm assuming you're already aware of this and have some higher need for MegaFiers I'm not aware of, but we're just trying to be thorough. If you're model has the BlendShapes already, you could map SALSA and RandomEyes directly to the models SkinnedMeshRender.

    I hope this help.
     
    Last edited: Oct 12, 2014
  47. Ony

    Ony

    Joined:
    Apr 26, 2009
    Posts:
    1,977
    Thanks Crazy Minnow. Based on that info I think I'll go ahead and get the asset. :)

    I use MegaFiers to handle morphs as it's really easy to use, doesn't require blend shapes to be part of the character object, and basically handles morphing (as well as a bunch of other things) smoothly and efficiently. It works for me!
     
  48. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
  49. Ony

    Ony

    Joined:
    Apr 26, 2009
    Posts:
    1,977
    Purchased, downloaded, and imported. I'll start playing with it in a few days and let you know how it goes. Thanks again! :)
     
  50. Crazy-Minnow-Studio

    Crazy-Minnow-Studio

    Joined:
    Mar 22, 2014
    Posts:
    1,399
    We would like to take a moment to thank everyone who has purchased SALSA with RandomEyes, and given us feedback to help make the product what it is today. If you're enjoying the product, we would appreciate a rating or review to help spread the word. As you may have seen, we are in the middle of our first big sale, and our best advertising comes from you.

    Thank You!

    The Crazy Minnow Team