Search Unity

  1. Engage, network and learn at Unite Austin 2017, Oct 3 - 5. Get your ticket today!
    Dismiss Notice
  2. Introducing the Unity Essentials Packs! Find out more.
    Dismiss Notice
  3. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  4. Unity 2017.1 is now released.
    Dismiss Notice
  5. Unity 2017.2 beta is now available for download.
    Dismiss Notice

[RELEASED] Realistic Eye Movements

Discussion in 'Assets and Asset Store' started by Faikus, Feb 12, 2015.

  1. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    647
    I think it has just been my confusion over these settings.... I may have been setting them at odds to one another.

    On the bright side I now realize I can trigger a "look at" by resetting the distance to 0 then a high distance (with Playmaker).

    Thank you for advice! :)
     
  2. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    That's an interesting hack to get your character to look at the player ;-), although you have more direct control over the look behavior if you call LookTargetController's API functions like LookAtPlayer or LookatPoiDirectly. I use them from code, but it should be no problem to call them with PlayMaker.
     
    wetcircuit likes this.
  3. Fletcher-Studios

    Fletcher-Studios

    Joined:
    Sep 25, 2014
    Posts:
    4
    This asset is amazing- it breathes so much life into my characters, its stunning. I have one quick question, the head following component; does that only apply to Mecanim characters, or will it work with Makehuman MHX2 format? Thanks and keep up the incredible work!
     
  4. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    @Fletcher Studios

    Thanks, I'm glad you find the asset helpful.

    Could you elaborate on what you mean with the format? As far as I understand, MHX2 is a format for MakeHuman for importing into Blender. To import MakeHuman characters into Unity you need the fbx format. Are you maybe asking about whether the rig needs to be Humanoid or can be generic? For Mecanim IK the rig needs to be Humanoid, for Final IK it can be generic as well.
     
  5. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Hi everyone, a new version is out! Here are the changes:

    • Fixed slow blinking for blendshapes (blinking when eyes sweep more than 25 degrees)

    • Fixed changing look target when nervousness is set to higher value.

    • Added support for Final IK.

    • Added setting „Keep Target Even When Lost“

    • Split „Max Eye Horiz Angle“ into two angles, one towards the nose and one away from it.

    • Added support for 3rd person perspective games.

    • Blendshapes configurations are now saved by blendshape name, not blendshape index, so you don't have to reconfigure when you modify the number of blendshapes in the 3D modelling software.
    Enjoy!
     
    AcidArrow and wetcircuit like this.
  6. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    4,130
    Awesome, it seems like all my little issues are taken care off.

    I hope Realistic Eye Movements is doing well for you, it's really great.
     
  7. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Thanks!
     
  8. montyfi

    montyfi

    Joined:
    Aug 3, 2012
    Posts:
    523
    Sound interesting. Could you please explain more?
     
  9. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    @montyfi The player-related functionality like "notice player" or "stare back" can now be given a gameobject as reference point. Before, it always used the main camera as the player, so it worked for first-person perspective games only. All you need to do now for a 3rd person perspective game is to have a gameobject parented to the player character's head so that it stays in between the eyes and assign it to the LookTargetController's slot "Player Eye Center".
     
    montyfi likes this.
  10. montyfi

    montyfi

    Joined:
    Aug 3, 2012
    Posts:
    523
    Amazing! Thank you!
     
  11. knchaffin

    knchaffin

    Joined:
    May 18, 2013
    Posts:
    58
    I have been messing with this for a couple of days and can't solve the problem, so I am asking here if anyone has any suggestions. I have eye control working with blinks, all just fine, but I cannot for the life of me get the character's head to move as the eyes move. One complication is that I have a generic skeletal rig on a Mixamo/Fuse model, so I figured that the EyeAndHeadAnimation.cs script is not finding the Head and Spine bones. So, I modified that script and forced it to find the bones as they are named in my rig and verified that the correct bones were found. Still no luck. I can manually change the Head bone transform rotation in the editor and the head moves fine. I have a fairly simple Animator with a blend tree between an idle and a walking animation, which are working fine. I mention this because the idle animation does mess with the Head. I followed wetcircuits advice and made sure that my Animator Base Layer property has the IK pass-through set. I am out of ideas. And yet I am so close! I'm also using the Brows Up blend shape to good effect when the eyes are looking upwards. This also raises the upper eyelids. I love REM so far!

    Thanks
     
  12. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    @knchaffin
    Unfortunately Unity does not yet support inverse kinematics on non-humanoid rigs. Can you switch your Mixamo rig from generic to humanoid? Alternatively, you could use another ik solution like Final-IK.
     
  13. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    647
    I think you might need to change the figure's animation type to Mechanim (in it's import settings)... not sure if it works with Generic.
     
  14. knchaffin

    knchaffin

    Joined:
    May 18, 2013
    Posts:
    58
    Thanks for your advice Faikus and wetcircuit. Either I am confused with Mixamo/Fuse, or something is not working correctly. I just went through the Mixamo rigging and animation steps again and downloaded the Unity format FBX's. I created a new character in Unity based on these FBX files and yet in the Scene Hierarchy view the bones are still labelled as Mixamorig:Head, etc. Should these show up with the Humanoid rig bone names at this point? I reimported the model FBX and made sure it was set to Humanoid, as it was defaulting to Generic. This is where I went awry a couple of weeks ago when I was first working with Mixamo/Fuse and used the default rig import setting (generic). I'm still not sure I am doing this correctly. At least now the FBX import rig type shows Humanoid and the "configuration" shows the correct Mixamorig to Humanoid rig bone names. When REM calls Animator.GetBoneTransform(HumanBodyBones.Head) , does the rig bone names get translated through the Avatar? Obviously, I'm a bit hazy about how Unity handles rigs and bone names. I still have a few hours of work to do to get this Humanoid character to where my Generic character was. So, I can't fully test yet.

    Thanks.
     
  15. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    647
    Ugh, no. I had to do this process 100x when I used Fuse for my spacecamp kids.

    1. In the project folder, select the Fuse figure.
    2. In the inspector go to RIG and change the dropdown to "Humanoid". That makes it Mecanim-able.
    3. go to ANIMATIONS and deselect "Import Animation" (not sure this step is necessary)

    Screen Shot 2016-01-01 at 12.47.34 PM.png

    If the figure has animations attached, you might need to download them separately to use in the Mecanim Animator window....
     
  16. knchaffin

    knchaffin

    Joined:
    May 18, 2013
    Posts:
    58
    Okay, I have my new Fuse/Mixamo character working properly with REM now. The key is to set ALLLL of the FBX assets to the Humanoid rig, as the default is Generic. Since I did not do that initially with my first character, I was never able to successfully redo the skeletal type later without getting a lot of errors. It could have been that I had not set all of the animations to Humanoid though, so the animations were not matching the model rig.

    I still have some things to work on to get my new character back to where my old one was.

    Thanks for your help!

    BTW, I have to extensively play with materials and shaders and textures to get the Fuse/Mixamo characters to look good. For some reason Mixamo defaults to setting a lot of the materials to use Standard Transparent shader, which causes all sorts of disappearing mesh problems. The eyelashes are particularly difficult to get right as I have to create a new material for that and then re-import the Body Diffuse texture to set the alpha transparency correctly to work properly with the body and the eyelashes.
     
    wetcircuit likes this.
  17. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    @knchaffin The bone names in the scene view keep their original names like "mixamorig:Head". Code like Animator.GetBoneTransform(HumanBodyBones.Head) returns the bone that is assigned as head in the Mecanim rig configuration window in the Mapping tab. To import your Mixamo character you don't have to do anything else other than what wetcircuit wrote: just set the rig to Humanoid. Unity finds the correct mapping itself. No need to map bones manually or rename them. Then assign an animator controller with IK pass activated to the animator, drag the LookTargetController and EyeAndHeadAnimator scripts onto the character, and load the Mixamo preset in the EyeAndHeadAnimator component. The whole process takes just a few clicks. You can have a look at the Mixamo character in the demo scene to see one in action.
     
  18. knchaffin

    knchaffin

    Joined:
    May 18, 2013
    Posts:
    58
    Yeah, mine is working fantastically now! I did have to change all of the Mixamo animation FBX imports to Humanoid, one at a time, as I had to do the model FBX. Then I did set the Animator to IK pass-through and dropped in your scripts, designated the eye bones, specified blend shapes for the eyelids control and the magic worked. I still have to go back and set up the blend shapes correctly for the eyelids, eyelashes and brows.

    Thanks for your suggestions.

    So, I was able to start from scratch and in 4 hours I had the new Fuse/Mixamo character working with REM. Most of that time was spent on fixing things that do not work correctly with the Unity shaders by default. I would not want to guess how many weeks or months I have taken in the past to create a fully rigged and animated character with eye and head control! This (Fuse, Mixamo, REM is a fantastic productivity tool suite!
     
  19. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Ah, hadn't seen you already had solved the problem when I wrote my last reply, glad it works for you now!
    One question: you didn't seem to have used the existing preset for Mixamo and instead assigned eye bones and blendshapes manually. Did the presets not have good settings for you?
     
  20. knchaffin

    knchaffin

    Joined:
    May 18, 2013
    Posts:
    58
    No, I used the Mixamo presets for blend shapes and rig. I do have to change quite a lot in the Mixamo produced materials and shaders. I did try some different things to try to get the Generic rigged character working, but that is unnecessary with the Humanoid rig.
     
  21. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    5,600
    I'm working with anime-style characters where the eyes are much bigger than real life.
    [​IMG]

    The eye bones here are set way back in the head — roughly halfway back through the skull. The mesh attached to these is just the iris. So when you rotate these bones a little bit, the iris appear to move much more than you would expect. To give you an idea of how it works, here's what happens when I rotate the right eye bone by 90°:
    [​IMG]
    That's not an earring... that's the iris of her eye!

    As a result, normal methods that would essentially have the eye bone LookAt a target do not work here at all.

    I'm wondering whether/how REM could deal with this situation?
     
  22. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Hi Joe,
    REM would rotate the iris such that it's on the line from the bone origin to the target, so I think it should work.
     
  23. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    5,600
    No, if it does that, then it'll end up rotating it much too far — that's the whole point I was trying to make. So I think it's not going to work.

    But thank you very much for the reply anyway!
     
  24. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Ah, I thought you were asking about whether the irises would look in the correct direction at all, but you meant the maximum angles. The maximum angles are adjustable. The maximum vertical angles are set in the normal setup process (you rotate eyes and eyelids to their maximum looking-up and looking-down positions and save these), and the maximum horizontal angle is just a number that is a parameter of the component. For the anime character a much lower number than the default 35 degrees would have to be set.
     
  25. JoeStrout

    JoeStrout

    Joined:
    Jan 14, 2011
    Posts:
    5,600
    No, it's not about the maximum angles. It's about moving too much. With this rig, if you're just rotating the bone, the irises move too much and point in the wrong direction, because the bone is set too far back in the head for that approach to work.

    I don't know how else to explain it — but I got the answer I needed, so I won't trouble you any further.
     
  26. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Sorry for having misunderstood you. I had another look at your screenshot and you're right, the asset doesn't work for this model.
     
  27. Freddy888

    Freddy888

    Joined:
    Sep 13, 2014
    Posts:
    162
    Hi,

    I am trying to get this to work with a Genesis 3 figure from Daz as in the reviews someone said it would work.

    I have followed the instructions but nothing happens and I get this error :

    NullReferenceException: Object reference not set to an instance of an object
    RealisticEyeMovements.LookTargetController.LateUpdate () (at Assets/RealisticEyeMovements/Scripts/LookTargetController.cs:268)

    The scripts I have attached to the highlighted node, perhaps it is not the right one ?

    Please could you help.
     

    Attached Files:

  28. Freddy888

    Freddy888

    Joined:
    Sep 13, 2014
    Posts:
    162
    So I tried with using mechanim and it gets weirder.

    I have the test model standing on a plane and she goes like this - she sinks into the ground, her arms are clenched and her mouth wide open.
     

    Attached Files:

  29. Freddy888

    Freddy888

    Joined:
    Sep 13, 2014
    Posts:
    162
    Okay I am getting somewhere :)
    I started a new project, I think I was causing problems with having two avatars perhaps.

    So, new project, single avatar. Mechanim bones appear to be working now and also game objects. So far I have done nothing with the eyelids.

    Set up a fresh new animator and did the IK thing to let her head move.

    The concept is working fine, but she is still stuck in this devilish pose and is still sinking into the ground with mouth open. It's pretty freaky to watch, like a zombie trying to get out of the ground !

    Any ideas on how I can stop her going into this strange pose please ?
     
  30. Freddy888

    Freddy888

    Joined:
    Sep 13, 2014
    Posts:
    162
    And finally after some digging, I found if I add an animation to her then she isn't a zombie any more !
    Sorry for the noobiness, but maybe my comments will help someone else :)
    I really like the effect now I have seen it in action - very cool.
     
  31. craigp

    craigp

    Joined:
    Aug 10, 2012
    Posts:
    4
    I thought Faikus might want to know that I really like this library, enough to flat-out recommend it in a video:

     
  32. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Thanks a lot Craig, that's a great video! I'll add in checks for incorrect eyelid bone assignment in a future version. Btw, if your character is a MakeHuman character I think you can make him stand less spread out by correcting the T-Pose in Mecanim: the legs should go straight down in the T-Pose, such that the feet are close together.

    Looking forward to your future videos!
     
  33. craigp

    craigp

    Joined:
    Aug 10, 2012
    Posts:
    4
    Hm! I'll look into that, it's a good tip. The pose is just a neutral stand, but I noticed MakeHuman models are bowlegged when using mocap. That's probably why.
     
  34. Nadan

    Nadan

    Joined:
    Jan 20, 2013
    Posts:
    260
    Hi recently bought the asset and learning to use it. I have some basic Mixamo characters. But I didn't see any bones for the eye lids? If I choose "Mecanim eye bones" and it seems they are not found. Any help?
     
  35. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Hi Nadan, Mecanim doesn't have bones for the eyelids, the eyelids are controlled by blendshapes. You don't actually have to configure that yourself, just import the Mecanim blendshape configuration via the Import button in the EyeAndHeadAnimator component, the preset is in the Presets folder.
     
  36. Nadan

    Nadan

    Joined:
    Jan 20, 2013
    Posts:
    260
    Hi Faikus, and thanks for the support!

    I have a free mixamo character. The rig animation type is set to Humanoid. When I choose Eye Control: Mecanim eye bones. It says "Eye bones not found; is the Mecahim rig set up correctly?". Then I choose "Import" and pick the "Mixamo" file from /presets but it says "Settings don't match target model".

    I have Unity 5.3.4p2 and Realistic Eye Movements 1.9.2.

    Here is the FBX if you want to try: www.ihmeellinen.net/temp/Goblin_D_Shareyko.fbx
     
  37. YourUncleBob

    YourUncleBob

    Joined:
    Jun 11, 2012
    Posts:
    51
    Non-Mecanim Setup

    When using the EyeAndHeadAnimator/LookAtTargetController scripts I'm only seeing motion with the eyes and eyelids. The eyes look great, the blinking looks great, but I don't get any head movement at all.

    For a non-mecanim body, the eye and head animator script has you specify bones for the eyes and eyelids. Looking through the code, it looks like it is trying to infer the bones for headParentXform and eyesRootXform (and maybe some others).

    Should these be exposed to script and setup in the same way as the eyes/eyelids, or should I be placing these components onto one of the character's joints rather than on the root of the character or am I missing something?
     
  38. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Ah, that model does indeed not have bones for the eyes, and the eyes are not separate meshes either. If you want to use that model, you would have to modify it with a 3D program like Blender or 3Ds Max to cut out the eyes into a separate mesh. Most other Mixamo models (like the ones you can create with Mixamo Fuse) have eye bones and separate eye meshes.
     
    Last edited: Apr 13, 2016
  39. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    For head movement you need either to declare the rig in Mecanim as Humanoid or use Final-IK. Is either of that possible in your setup?
     
  40. YourUncleBob

    YourUncleBob

    Joined:
    Jun 11, 2012
    Posts:
    51
    Yeah, I'll use Final-IK, thanks. I'm working on getting that integrated.

    With this setup, it seems that headParentXform does need to be exposed or I get lookatpoints down at my character's feet. My first shot at Final-IK had my characters always staring at the ground. With headParentXform being set, I'm getting closer.
     
  41. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Hi Bob, you're right, there needs to be additional code for the case of non-Mecanim setup to tell the asset where the head is. I'll add it to the next update. In the meantime, you can modify the code yourself: In the Initialize function, replace this code:

    Code (CSharp):
    1.  
    2.    Transform headXform=null;
    3.    if (animator != null)
    4.      headXform = animator.GetBoneTransform(HumanBodyBones.Head);
    5.    if ( headXform == null )
    6.      headXform = transform;
    7.  
    with this:

    Code (CSharp):
    1.  
    2.    Transform headXform=null;
    3.    if (animator != null)
    4.      headXform = animator.GetBoneTransform(HumanBodyBones.Head);
    5.    #if USE_FINAL_IK
    6.      if ( headXform == null && lookAtIK != null )
    7.        headXform = lookAtIK.solver.head.transform;
    8.    #endif
    9.    if ( headXform == null )
    10.      headXform = transform;
    11.  
    Please let me know if that works for you.

    (Btw, with Final-IK, don't forget to set the script execution order as descriped in the docu.)
     
  42. YourUncleBob

    YourUncleBob

    Joined:
    Jun 11, 2012
    Posts:
    51
    Great, thanks. Beats the way I was doing it. I forgot that the Final-IK setup already knew about the head. Everything is looking good now, thanks.
     
  43. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Good to hear, thanks for the info!
     
  44. Selzier

    Selzier

    Joined:
    Sep 23, 2014
    Posts:
    493
    Really nice asset, thanks!! I'm having a bug where the character's spine "spins in circles" when using Final IK. Here's a video showing the problem. Any tips?

     
  45. Slowbud

    Slowbud

    Joined:
    Jul 19, 2015
    Posts:
    27
    I'm using "Look AT IK" and and "Realistic Eye movement" together on at least 2 characters and never had problems. But I saw in the vid you assigned a lot of spines in the "Look AT IK" component. Maybe reduce them?
     
    Selzier likes this.
  46. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Hi Selzier, did you change the script execution order to position the REM scripts after the FinalIK scripts and can you check whether they are still in this order? Also, you can position a dummy object at the place FinalIK is told to look at and see what's happening with it.

    In LateUpdate, after the line
    Code (csharp):
    1. lookAtIK.solver.IKPosition = headTargetPivotXform.TransformPoint( eyeDistanceScale * Vector3.forward );
    place this line:
    Code (csharp):
    1. Utils.PlaceDummyObject("HeadTargetDummy", lookAtIK.solver.IKPosition);
    And see where the created head target object is moving.
     
  47. Selzier

    Selzier

    Joined:
    Sep 23, 2014
    Posts:
    493
    Script Execution Order appears to be correct:
    [​IMG]

    Here's the Dummy Object:

     
  48. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    111
    Thanks for your videos, they are very helpful in understanding the issue. Slowbud is right, there is a problem with the current implementation of REM when the FinalIK chain goes down very deep, there is a feedback mechanism due to where the angles are computed from. I will change that in the next update. For now you can fix it by replacing this line (line number 688):
    Code (CSharp):
    1. headParentXform.parent = spineXform;
    with this line:

    Code (CSharp):
    1. headParentXform.parent = transform;
    You can also let her look more directly at you by setting head weight to 1 in LookAtIk (it was set to 0.5, that's why she didn't look at you when you only had LookAtIk active) and maybe reducing the body weight.
     
    Selzier likes this.
  49. Selzier

    Selzier

    Joined:
    Sep 23, 2014
    Posts:
    493
    Fixed now thanks! Great support Faikus much appreciated. Slowbud nice catch :)
     
  50. cly3d

    cly3d

    Joined:
    Feb 18, 2014
    Posts:
    45
    The picture doesn't do justice to the feeling when wearing an oculus rift and getting up close to her.
    Thanks for an awesome asset.

    Feature request
    I'd really like to request if this could be implemented in a future update:
    - Just as there is a "list' of elements that can be created for "Points of interest"... it would be awesome to have a list of "actions" to perform if you get close to the personal space parameter. Sort of a random list of actions. It's not always that the person shys away.

    Maybe he/she "smiles" (can be done via activating blend shape) or a list of preset animations such as wink, lift hand up etc..
    Being able to build a list and populate it with actions / audio clips would really take this asset a notch up!
    I suppose it can already be done with playmaker and triggers, but the personal space trigger already there just begs for some extra pizzazz:)

    Thanks again. maya_hazelwood.jpg