[RELEASED] Realistic Eye Movements

I'm having a weird problem. I am using the 03n "stunner jane" character for my testing, and the most current 2020 LTE version of unity.

When I first set things up according to the quickstart (add the eye and head animator and look target controller scripts, import the 03n_uma preset), everything works great. The eye and head movements work in the editor and in builds.

However, when I quit unity and came back this morning, it fails to work - runs immediately stop with errors like "eye bone not found", and the eye and head animator window says "please complete setup". So, I imported the preset again, and again it works ... until I quit and reopen my project.

Am I missing saving something somewhere?

EDIT: Nope, I'm wrong, importing the preset doesn't ALWAYS work. I had to delete both scripts, save the project, and then reopen and add them back to get it working again. When I got the error on reopen and tried importing the preset, it says "error, settings don't match target model". And there's errors in the console that it can't find the eyes.

And another EDIT: I was wrong about it working in builds - it only seems to ever work in the editor's runtime mode, right after adding the scripts. In fact, if I do a build, when the build completes, I'm back to getting the errors again.

UMAs are a little weird in that their bones don't really exist until runtime, even though you can see them in the editor, so I made sure I wasn't doing anything with the scripts until Uma triggered the "enabled" message. So I'm not sure what to do.

Last Edit for now: I've been playing with this a lot to see exactly what's going on. The eyes definitely do NOT move in builds, but work perfectly in editor play mode. And either leaving the editor, or doing a build, makes the "eye bones not" found errors trigger, and I haven't found any solution other than do delete both scripts and then add them back. Which is unfortunate, because I have everything working just like I want in the settings.

 

Please add these lines to Scripts/EyeAndHeadAnimator.cs, after line 847:

Code (CSharp):

1. if ( controlData.eyeControl == ControlData.EyeControl.MecanimEyeBones && animator != null && false == animator.isHuman )
2.     return;
3.  

This will delay initialization until the animator is set to human. Does this fix your issue?

 

That didn't work, or rather it did in a way - it outright stopped the eye movements from happening at all, but no errors were thrown, so at least I didn't have to delete anything to proceed.

I had a feeling the problem was with UMA initialization - there's an event, onEnable, which is the all clear for doing anything UMA related in application code. Since Initialize() in EyeAndHeadAnimator is called in its Awake method, it's fired off too early, so it can't find the bones (they don't exist yet, because UMA takes quite a while to get going).

So, I had an idea - I commented out all the calls to Initialize() in both files, and then added a call to it (along with the "look randomly" code from the end of Awake) to the end of my OnEnable handler. That works, both in the editor and in builds!

It seems like requiring users to call Initialize() themselves is a pretty small price to pay (since they'll know a safe place to put it), rather than having lots of calls to Initialize all over the code that simply call and return. I'm going to leave it like this for now since it works great, but if you come up with something better in a subsequent release, that's fine too!

Minor problems aside (thanks for responding so quickly!), this is a really nice product - it's eerie how much better it makes things look, with very little effort.

 

Hi i used to use this product fine but i remember i also used to have this message but i found the solution somewhere and i dont remember it. What am i supposed to do when i get this message?

Cannot import, eye control set to Mecanim eyes, but left eye not found.
UnityEngine.Debug:LogError (object,UnityEngine.Object)
RealisticEyeMovements.ControlData:CanImport (RealisticEyeMovements.ControlDataForSerialization,UnityEngine.Transform,UnityEngine.Transform) (at Assets/RealisticEyeMovements/Scripts/Internal/ControlData.cs:200)
RealisticEyeMovements.EyeAndHeadAnimator:CanImportFromFile (string) (at Assets/RealisticEyeMovements/Scripts/EyeAndHeadAnimator.cs:308)
RealisticEyeMovements.EyeAndHeadAnimatorEditorrawImportExportButtons () (at Assets/RealisticEyeMovements/Scripts/Editor/EyeAndHeadAnimatorEditor.cs:255)
RealisticEyeMovements.EyeAndHeadAnimatorEditor:OnInspectorGUI () (at Assets/RealisticEyeMovements/Scripts/Editor/EyeAndHeadAnimatorEditor.cs:617)
UnityEngine.GUIUtilityrocessEvent (int,intptr,bool&)

I get this after i push import and select the DAZ.json file for easy setup for my Genesis 8 figure. in the video tutorial they have no problems at all, but i get all this errors

 

This means the character rig is not set to Humanoid in the import settings.

 

Hello, I haven't found a way to make it properly work with an animated head (I got weird neck twists) and switching the lookat target relative to an event.
Has anyone a clue for these to work?
Thanks in advance

 

Using the asset on an animated character and switching look targets is how most people use the asset. If something doesn't work for you, please provide more information about your setup. Do you use a humanoid Mecanim rig? What is the head control set to? Try setting it to AnimatorIK and make sure the animator controller has the IK pass enabled.

 

Thanks for your response. As I was using a timeline I used a generic rig, but now it's better by switching to an Humanoid. Last question however: how do I assign different lookats targets and do I animate this? Do you know if it's possible to do that in a timeline?

 

You can use signals. You can call functions on LookTargetController directly, like LookAtPlayer(), or if you want to use a specific parameter, you can add a script on the character like this:

Code (CSharp):

1. public class LookSignalReceiver : MonoBehaviour
2. {
3.     [SerializeField] Transform lookTarget;
4.  
5.     public void LookAtTarget()
6.     {
7.         GetComponent<RealisticEyeMovements.LookTargetController>().LookAtPoiDirectly(lookTarget);
8.     }
9. }
10.  

You can then assign the target in the scene to the lookTarget parameter and let the signal call LookAtTarget() on this LookSignalReceiver component.

 

Will try that. Thanks again

 

To begin with i must say a huge thank you for the amazing asset, it has already helped me out many times
Now I'm working on a project on Universal Windows Platform and only on it do i get this errors:

Code (CSharp):

1. Assets\RealisticEyeMovements\Scripts\Eye And Lead Animator.cs(304,26): error CS0165: Use of unassigned local variable 'import'
2. Assets\RealisticEyeMovements\Scripts\Eye And Lead Animator.cs(827,11): error CS0165: Use of unassigned local variable 'import'

Can you help me figure it out?

Unity 2021.2.8f1 and 2022.1.0b3

 

In file Scripts/EyeAndHeadAnimator.cs, please replace lines 289 and 812 with this:

Code (CSharp):

1. EyeAndHeadAnimatorForSerialization import = null;

 

Big Thanks! It works

 

hey dev, this asset is amazing, 11/10 score!!

please, how do I extend your base Eye And Head Animator or edit it, to add some custom features?

I want to add a simple boolean toggle to disable eyelid blinking but still keep weight to control eye widen or have a on command blink. I would also like to be able to re-enable blinking on the fly during runtime using it.

any help appreciated

 

One quick way without changing code would be to simply set the parameters minNextBlinkTime and maxNextBlinkTime in EyeAndHeadAnimator to some very high values like 100000 and then call EyeAndHeadAnimator.ResetBlinking() to apply them immediately (otherwise they will be applied after the next blink).

Another way is to modify Scripts/Internal/BlinkingComponent.cs, this code at line 96:

Code (CSharp):

1. if ( blinkState == BlinkState.Idle )
2. {
3.     timeSinceLastBlinkFinished += deltaTime;
4.     timeTillNextBlink -= deltaTime;
5.     if ( timeTillNextBlink <= 0 )
6.         Blink();
7.  
8.     return;
9. }
10.  

For example, you could add a boolean variable IsAutomaticBlinkingEnabled to EyeAndHeadAnimator and replace the second if statement with something like:

Code (CSharp):

1. if ( timeTillNextBlink <= 0 && eyeAndHeadAnimator.IsAutomaticBlinkingEnabled )

You can always trigger a single blink by calling eyeAndHeadAnimator.Blink().

 

thanks!!!!

 

Im using Daz characters, and the eye movement and follow player are working great. However it seems to cancel out my facial, and upper body animations.

 

REM doesn't modify blendshapes for DAZ characters or upper body parts like arms, so something else must be going on in your scene. What drives your face animations? If you want you can also send me a link to a test scene via private message.

 

@Faikus ,Tore, big problem. Sorry going on further in German, using your Eyemovement since beginning. Suddenly Eyes don*t look in direction where "Sightline" shows. If interestet, I try to translate the answers,

Situation: In HDRP: Ich mache das Setup, aber die NPCs schauen immer ca. 10° rechts und ca. 5° nach unten an mir vorbei (VR). Ich musste wg. HDRP ein anderes Augenmodell nehmen und dachte erst es liegt daran. Habe nun eine Möglichkeit gefunden, meine alten NPC Augen auch in HDRP zu nutzen. Nun witzig, die Figur, die mit dem neuen Funktionierte, funktioniert mit den alten Augen nicht. Neu Importierte Figuren, mit den Non HDRP Augen funktionieren gut. Ich finde einfach keine Ansatz, was das Problem sein könnte. Habe wirklich nun 2 Tage rum experimentiert. Komplette Setups neu durchlaufen, Headwinkel vorher auf 0 gestellt. NIX. Die NPCs schauen an mir vorbei. Irgendeinen Ansatz?
Nachtrag zur weiteren Klassifizierung: Kopf und Augen sind anders ausgerichtet als die einblendbare "Sightline", die stimmt.

 

Hi Bernd

I'll give a short answer in English here for those who have a similar problem. We can continue the conversation in German per mail, to keep this thread English and easier to read for others.

If the eyes don't seem to look at the target directly even though they are oriented straight ahead at scene start, here are some points to check:

Are the eyeballs sitting in the right position, that is, the center of the eye socket?
Are the eyeball pivots in the center of the respective eyeball? Outside of play mode, select the eye and rotate it around its local axis and see whether it rotates correctly.
Are any other scripts or components active on the character that might be influencing eye or head movement?

In general it's best to start checking in the editor whether the eye's forward axis is indeed not looking where it should: sometimes the position of the eye in the eye socket makes it look like the character is looking somewhere else.

 

In my VR game's opening scene, I want my main NPC (that I have REM on), to look at Katana, then look at the player, then back to the Katana, then finally back to the player. I'm using Pixel Crushers' dialogue system, so I'm using Unity Events to enable/disable empty objects (Targets) to change what she's looking at. I have an empty object attached to the Katana, and another one attached to my Center Eye. I have the Third Person Player Eye Center set to the object attached to my Center Eye. I'm then using the bool on setActive between the two objects to change what she's looking at depending on where we are in the dialogue. It just doesn't seem to want to behave.

She looks at the Katana in the beginning (when that object's Target is set to true, and the Center Eye's Target is set to false), she then perfectly turns her attention to the player when I switch that around. However, when I try to change it back to her looking at the Katana, she continues looking at the player.

I also used the script above (LookSignalReceiver). Is there a way to change the target for the Look Target Controller using Unity Events easily? Below I have the ChangeLookTarget, which is the script I think you called LookSignalReceiver. Is there a way to have a tranform option?

 

REM only checks if a POI is enabled when deciding every few seconds where to look next: in the POI list, only enabled POIs are considered as possible new look targets. So disabling a POI can take a few seconds to make the character look at another one.

It's better to call one of REM's look functions directly. In your case, just let the event call your script ChangeLookTarget's LookAtTarget function, instead of toggling its enabled status.

 

When my player exit from the Notice Player Distance, the character still looks at my player (even after waiting Max Look Time seconds)
Why ?
How to change that ?

 

I cannot reproduce that. Is it possible that your character has LookAtPlayerRatio set to some non-zero value and that once you exit the notice player distance and the character's looking at player time is up, they happen to choose to look at the player? What happens if you set LookAtPlayerRatio to zero and then repeat the experiment?

 

Hey guys!

Trying to use this asset with a character from Character Creator 3 but when I try to import a preset via EyeAndHeadAnimator component I get the Cannot import, blendshape not found: Eye_Blink error. I am new to Character Creator 3, so I don't really know how should I export model to Unity so that I have this blendshape. Could please anyone help me on this one?

 

Maybe the name of the main mesh was changed. The body SkinnedMeshRenderer should be called "CC_Base_Body" for REM to find the blendshapes.

 

Do you mean the mesh in the file itself or in the hierarchy? The one in the hierarchy is names the way you said.

 

If you click on Derek's CC_Base_Body, and in the inspector panel check the blendshapes, does he have blendshapes like Eye_Blink?

The meshes in your "PlayerArmature" character have non-standard CC3 names, so I assume this is not a CC3 character and you are only trying to apply the CC3 preset to your Derek character and that's where you are getting the error?

 

Yes, I do have "Eye_Blink" blendshape.
My problem was that I attached the EyeAndHeadAnimator component to the parent gameObject, no to the model root.
Thank you very much for your message, you helped my to understand my mistake.

 

@Faikus I just upgraded from Character Creator 3 to Character Creator 4 (just release a week ago). I have some error related to the eye_blink when using the CC3 preset (please see attached picture. I am using Unity 20202.3.27f1. Any help would be greatly appreciated. Thank you for your help.

 

Here's the preset for CC4: link

 

@Faikus That's is fantastic! I wasn't expecting a reply and solution so quickly. Very useful asset with amazing support!

Thank you so much!

 

Hi there,

I have a character based on a CC3 model which has skinned mesh clothing pieces that cover the face (inner and outer mask meshes and a cowl that covers the head). These share the body's armature and normal animations through an Animator on the overall model move this clothing as you would expect, but when I add REM and allow it to add head movements, these pieces of clothing stay still when the head moves. How can I fix this?

Thanks!

 

That is weird, all REM does is move the head bone, so anything skinned to that bone should move with it. In my projects with skinned clothing on CC3 characters covering the head it works fine. In Edit mode, when you select the head bone and rotate it in different directions, does the cloth move with it correctly?

 

Hmm, no it doesn't - it moves a bit, but mostly it just stretches and distorts. I think the reason it only appeared to be happening with REM is simply that the idle animation I am testing with doesn't involve very much head movement, so the fact the cloth follows the head "a bit" was enough.

I guess the model as exported from Character Creator must have dodgy bone weights in that part of the suit - that seems entirely possible since that is definitely happening with the gloves and boots. These assets originally came from DAZ - the import and conversion into a CC3 character seems to be a slightly-less-than-perfect process...

Anyway, not REM's problem - thanks for getting back to me so quickly.

 

I'm having trouble getting the Animator IK mode working.

My avatars are all set to Humanoid, and I have checked IK Pass in their animators.

The avatars will look at the player if I set Head to use Transform, but not if it's set to Animator IK.

I can't use Transform because I am also using Dynamic Bones and the bones and colliders seem to update before the Transform update occurs, so they don't follow the head precisely. And I'd like the head movement to blend some with the animation anyway.

[Edit]

Changing the script execution order so Dynamic Bone is executed towards the end seems to have solved the problem of the bones not following the head transform, but I'd still like to know how to get the Animator IK mode working so I can have more dynamic head motion.

 

Another issue I've noticed is the characters won't tilt their heads to look up at the player. For example, a character which is sitting and looking down won't raise their head much to look up at you when you approach and are standing over them. This isn't an issue with the weighting either because I tried setting both head and neck up/down weights to 1.0 and that doesn't seem to help much. Their chest being slightly tilted forward probably doesn't help, but people can definitely crane their necks a little more than to just look straight ahead. I assume I can adjust this somewhere in the source code, but it would be better as a default parameter you can adjust in the editor like the eye look. Unless I missed that the eye look settings will also record head and neck rotation if you adjust those as you modify the values?

 

Can you test whether OnAnimatorIK works at all on your character with this simple script:

Code (CSharp):

1. using UnityEngine;
2.  
3. public class TestAnimatorIK : MonoBehaviour
4. {
5.     [SerializeField] Transform target;
6.  
7.     Animator animator;
8.  
9.     void Awake()
10.     {
11.         animator = GetComponent<Animator>();
12.     }
13.  
14.     void OnAnimatorIK(int layerIndex)
15.     {
16.         animator.SetLookAtPosition(target.position);
17.         animator.SetLookAtWeight(1, 0.2f, 1, 0, 0);
18.     }
19. }
20.  

Just put the script on your character and assign an object to the target field. When you press play, does the character's head keep turning towards the target?

 

Exposing the parameters determining whether the subject can get something into view is a good idea, I'll do it in a future update. For now, you could adjust kMaxVertHeadAngle in Scripts/Internal/HeadComponent.cs, line 22.

 

I'll have to test that other script you sent me later. But regarding this, I found that parameter and adjusting it seems to work. But I have found a small bug.

If I've manually set the head/chest rotation for an avatar in the inspector, rather than doing it in an animation (because I was lazy), the location the character looks at when trying to look at the player will be incorrect.

For example, if a character is sitting on a bed and and leaning back against the headboard and I tilt their head up manually in the editor so they're looking forward, when the player approaches, they'll look down instead of at them. I assume because of how the angles are being calculated and set without regard to what the actual forward direction of the head, neck, and chest bones are.

Of course, I haven't yet tested this with the pose being set via an animation, so it's entirely possible this is just a bug in general and my setting the angle in the inspector isn't actually the issue, which would be bad. I'll have to test that later as well.

 

As stated in the manual, that head has to look straight ahead with respect to the character transform when the scene starts. Once the scene plays, the character can be moved by animations etc.

 

[Update] For anybody having this issue the cause is it wont save poses on a prefab, you have to unpack your prefab, save poses and save your prefab again.

Hello, I am having an issue where the saved pose for the eyelids rotation always get lost if I reload the scene or play the game.

https://www.dropbox.com/s/d9yic7kqyjn5oid/Eyelids-Issue.mp4?dl=0

Here in the video you can see it happening, here is what I am doing:
1- First showing the open, look down and look up poses
2- I then load the look down pose and rotate the upper eyelids, then saving.
3- I now verify the rotation was saved
4- Play the game and notice the eyelids are not following
5- Stop the game and check the saved positions, the eyelid rotation is gone.

I hope you can help me with this.
Thank you.

 

As Hazneliel told me via email, this happens on prefabs. He solved it for now by unpacking the prefab, doing the change, then recreating the prefab. I'll look into this for the next update. Thanks for the info!

 

Hello,

I'm in search of a way to have the eyes follow an object with nearly zero lag follow time, so basically locked to the target. What would I need to adust to make that happen?

Thank you!

 

In Scripts/EyeAndHeadAnimator.cs you can find this code from line 1629:

Code (CSharp):

1. //*** After the eyes saccaded to the new POI, adjust eye duration and speed so they keep tracking the target quickly enough.
2. {
3.      const float kEyeDurationForTracking = 0.005f;
4.      const float kEyeMaxSpeedForTracking = 600;
5.  

You can adjust these two constants to make tracking faster, by reducing the duration kEyeDurationForTracking and increasing the max speed kEyeMaxSpeedForTracking.

 

Hey Thanks for the response and code to adjust. Will give this a go

 

Hi,

When I import this to my project I get immediately errors... I did nothing except imported the latest package. My Unity: 2021.3.15f1.

Assets\RealisticEyeMovements\Scripts\EyeAndHeadAnimator.cs(1549,22): error CS1729: 'Ray' does not contain a constructor that takes 2 arguments
Assets\RealisticEyeMovements\Scripts\EyeAndHeadAnimator.cs(1550,23): error CS1729: 'Ray' does not contain a constructor that takes 2 arguments
Assets\RealisticEyeMovements\Scripts\EyeAndHeadAnimator.cs(1551,27): error CS1729: 'Ray' does not contain a constructor that takes 2 arguments

 

Right... I commented the offensive lines out, and it works So... what did I possible broke in the process?

 

It seems you redefined "Ray" somewhere in your project. Ray is originally a struct defined in the Unity engine, so you might eventually get more errors in other assets that use Unity Ray if you leave your redefinition in. I suggest to wrap your own redefined Ray in a namespace.

For RealisticEyeMovements, you can keep the lines commented out. You didn't break anything, the variables that get updated in these lines are just for convenience for anybody who is interested to get the current eye directions. If you don't use them, you can just keep the lines commented out. Or you can add "using Ray = UnityEngine.Ray;" at the top to the other "using" statements.

But as I said, I would wrap the redefined Ray in a namespace, because eventually you can run into the same problem in code from other assets. You can find out where the redefinition happened by putting your cursor into the Ray word in the file, and then pressing F12.

 

You are right... I do have "Ray" somewhere else in my project. You know my project better than I do I'll fix that...

Amazing product by the way... I'm using it with DazToUnity, works like a charm!