[RELEASED] Realistic Eye Movements

Do you get any error messages/warnings in the console? Are any other scripts active on the character? Outside of Play mode, in the EyeAndHeadAnimator component, when you click the "Load" button on the "Eyes closed, looking straight" line, are the eyes closed?

 

It is sovled. Sorry late reply

 

Are you planning on adding additional slots for the eyelid? I cannot keyframe the more modern day mocap faces because this plugin only allows one slot for upper and lower.

 

Have you tried parenting the eyelid components to one? For example, parent the right upper eyelid components to the right upper center eyelid, then assign that right upper center eyelid to REM.

 

Yes. And that's not a good solution. Because mocap faces have become so detailed nowadays, that each 4 to 6 bones of the eyelid moves differently from one another, I need to offset them separately.

 

I see. I don't have an ETA for separate slots yet, but I plan to add them in a future update.

 

Is there any way I can set it all the parameters by script? Without using the presets?
I use UMA for my character and external Eyes as game objects and my bone structure is different from the presets
Thanks Ahead

 

If you want to add the components at runtime then you need a preset. If your UMA character is different than what the provided UMA preset assumes, just set up the REM components in the editor for your new character manually once and save a new preset.

 

Thanks for the reply
I’ll do this as soon as I’ll get home.

Is there any way I can deserialize the preset .dat
And after serialize it back after few changes?

 

Yes, check out the ImportFromFile, ExportToFile functions in EyeAndHeadAnimator.cs.

 

Is it possible to only move the eyes with the function like:

Code (CSharp):

1. this.GetComponent<RealisticEyeMovements.EyeAndHeadAnimator>().LookAtAreaAround(HeadTarget.transform);
2.         this.GetComponent<RealisticEyeMovements.EyeAndHeadAnimator>().LookAtSpecificThing(EyeTarget.transform, .075f);

I want the head to look at an offset and the eye to focus on the target.

 

You could lower the head weight slider to something like 0.5.

 

Hm.. so no way to have the eye and head move separate from one another?

 

Not out of the box, but you could modify GetCurrentHeadTargetPos() in EyeAndHeadAnimator.cs.

 

Fantastic asset - thank you!

Having an issue with a custom Fuse model (old lady sitting) and REM.

EyeAndHeadAnimator refuses to accept Mixamo preset file. The error that pops up: Settings don't match target file. See attached image.

Tested REM with standard Fuse characters (the blonde girls in the attached image) and it works fine.

Sent the custom Fuse model to Mixamo and rigged it with facial blendshapes - downloaded as Unity FBX - and it works fine in Unity. But REM doesn't work.

Checklist for issues - humanoid rig, optimized game objs unchecked, didn't import the anim with the rigged character, added anim controller, bone hierarchy matches standard Fuse models, etc.

Simply can't figure out what's going wrong here. Any advice or suggestions would be most appreciated.

 

Your character doesn't have eyelashes, which the preset is looking for as they are morphed as well, so the preset doesn't work on your character. You can just set up the component without preset and even save a new preset in case you have other characters like this. Just look at the settings of the Mixamo model in the demo scene: for closed eyelids, set the blendshapes Blink_Left and Blink_Right to 100, for Looking Up set EyesWide_Left and EyesWide_Right to 100 and Squint_Left and Squint_Right to 40, and for Looking Down set Blink_Left, Blink_Right to 50 and EyesWide_Left, EyesWide_Right to 100.

 

Figured that eyelashes might be the problem - but Maya still has this bloody OBJ export bug - so I had to take the custom model done in Maya into Max and then export the object with combined parts (body, eyes, eyelashes) from Max to Fuse. (The Max obj export functionality is fine for Fuse.) However, Fuse still combined the eyes and eyelashes meshes into one mesh when I uploaded the custom model to Mixamo for rigging. So I finally just deleted the eyelashes and painted them in - she's an old lady, eyelashes are not exactly the selling point, if you know what I mean.

That said, I'll try and create a custom preset with your input on the blendshapes. Thank you for this!!

 

Got it working beautifully! - could actually tweak the blendshape settings a bit more - very cool, somewhat spooky.

 

Is there a more direct way than reverse-engineering these scripts for setting the eyes to closed during runtime - for things like sleep or death? Setting eyeWidenOrSquint to -1 works in the inspector, but setting that during play while also preventing head movement is not working.

 

What is not working for you? You could animate the parameters eyeWidenOrSquint to -1 and headWeight to 0 to achieve the effect. That works at play time as well.

 

That was it. The places I had disabled head movement were preventing eyeWidenOrSquint from being applied. I just removed those and switched to headWeight = 0, and that does the trick. Thank you!

 

I'm having some troubles with the setup of the plugin. What I'm trying to achieve:
- Use Realistic Eye Movements when the character is idle
- Use regular animation clips to control the character's head in certain situations

We are not using a Mecanim humanoid rig, so I set the head transform in the inspector. I made sure that in its default state, the character is looking straight ahead, and the root object (which has the EyeAndHeadAnimator component) forward direction is towards the front:


When letting the script control its head movement in idle mode, everything works as expected. However, when I try to control the head through animation I encounter two problems with this setup, so I guess I'm missing some settings here.

1: When setting the head weight to 0, the character turns its head towards the global Z-axis, instead of the default state:


2: Even with head weight set to 0, the regular animation clips have no effect on the character's head movement.

Any help would be greatly appreciated!

 

When headWeight is 0, REM doesn't do anything with the head. Please try to disable or remove the component during play mode when this happens and see whether the problem persists. If yes, something else is doing that.

 

Disabling the component fixes the problem (the head goes to the animator's position/rotation as expected). Setting AreUpdatedControlledExternally to true has the same effect, so I'm pretty sure REM is still affecting the head even though headWeight is 0.

Simply switching AreUpdatedControlledExternally is not really an option, as the head position will snap to the new position, instead of smoothly lerping.

 

Ah yes, I missed that you set head transform. Can you try replacing line 1513 in EyeAndHeadAnimator.cs, this one:
Quaternion lerpedQ = Quaternion.Slerp(Quaternion.identity, lookQ, ikWeight);
with this:
Quaternion lerpedQ = Quaternion.Slerp(headBoneNonMecanimXform.rotation, lookQ, ikWeight);

 

This fixed it, thanks for the quick reply!

 

I look forward to when that time comes.

 

I have been having problems getting eye/eyelid settings to save in a prefab, specifically the "Looking down" one. Unity is not detecting that the prefab has changed when I hit "Save". I ended up having to save the position, then change another property in the prefab to force Unity to save it. I am on 2019.1.10f1.

 

Thanks for telling me, I'll take a look and fix it in the next update.

 

@Faikus I've just downloaded a character that uses blendshapes for eye movements (left/right/up/down) Is there any chance of this being supported as an additional option under EyeControl? (I know EyelidControl already supports this)
Thanks

 

Not in the next updates; other features have a higher priority. But I do plan to support it eventually.

 

Ok. Thanks.

 

"Adding the Scripts at Runtime. You can add the scripts at runtime (for example if you generate your characters at runtime) and load preset to have their eyes animated correctly."

UWP problem.

I load avatars from web and instantiate them. But ImportFromFile does not exist in UWP.
I can manually change target to windows and load settings. Then change target back to UWP but that is hardcoded way and does not support generating avatars.

EyeAndHeadAnimator eyeAndHeadAnimator = _rightFoot.AddComponent<EyeAndHeadAnimator>();
eyeAndHeadAnimator.ImportFromFile(Application.streamingAssetsPath + "/mypreset.dat");

I tried to make hardcoded function for this, but work only partially.

How I can do this?

 

If your project is IL2CPP, you can actually use serialization in UWP. In EyeAndHeadAnimator.cs, change line 13 to

Code (csharp):

1.  
2. #if !UNITY_WP8 && !UNITY_WP_8_1
3.  

and then you can use ImportFromFile.

 

Hey Your presets file is missing the CC3 scripts when downloading from unity, can you send it asap please, warmest rayopaz@aol.co.uk

 

Hi,

I've attached a screenshot of the preset folder after a fresh import, does yours not look like that?

Here are the presets in any case:
CharacterCreator3.dat
CharacterCreator3_GameBase.dat

 

Thanks Faikus, no they are missing, I uninstalled from the app file on my PC and re downloaded in case it was just re-writing the older version again, it would seem the unity source file is not the one on display on the asset store I'm assuming these files will be enough to get me going. Warmest Ray

 

Ah, it's probably because of the Unity version; the newest update requires Unity 2018.4 or newer. What version are you on in your project?

 

I am having an issue using this asset with characters lying down from CC3, sent to Maya for adding an accessory and positioning in my environment, then to Unity. I have had it working before but recently it is just not working. I am getting an eye turning problem and a heavy twist of the neck offset for an unknown reason. This is more bizarre as I have another character in my scene fully working. Any assistance or test file I can send to take a look into this? Have a deadline fast approaching and need to figure this out our cut a feature.

For reference, I am utilizing CC3.2, Unity 2019.4.1f1, with the preset CC3 Game Base (though I have tested manually doing it as well). The perfect world my pipeline would be export from CC3 for Unity. Take that FBX and bring into Maya to do a lil weight painting I need to do to add the asset, replace it in the folder CC3 exported, and have it all just sync up magically. Animation playback is fine until REM is applied, then the eyes get a weird angle and the head motions are even weirder.

This is a VR project if that matters. This is the view of a character that used to (though this is a new FBX and Prefab but same pipeline) look at you.



This is an example of a character this does it correctly. They were created on two different versions of CC3 as Reallusion forced an upgrade. I am assuming that could be the issue but the issue is only occurring with REM enabled when I send from CC to Maya to Unity



On import to Maya of the FBX I get the below error. REM is the only character-based script/functionality I have tested to have a negative result from this scaling inheritance. Any tips?

 

Can you try putting that character into an empty scene, then adding the REM components and importing the preset, then trying out the buttons "Look up" etc to see if the setting has been imported correctly, then playing the standing idle animation from the demo scene on it? You can also send me a test scene with the problematic character to tamulur@yahoo.com.

 

Hi. I just bought this asset for my UMA characters and have implemented the code as suggested in the readme. I placed UMA.dat in the StreamingAssets folder in my project, and the code appears to try to load it. However, I see this error in the console:

SerializationException: Unable to load type RealisticEyeMovements.EyeAndHeadAnimatorForExport required for deserialization.
System.Runtime.Serialization.ObjectManager.DoFixups () (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize (System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Serialization.Formatters.Binary.__BinaryParser serParser, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream) (at <fb001e01371b4adca20013e0ac763896>:0)
RealisticEyeMovements.EyeAndHeadAnimator.CanImportFromFile (System.String filename) (at Assets/RealisticEyeMovements/Scripts/EyeAndHeadAnimator.cs:362)
RealisticEyeMovements.EyeAndHeadAnimatorEditor.OnInspectorGUI () (at Assets/RealisticEyeMovements/Editor/EyeAndHeadAnimatorEditor.cs:63)
UnityEditor.UIElements.InspectorElement+<>c__DisplayClass58_0.<CreateIMGUIInspectorFromEditor>b__0 () (at /Users/builduser/buildslave/unity/build/Editor/Mono/Inspector/InspectorElement.cs:526)
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr) (at /Users/builduser/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:197)

I also see this if I import at runtime using the button on the Eye and Head Animator component.

I believe this is because I'm using assembly definitions for my project. I can't import REM if it doesn't have one; if I add asmdefs for this asset, the binary preset can no longer be imported.

Is there any way for me to make this work? I really don't want to unwind all my asmdefs, as they have sped up my build enormously.

 

No worries for now. I removed asmdefs for my main game project code and all is well...

 

Can you try with a new UMA preset that you export with your target assembly definition setup in place? Serialization of presets stores info about the types of data with respect to the assemblies used at export time.

 

I think it's a bit of a chicken-and-egg problem. Here's my plan:

With no asmdef for my main code, I can load the UMA preset. I believe I can copy the component to my clipboard while running, exit play mode, add the asmdefs back for my code, apply & run (ignoring the import error that is thrown now), and paste component values in the running component. Then I can export with asmdef info embedded in the binary export file and that should do the trick.
thanks!

 

Maybe it's easier to do a quick test with a non-UMA character first (e.g. one of the demo chars), so the character is there in non-play mode as well. Something like this:
- import the old preset, with no asmdef
- create asmdef
- export preset
- enter play mode, try to import the new preset

 

Hi Faikus,
I'm having some trouble with making the character look at a moving target. It appears the "LookatpoiDirectly()" function is called only once, so the transform that the character looked at has its position set once at doesn't move from where it was at calling time.

In HeadAndEyeAnimator.cs, I've tried putting some kind of positional update into the OnAnimatorIK(), but it doesn't seem to work and I don't think it's a very robust solution anyway.

Would calling the function continuously be better? I don't want to reset the timing though.

What would be the best way to implement "LookatPOIDirectly()" for a moving target?

 

LookAtPOIDirectly does track the transform, even after being called only once. You can see it in the demo scene: in play mode, press the "Sphere" button to let the character look at the sphere (that calls LookAtPOIDirectly only once), then grab the sphere and move it in the scene window: the character tracks it. LookAtPOIDirectly has two forms, one being called with a Vector3, one with a Transform. Maybe you're calling it with a Vector3 as parameter?

 

Ah I see, I thought I remembered it working that way, but this time it's not tracking...
I'm using the LookAt... with the transform. I'm quite curious why there are 2 forms. The programming of this asset is certainly far above my level.

I want the character to look at their own hand after they raise it in front of them. But when I use LookAtPoiDirectly() it just looks at the position where the hand was at the start (by their side), and doesn't look at the raised hand in front of them. I'm telling the character to look at their hand transform, in the rig. Is that my mistake?

 

The hand is probably moved after RealisticEyeMovements has oriented the eyes and head in LateUpdate each frame. Whatever moves the hand must be running after REM's LateUpdate each frame, undoing REM's work (so not the normal animation, which runs before LateUpdate). Are you using FinalIK to move the hand? If not, you can try moving EyeAndHeadAnimator to the end of the ScriptExecutionOrder, so that whatever moves the hand runs before it.

 

Any progress on that front yet? I'm really waiting for this feature.
EDIT - Lol, how did my account change? I guess because I'm using hotmail instead of gmail.