Search Unity

  1. Tune in to get all the latest Unity news live from Berlin on June 19, 6pm CEST. Set up a YouTube reminder!
    Dismiss Notice
  2. Unity 2018.1 has arrived! Read about it here
    Dismiss Notice
  3. Scriptable Render Pipeline improvements, Texture Mipmap Streaming, and more! Check out what we have in store for you in the 2018.2 Beta.
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Learn more.
    Dismiss Notice
  5. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  6. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

[RELEASED] Realistic Eye Movements

Discussion in 'Assets and Asset Store' started by Faikus, Feb 12, 2015.

  1. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    I haven't heard about that before. I use Mixamo characters a lot as well and don't see that problem. Could you check the console, is there an error message?

     
  2. praesidenter

    praesidenter

    Joined:
    Mar 5, 2016
    Posts:
    17
    I can't remember an error message in the console. I'll try to reproduce this bug this evening, but as said before: it appears to happen randomly.
     
  3. praesidenter

    praesidenter

    Joined:
    Mar 5, 2016
    Posts:
    17
    So I managed to capture this on a video. On 0:09 you can see the character looking at the player and right after that the head snaps back to the default position and won't move (the eye movements still work though). No errors in the console.

     
  4. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Thanks for the video. Are there any other scripts active on the character that interact with REM? If you could PM me a test scene that shows that behaviour, I'll find the problem.

     
  5. pan-master

    pan-master

    Joined:
    Nov 11, 2013
    Posts:
    110
    eye blinking is wrong, Human eye blinking is not a liner interpolation between A-B possition. The eye closes faster than openes, The more nervous person is the quicker eye is opening, the more nostalgic person is the longer it takes for the eye to open itself up
    upload_2017-3-4_15-57-3.png
     
    Last edited: Mar 4, 2017
  6. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Hi pan-master, you're right, the blinking can be better. I'll look up some more concrete references and implement their findings in a future update.
     
  7. reallypeople

    reallypeople

    Joined:
    Jul 11, 2013
    Posts:
    21
    Hi, great asset. We had it working perfectly on IOS, but I'm just starting to try an Android build and am getting an error trying to load the REMsettings.dat file from the streamingassets folder, using the exact same code that was working perfectly on IOS. I'm targetting Android 5.0 and building using unity 5.5.2f1

    eyeAndHeadAnimator.ImportFromFile(Application.streamingAssetsPath + "/REMsettings.dat");

    This works fine in the editor but not on a connected Android tablet. Any ideas?
     
  8. BlackboxJason

    BlackboxJason

    Joined:
    Feb 15, 2017
    Posts:
    9
    Hey Faikus, Great work on this. I have been using this asset for a while now but I ran across a weird issue when using it on a character that is walking forwards. It tracks the player object fine for a while but as it moves towards the object the head spins around and looks at the floor behind the character.



    Check out the video to see what I mean.

    I am using Unity 5.5 and there are no other scripts on that object other than REM. This issue does not appear when the animation contains no forward movement.
     
  9. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Thanks for the video. I'm working on an update that has (among other things) better support for when the character moves. Until then, you can add these lines at the end of the function Initialize of EyeAndHeadAnimator.cs:
    Code (csharp):
    1.  
    2.             foreach (Transform t in createdTargetXforms)
    3.                 t.parent = transform;
    4.  
    or you can disable eye animation when starting to move with lookTargetController.ClearLookTarget() and later start it again with lookTargetController.LookAroundIdly().

     
    Last edited: May 11, 2017
  10. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Hi, on Android the contents of StreamingAssets is in the packaged file, so you have to access the folder differently:

    path = "jar:file://" + Application.dataPath + "!/assets/";

    (from https://docs.unity3d.com/Manual/StreamingAssets.html)

     
  11. reallypeople

    reallypeople

    Joined:
    Jul 11, 2013
    Posts:
    21
    Thanks Faikus,

    I added the following:

    if (Application.platform == RuntimePlatform.Android)
    {
    eyeAndHeadAnimator.ImportFromFile("jar:file://" + Application.dataPath + "!/assets/REMsettings.dat");
    }
    else
    {
    eyeAndHeadAnimator.ImportFromFile(Application.streamingAssetsPath + "/REMsettings.dat");
    }

    But it's still failing at ImportFromFile when running on an Android device.

    The page you linked to says that on Android you need to use the www class to load from the streaming assets folder, so is it possible you could provide an example of how to modify ImportFromFile (and/or CanImportFromFile) to achieve this?
     
  12. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Sure, try this:

    Code (csharp):
    1.  
    2. using System.Collections;
    3. using System.IO;
    4.  
    5. IEnumerator LoadPreset(string presetFilename)
    6. {
    7.    string path = "jar:file://" + Application.dataPath + "!/assets/" + presetFilename;
    8.    WWW loadPreset = new WWW(path);
    9.  
    10.    yield return loadPreset;
    11.  
    12.    string newPath = Application.persistentDataPath + "/" + presetFilename;
    13.    File.WriteAllBytes(newPath, loadPreset.bytes);
    14.    eyeAndHeadAnimator.ImportFromFile(newPath);
    15. }
    16.  
    17.  
    Just call this with StartCoroutine(LoadPreset("MyPreset.dat"));


     
  13. reallypeople

    reallypeople

    Joined:
    Jul 11, 2013
    Posts:
    21
    Thanks, that gets me further, but it's still not perfect.

    On the Android device I'm getting a "NullReferenceException: Object reference not set to an instance of an object at RealisticEyeMovements.EyeAndHeadAnimator.SetMicroSaccadeTarget (vector 3 target global)" immediately after the ImportFromFile

    In the Editor, I get a 'SerializationException: serializationStream supports seeking but its length is 0' at the same point.

    Does this give you enough to figure out what's going on?
     
  14. reallypeople

    reallypeople

    Joined:
    Jul 11, 2013
    Posts:
    21
    BTW I can see the dat file being created in the files folder on the device, and it looks the right size (7.52KB) so that bit appears to be working.
     
  15. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    In the editor you would still have to load it the old way, as the Streaming Assets are not in a jar file. So you could do something like this:

    Code (csharp):
    1.  
    2. #if UNITY_EDITOR || UNITY_STANDALONE
    3.    EyeAndHeadAnimator eyeAndHeadAnimator = newCharacterGameObject.AddComponent<EyeAndHeadAnimator>();
    4.    eyeAndHeadAnimator.ImportFromFile(Application.streamingAssetsPath +
    5.    "/mypreset.dat");
    6.    LookTargetController lookTargetController = newCharacterGameObject.AddComponent<LookTargetController>();
    7.    lookTargetController.Initialize();
    8. #else
    9.    StartCoroutine(LoadPreset("mypreset.dat"));
    10. #endif
    11.  
    As to the error on the Android device, maybe your code creates the LookTarget component before the EyeAndHeadAnimator is initalized? Can you try replacing, in the LoadPreset coroutine, the line

    Code (csharp):
    1.  
    2. eyeAndHeadAnimator.ImportFromFile(newPath);
    3.  
    with

    Code (csharp):
    1.  
    2. EyeAndHeadAnimator eyeAndHeadAnimator = gameObject.AddComponent<EyeAndHeadAnimator>();
    3. eyeAndHeadAnimator.ImportFromFile(newPath);
    4. LookTargetController lookTargetController = gameObject.AddComponent<LookTargetController>();
    5. lookTargetController.Initialize();
    6.  
    and delete the code you have in some other place that creates the two components?

     
  16. reallypeople

    reallypeople

    Joined:
    Jul 11, 2013
    Posts:
    21
    Thank you so much. Works perfectly in both the editor and the android build now.
     
  17. AVCVGames

    AVCVGames

    Joined:
    Nov 3, 2013
    Posts:
    3
    Hi, I just purchased REM and tried to implement in my Mixamo/Fuse model, but unfortunately it's been a lost cause. The problem is my Mixamo model's eyes are labeled "default" (which I changed to Eyes) but even worse, the eyes are not separate objects and REM seems to be looking for these as separate objects. When trying to use the Mixamo preset, here's the error I get:

    X_Nurse2 (1): Cannot find path mixamorig:Hips/mixamorig:Spine/mixamorig:Spine1/mixamorig:Spine2/mixamorig:Neck/mixamorig:Head/mixamorig:LeftEye
    UnityEngine.Debug:LogWarning(Object, Object)
    RealisticEyeMovements.Utils:CanGetTransformFromPath(Transform, String) (at Assets/RealisticEyeMovements/Scripts/Utils.cs:25)
    RealisticEyeMovements.EyeRotationLimiter:CanImport(EyeRotationLimiterForExport, Transform) (at Assets/RealisticEyeMovements/Scripts/ControlData.cs:48)
    RealisticEyeMovements.ControlData:CanImport(ControlDataForExport, Transform) (at Assets/RealisticEyeMovements/Scripts/ControlData.cs:747)
    RealisticEyeMovements.EyeAndHeadAnimator:CanImportFromFile(String) (at Assets/RealisticEyeMovements/Scripts/EyeAndHeadAnimator.cs:319)
    RealisticEyeMovements.EyeAndHeadAnimatorEditor:OnInspectorGUI() (at Assets/RealisticEyeMovements/Editor/EyeAndHeadAnimatorEditor.cs:50)
    UnityEditor.DockArea:OnGUI()

    Here's a screenshot of my model's tree:
    REM_Mixissue.jpg
    I'm not sure if there was some setting I should've checked or missed in Fuse or not, but I could use some help here. I also made sure IK was uncheked, but that doesn't seem to have any effect.
     
  18. wetcircuit

    wetcircuit

    Joined:
    Jul 17, 2012
    Posts:
    752
    R.E.M. works with Fuse figures.

    Make sure your figure's rig is set to Humanoid (import settings), and the Mecanim Animator Controller is set to "IK Pass" (see blue checkmark in this pic)

    Screen Shot 2017-08-16 at 10.44.57 PM.png

    Here's my screen cap from very early in this forum (page 1 or 2) for my Fuse figures. The REM options may have changed but the basics are still there… Most important is to set:
    Eye Control: MECANIM EYE BONES
    Eyelid Control: BLENDSHAPES



    Good luck!
     
  19. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Hi,

    REM doesn't look for separate eye meshes, but for eye bones. In addition to what wetcircuit said about the Humanoid rig, in that same import tab (where you choose between Humanoid and Generic rig), make sure that either "Optimize Game Objects" is unchecked (otherwise it removes bones like eye bones from the game object) or if you want to keep it checked, expose the eye bones.

    If the bones still don't appear when you uncheck Optimize Game Objects, then this Mixamo figure wasn't exported with eye bones. I think there is a checkbox in Mixamo Fuse where you can set the number of bones to export, maybe look for that. Otherwise, what you could do is to separate the eye meshes in a 3D program and set REM's eye control to eye game objects instead of Mecanim eye bones. Or you just disable the eye mesh and add eyes from a Unity asset like Eye Advanced.

     
    Last edited: Aug 17, 2017
  20. pjdlh

    pjdlh

    Joined:
    Aug 29, 2017
    Posts:
    1
    Hi,

    Great asset. Had it working well in another project then updated from asset store last night and imported into project but cant see the Head Transform control in either script so nowhere to reference head and as a result, not getting any head movement...how do I fix this?
     
  21. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    The head transform field appears if your character doesn't have an animator or if the rig doesn't have a head bone. Did you add an animator to your character?

     
  22. Alvarezmd90

    Alvarezmd90

    Joined:
    Jul 21, 2016
    Posts:
    44
    I have an issue which I think is able to be solved some way. It's just haven't managed to figure it out.

    I'm using a LookAtIk component from FINAL IK and it is also moves the body to follow the player. Only then it causes the npc to shake like crazy. When I set the slider of body weight to zero it works normally again. Why is that? I have another scripts attached that does this in the update:

    eyeAndHeadAnimator.Update1();
    lookAtIK.solver.Update();
    eyeAndHeadAnimator.Update2();
     
  23. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    If you just use LookAtIK and don't need complicated updates between different FinalIK components, you don't need the Update1/Update2 thing (it should work anyway of course, maybe there is something else in your setup that causes the problem), just add a LookAtIK component, set the desired spine objects and set the head/body sliders, and you don't need to call any update functions. Is there a reason you manually call the update functions?

     
  24. betaFlux

    betaFlux

    Joined:
    Jan 7, 2013
    Posts:
    34
    Greetings. Does anyone know which variable of the REM look target controller can be used to change the LookAt position dynamically? I've tried "targetPoi" but that does nothing.
     
  25. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    If you want to switch the target from one object or position to another via code you can call a method like LookAtPOIDirectly (see the API). If you want to move the target around (without switching between targets), just assign an object (can be an empty GameObject) to the POI list and move that. Let me know if you need any more info.

     
  26. betaFlux

    betaFlux

    Joined:
    Jan 7, 2013
    Posts:
    34
    Thanks for the hint, though if I use 'LookAtPOIDirectly(dynamicTargetTransform)' nothing happens. I guess this is because I need to add the dynamicTargetTransform to the POI list, right? It would be nice if you could add a method to make the look target controller work on any Transform/Vector3 without the need to add it to a list first.
     
  27. id0

    id0

    Joined:
    Nov 23, 2012
    Posts:
    113
    I made it like that:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using RealisticEyeMovements;
    5.  
    6. public class InteractivePoints:MonoBehaviour{
    7.  
    8.     public float gizmoOffset = 2;
    9.     public float pointGizmoOffset = 0;
    10.  
    11.     public Transform[] points;
    12.     public LookTargetController playerLook;
    13.  
    14.     void OnTriggerEnter(Collider other){
    15.  
    16.         if(other.CompareTag("Player")){
    17.  
    18.             other.GetComponent<LookTargetController>().pointsOfInterest = points;
    19.         }
    20.     }
    21.  
    22.     //___________Trigger Icon____________
    23.  
    24.     void OnDrawGizmos(){
    25.  
    26.         Gizmos.DrawIcon(new Vector3(transform.position.x, transform.position.y + gizmoOffset, transform.position.z), "ITrigger.png");
    27.  
    28.         foreach(Transform point in points){
    29.            
    30.             Gizmos.DrawIcon(new Vector3(point.transform.position.x, point.transform.position.y + pointGizmoOffset, point.transform.position.z), "LookPoint.png");
    31.         }
    32.     }
    33.  
    34.  
    35.     void OnTriggerExit(Collider other){
    36.  
    37.         if(other.CompareTag("Player")){
    38.  
    39.             other.BroadcastMessage("ChangePoints", SendMessageOptions.DontRequireReceiver);
    40.         }
    41.     }
    42.  
    43.     public void ChangePoints(){
    44.        
    45.         playerLook.pointsOfInterest = points;
    46.     }
    47. }
    48.  
     
  28. betaFlux

    betaFlux

    Joined:
    Jan 7, 2013
    Posts:
    34
    @id0 Thanks but that's still overkill for my use case. For now I'll just stick with Unity's "Head Look Controller" as it's a single script with a simple 'target' variable. Still, REM will always be one of my favourite assets when it comes to human behaviour simulation.
     
  29. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    You don't need to add it to the list, the method should make the character look directly at the transform or position specified. In the demo scene, that method is what makes the character look at an object when you press the corresponding button. If you send me a screenshot of your setup (so I can see the two components LookTargetController and EyeAndHeadAnimator in the inspector and the values they have), I hopefully will be able to tell you what's going on. Also, do you get an error message in the console?

     
  30. betaFlux

    betaFlux

    Joined:
    Jan 7, 2013
    Posts:
    34
    @Faikus Sorry, the error was on my side. :-/ It took some time until I realised I had another IK script overriding the head movement. Nevertheless, thanks for the support!
     
  31. Dev-A

    Dev-A

    Joined:
    Oct 11, 2013
    Posts:
    28
    Any updates, progression, direction with the asset?
     
  32. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Yes, there will be new features in the next update like more control over the eyelids. I don't have a concrete timeline yet, but as soon as my client projects will leave me time, I'll finalize it. Stay tuned!

     
    Dev-A likes this.
  33. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    Hi,

    I'm having issues. I'm trying to set up REM with FinalIK look at but my character looks at the floor:

    REM_Issue_Capture.PNG
     
  34. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Did you uncomment the line "// define USE_FINAL_IK" at the top of EyeAndHeadAnimator.cs?

    What Final-IK components do you have on your character?

     
  35. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    Hi, yes I did uncomment that line.

    I just use look at ik from final ik, however I have salsa on there too and my character is an UMA so I followed your instructions in the manual for adding the components at runtime.
     
  36. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    I've just set up UMA with FinalIK's LookAtIK, this code works fine if you call StartCoroutine(AddREM(newGO)) with the generated character as newGO:

    Code (csharp):
    1.  
    2.        IEnumerator AddREM(GameObject newGO)
    3.        {
    4.            yield return null;
    5.            yield return null;
    6.  
    7.            Transform[] spines = {RealisticEyeMovements.Utils.FindChildInHierarchy(newGO, "Spine").transform,
    8.                                            RealisticEyeMovements.Utils.FindChildInHierarchy(newGO, "Spine1").transform };
    9.  
    10.            RootMotion.FinalIK.LookAtIK lookIK = newGO.AddComponent<RootMotion.FinalIK.LookAtIK>();
    11.            lookIK.solver.SetChain(   spines,
    12.                                                RealisticEyeMovements.Utils.FindChildInHierarchy(newGO, "Head").transform,
    13.                                                null,
    14.                                                newGO.transform);
    15.  
    16.            RealisticEyeMovements.EyeAndHeadAnimator eyeAndHeadAnimator = newGO.AddComponent<RealisticEyeMovements.EyeAndHeadAnimator>();
    17.            eyeAndHeadAnimator.ImportFromFile(Application.streamingAssetsPath + "/UMA.dat");
    18.  
    19.            RealisticEyeMovements.LookTargetController lookTargetController = newGO.AddComponent<RealisticEyeMovements.LookTargetController>();
    20.            lookTargetController.Initialize();
    21.        }
    22.  
    23.  
    Can you try that and compare with your code and tell me if there still is a problem?
     
  37. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    Hi, that works perfectly now. I was using the UMA create bones feature and adding final IK to the prefab, so perhaps this is why.

    Thanks for your help.
     
  38. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    Whoops, I spoke too soon. Once in my main scene and spawned in I get this error on each character:

     
    Last edited: Apr 19, 2018
  39. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    It seems animator.GetBoneTransform(HumanBodyBones.LeftEye) returns null, so either there are not eye bones or the animator doesn't have eye bones assigned to the avatar. Is there anything about your setup that could cause this?
     
  40. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    I removed all the bones created for the UMA and then added a 2 second delay to adding the REM scripts and it works without the errors but now the characters are not looking directly at me, they're looking to the side of me. Strange.
     
  41. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Does the head movement work correctly if you just use FinalIK without REM? Also, can you send me a test project that shows the problem?
     
  42. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    It seems my male UMA characters have the issues, the females are fine.

    Here's the male with issues:


    Here's the female which is correct (ignore the male voice, not recorded female voices yet)
     
  43. Faikus

    Faikus

    Joined:
    Jan 3, 2011
    Posts:
    121
    Thanks for the videos. That is weird. Maybe something about the UMA setup for male and female is different. It would be helpful if you could send me a small test project that has the problem.
     
  44. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    1,829
    Thanks, I'll see what I can do.