Video Any Eye tracking software solution?

Hello and good day!

I'd like to know if in 2020 we have a tangible solution for eye tracking using a web camera. Specifically I'd like to know if there is a solution compatible with Unity that does the following:

-Use the front camera of an android/ios device
-Detect where the user looks in the screen aka get the x,y coordinates of the user's eyes on the screen
-no external hardware is used (purely software based)

Thank you very much for your time.

 

Hi Machael,
, you piqued my interest because I looked at doing eyetracking with Unity using an X-Box Kinect in 2012 for a research project I was awarded, but never got very far, partly due to the fact that I was just learning Unity.

First to qualify my comments, I just left my position teaching HCI and managing an eye-tracking lab at a University in the UAE last June (long story). And, in my graduate work in 2010 I was tasked with creating an eye-tracking system using a web cam and open source, OpenGaze, software for a study on autistic children (the system I created was not a viable solution - my job was to find out if it could be done).

So, I have not kept up on the state of eyetracking for over a year now, and because we used dedicated eye-tracking Tobiii systems in our labs, I never followed the advances in using mobile cameras to capture eye movements that closely.

I do know that in 2016 there was a paper that came out that got mobile camera eye-tracking down to 1cm accuracy, using machine learning. 1cm is a lot of space on a mobile screen, so that is not very accurate! Probably not accurate enough to control a game, if that is what you are after. If that is what you want, I think it would be best to have system level integration where, so that Unity would get input without having to know about the device or needing a plugin, which was one of the problems I ran into in 2012.

After seeing your comment, a cursory search of the web makes it seem that things have not really improved, though there do seem to be some mobile apps out there.

 

I see. I have seen an open CV solution using python that could detect left center and right positions of the eye but that would require porting to open CV unity from the asset store (free and paid versions).

Here's the link:

https://pysource.com/2019/01/04/eye-motion-tracking-opencv-with-python/

1 cm accuracy is high enough for the application I'd like to build. You think it's possible to make it natively or even as a java/android plugin?

 

If you have the source, I'm sure it possible. Android software is out of my depth, but my guess is that is how you'd want to approach it at first anyway. Create a plugin on a device and some small test app that you use could see that plugin returning the data you need from eye-track input.

You could then look at how to hook that up to a Unity android build, or whether or not you could move the plugin code into a unity package, or what. Again that is out of my experience, but I think you could find people here who are who are doing Android apps, or perhaps out there in the Google community who could give you a more informed strategy.

 

Actually I'd much rather have it written in open CV unity plugin so I will be able to debug it within the unity editor rather than having to build it on the android device every time.
The problem is I barely understand open CV in python so porting it with the unity plugin is going to be difficult for me.

 

I feel for you - I don't know enough about Python either!

One semester it was thrown in my lap as an overload course, out of the blue two weeks after the semester started, and it was some curriculum committee member's bright idea to switch from Java to Python for our introductory programming series, somehow without informing all of the faculty. I had never used Python, and man did I do a scheisse job teaching that. (I guess you can't say S*** in this forum! )

It has kind of jaded me against Python, which is too bad because I know it is a cool language. It's kind of like when you OD on some really great tasting food and throw up, and it never tastes the good after that.