Real Time Eye Tracking for Human Computer Interaction


Real-time Eye-tracking for Human-Computer Interaction

The ideal human-computer interaction system should function robustly with as

few constraints as those found in human-to-human interaction. One of the most

effective means of interaction is through the behavior of the eye. Specifically,

knowledge of the viewing direction and thus the area of regard offers insight into the

user's intention and mental focus, and consequently this information is vital for the

next generation of user interfaces.

The present technology enables the ability to capture the direction the eyes point

in while the subject is a distance away from the camera. This offers the potential for

intuitive human-computer interfaces, allowing for a greater interactivity, more intelligent

behavior, and increased flexibility. A two-camera system is provided that detects

the face from a fixed, wide-angle camera, estimates a rough location for the eye region

using an eye detector based on topographic features, and directs another active

pan-tilt-zoom camera to focus in on this eye region. Additionally, an eye gaze estimation

approach is provided for point-of-regard (PoG) tracking on a large viewing

screen. To allow for greater head pose freedom, a calibration approach facilitates

finding the 3D eyeball location, eyeball radius, and fovea position. Moreover, both

the iris center and iris contour points are mapped to the eyeball sphere (creating a

3D iris disk) to get the optical axis; the fovea is then rotated accordingly, and the

final, visual axis gaze direction computed. This gaze estimation approach may be

integrated into a two-camera system, permitting natural, non-intrusive, pose-invariant

point of gaze (PoG) estimation in distance, and allowing user translational freedom

without resorting to infrared or complex hardware setups such as stereo-cameras or “smart rooms.”


 Advertisement & Marketing:

augmented displays, projected

wall or semi-transparent surfaces

(i.e. Shopping displays, LCD

screens, computer screens,

smart boards, tabletop displays,

televisions, tablets, cell phones,

wearables, billboards, clothing

racks, commercial displays)

 Website usability testing

 Assistive technology

 Automobiles to control dashboard

or instrument cluster


 Digital and operational training


 Human behavior

 Augmented interactive or noninteractive

home appliances

 Incorporation into gaming devices


 Real-time 3D eye ball estimation

from a single regular camera

(no infrared)

 3D eye gaze estimation

 3D pose invariant

 Allows users translational freedom

relative to the camera, and

permits varied movement in

depth freedom

 Feasible for gaze tracking in a



U.S. Patents 8,885,882 & 9,311,527

U.S. Patent Application 15/065,472


Dr. Lijun Yin is a Professor of Computer Science at Binghamton University. His

research areas include 3D object modeling and analysis, motion tracking, and facial

expression recognition and generation.

Figure from U.S. 8,885,882: showing the system in use



Patent Information:
For Information, Contact:
Scott Moser
Binghamton University
Lijun Yin
Michael Reale
© 2019. All Rights Reserved. Powered by Inteum