Real Time Hand Tracking and Pointing for Human Interaction


Real-Time Hand Pointing Estimation

for Human-Computer Interaction

Hand gesture is an efficient means for humans interacting with computers.

Pointing gesture can resolve ambiguities derived from the verbal communication,

thus opening up the possibility of humans interacting or communicating

intuitively with computers or robots by indicating objects or pointed

locations either in the three dimensional (3D) space or on the screen. However,

it is a challenging task to estimate the 3D hand pointing direction automatically

and reliably from the streams of video data due to the great variety and

adaptability of hand movement and the undistinguishable hand features of the

joint parts.

The present technology provides a hand pointing estimation system based

on two regular cameras, which includes hand region detection, hand finger

estimation, two views' feature detection, and 3D pointing direction estimation.

The hand detection system has similarities to a binary pattern face detector,

in which a polar coordinate system is proposed to represent the hand region,

and achieved a good result in terms of the robustness to hand orientation

variation. To estimate the pointing direction, an Active Appearance Model

(AAM) based approach was applied to detect and track 14 feature points

along the hand contour from a top view and a side view. Combining two

views of the hand features, the 3D pointing direction is estimated.



 Medical systems and assistive


 Entertainment (i.e. computer

and console video games)

 Crisis management and disaster


 Computer animation techniques


 Real time robust tracking of

hand using two orthogonal

cameras without any intrusive

glove or marks

 Accurate pointing in a cursor


 Intuitive drawing in a 3D


 Feasible for finger pointing in

a long distance


U.S. Patents 8,971,572 9,128,530,

and 9,372,546



Dr. Lijun Yin is a Professor of Computer Science at Binghamton University. His

research areas include 3D object modeling and analysis, motion tracking, and facial

expression recognition and generation.

Patent Information:
For Information, Contact:
Scott Moser
Binghamton University
Lijun Yin
Shaun Canavan
Kaoning Hu
© 2019. All Rights Reserved. Powered by Inteum