High Occlusion Eye Tracking

Smart overview of the Invention

The patent application describes an advanced eye tracking system designed for devices like head-mounted displays (HMDs) used in extended reality (XR) applications. The system employs a reference estimator for a rotational model of the eye with N degrees of freedom and a gaze estimator that uses this model as a constraint to estimate the gaze vector. Unlike conventional models with fixed eye centers, this system uses a centroid region, allowing for more accurate tracking of eye rotation. The system can operate with limited light sources, relying on ambient light and other eye features besides glints for tracking.

System Components

The eye tracking system incorporates a reference estimator and a gaze estimator. The reference estimator calculates the eye's position in five degrees of freedom relative to the device, creating a rotational model. The gaze estimator then tracks the eye's elevation and azimuth, constrained by this model. This dual-pipeline approach allows the system to maintain a stable rotational model until a trigger event necessitates a recalibration. The system is designed to function efficiently even in devices with structural limitations, such as glasses-type HMDs, which may have restricted camera placement and limited power resources.

Operation and Features

In operation, the system uses images from eye-facing cameras to determine gaze direction. A trained neural network can be part of the gaze estimator, processing eye image data under the constraints of the rotational model. The system can estimate gaze vectors using both glints and other eye features. If glint-based vectors are unreliable, the system defaults to other features for gaze estimation. This flexibility ensures robust performance even when traditional light source-based tracking is not feasible.

Trigger Events and Model Updates

The system updates its rotational model based on specific trigger events, which could be time-based or detected through device motion sensors. When a trigger event occurs, a new rotational model is estimated, ensuring the gaze estimator continues to operate accurately. This mechanism allows the system to maintain low latency and high reporting rates while adhering to the physical and computational constraints of compact devices.

Applications and Benefits

While primarily designed for XR systems, the described eye tracking methods and apparatus have broader applications, including health monitoring and interactive media. The system's architecture supports high performance in gaze estimation while fitting within the power and form factor constraints of modern wearable devices. Its ability to adapt to limited lighting conditions and structural constraints makes it particularly suitable for next-generation HMDs and other similar technologies.