Camera Calibration with Gaze Tracking

Smart overview of the Invention

The patent application describes a method for calibrating forward and backward-facing cameras on a head-mounted device using gaze tracking. This process allows the device to determine where a user is directing their gaze in an external scene. The calibration does not require any additional equipment beyond the head-mounted device itself, making it more convenient and accessible for users.

Background

Uncalibrated cameras on a device can produce disjointed information that doesn't align with each other. Traditional calibration methods require identifying a common point in overlapping fields of view or using specialized equipment for non-overlapping fields. Gaze tracking provides an innovative solution by using the user's eye direction as a calibration proxy, enabling accurate alignment without extra tools.

Calibration Process

The calibration system leverages gaze tracking to align the cameras on the head-mounted device, ensuring accurate gaze target identification. The external camera captures the scene, while the internal gaze camera monitors the user's eye. If the device is deformed, the cameras may become misaligned, reducing accuracy. The calibration system adjusts for these changes, maintaining precise gaze tracking.

Technical Details

The system calculates a gaze vector from the eye's direction and a camera vector from the external camera to the gaze target. By comparing these vectors, the system determines calibration information, adjusting for any rotational or positional changes between the cameras. This ensures the device can accurately determine where the user is looking in the external scene.

Implementation

The head-mounted device includes a controller that processes images from both cameras. It determines the user's gaze direction and correlates it with the external scene. The calibration system uses this data to maintain alignment between the cameras, even if the device's frame is deformed. This method enhances the device's ability to track gaze accurately, improving user interaction with augmented reality and other applications.