Electronic Device That Displays Text Summaries

Smart overview of the Invention

The patent application describes a head-mounted electronic device equipped with cameras and displays that can detect and summarize text from the surrounding physical environment. It utilizes a trained model, such as a large language model, to generate text summaries based on the detected text, contextual information, response length parameters, and user queries. These summaries are then presented to the user through the device's display.

Background

Head-mounted devices often feature video passthrough or see-through displays, allowing users to view their physical surroundings while interacting with digital content. This application fits within the broader context of enhancing user interaction with their environment by providing summarized information directly in their field of view.

Functionality

The device comprises cameras, displays, processors, and memory to execute instructions for capturing images of text in the environment. Once a user indicates a desire for a summary, the device processes the captured images and sends relevant information to a trained model. The resulting summary is displayed on the device's screen, offering an efficient way to consume information from physical texts.

Content Display

The head-mounted device supports various types of virtual content display: world-locked, head-locked, and body-locked. World-locked content remains fixed relative to physical objects in the environment. Head-locked content stays in a constant position relative to the user's head movement. Body-locked content is positioned based on the user's body orientation and remains consistent even as the user moves.

Technical Aspects

The device includes control circuitry for executing operations using both hardware and software components. Input-output circuitry facilitates interaction with external devices and captures environmental data. Displays may be transparent for overlaying digital content on real-world views or opaque with passthrough cameras for an immersive experience. This allows users to access enhanced reality environments with integrated summaries of surrounding texts.