Invention Title:

REAL TIME VOICE MODE GENERATIVE RESPONSE ENGINE WITH TURN DETECTION

Publication number:

US20260311720

Publication date:
Section:

Physics

Class:

G10L25/78

Inventors:

Assignee:

Applicant:

Smart overview of the Invention

Disclosed are systems and methods for enhancing real-time interactions using a generative response engine. The technology focuses on improving user experiences through dynamic and natural interactions. It involves receiving audio frames, identifying user speech, and determining the end of a speech instance to trigger the generative response engine's inference operation. This approach aims to provide seamless and responsive interactions by optimizing latency and handling speech nuances effectively.

Challenges in Speech-Based Interactions

Generative response engines, like large language models, have transformed natural language processing but face challenges in speech-based interactions. Unlike text, speech involves pauses and interruptions, complicating the detection of complete speech instances. Latency issues can disrupt conversational flow, making interactions feel unnatural. Traditional sliding window algorithms for speech detection are insufficient, as they add latency and may prematurely end speech detection, impacting user experience negatively.

Innovations in Real-Time Processing

The proposed technology introduces dynamic speech instance detection, leveraging silence identification and complete thought recognition. This method ensures the generative response engine responds within a natural delay period, enhancing user interaction quality. Additionally, the system can incorporate video frames, selectively provided to the engine to enrich context without overwhelming it with excessive data, thus maintaining efficient processing.

Handling Interruptions

An interruption detection engine is designed to improve conversational flow by identifying interruptions and adjusting the generative response engine's output accordingly. When interruptions occur, the system can halt responses and update the engine's state, discarding unpresented content. This feature ensures that the engine adapts to real-time changes in user input, maintaining coherent and contextually relevant interactions.

System Architecture

The generative response engine, capable of generating content from various prompts, interfaces with users through a front end comprising a graphical user interface and APIs. This setup facilitates communication and output presentation, supporting diverse applications like CHATGPT, DALL-E, and SORA. The graphical user interface serves as the primary platform for user interaction, enabling seamless exchanges between the user and the generative response engine.