LIVE VENUE PERFORMANCE CAPTURE AND VISUALIZATION OVER GAME NETWORK

Smart overview of the Invention

Introduction to Interactive Content Integration

The disclosed systems, methods, and apparatuses aim to enhance live performances by incorporating real-world performers into interactive content. This integration involves identifying specific joints or ligaments of performers from images, which are then represented as performer markers. These markers are used to generate three-dimensional models of performers, which are applied to virtual characters within a three-dimensional space. The virtual characters are then rendered into a two-dimensional space, allowing real-world performers to be seamlessly integrated into interactive content.

Background and Motivation

Traditional live performances, such as concerts or theater shows, typically involve one-way interactions where performers entertain an audience. Audience participation is usually minimal and passive, such as clapping or singing along. This limited interaction leaves room for improvement in the live performance experience, prompting the need for systems that allow for more engaging and participatory audience involvement.

Technical Implementation

The system captures the movements of real-world performers using motion capture technology, which can be markerless and employ computer vision or deep learning techniques. Cameras are utilized to capture images or video of the performers, and these images are processed to identify performer markers. The content server then uses these markers to create three-dimensional models of the performers, which are integrated into virtual characters that mimic the performers' movements within the interactive content.

Interactive Venue Experience

An exemplary venue setup includes cameras, a content server, and portable electronic devices. The venue hosts events where interactive content is presented to the audience, soliciting their participation in activities such as video games or interactive polls. Virtual characters in the interactive content emulate the real-world performers, allowing the audience to engage with the performance in real time. This setup transforms traditional venues into interactive environments, enhancing audience participation and engagement.

System Components and Functionality

The content server plays a crucial role in enabling the interactive experience by processing images to identify performer markers and generating virtual characters. It facilitates the presentation of interactive content on portable devices, allowing audiences to experience and participate in the performance actively. The server's capabilities include real-time tracking of performers and integration of their movements into interactive content, providing a dynamic and immersive experience for the audience.