US20260289746
2026-09-24
Physics
G06T5/70
The described technology focuses on generating videos of unlimited length from user inputs such as text or images without temporal inconsistencies. This process involves iteratively denoising video frames while conditioning the generation based on user input. The system can integrate initial anchor frame features with user input to enhance the consistency of the video frames produced. By conditioning each denoising iteration on previous frame features, the system ensures continuity and coherence throughout the video sequence.
Existing machine learning models for video generation often struggle with creating long sequences due to computational constraints and quality degradation. While short-frame sequences are common, they are insufficient for applications like advertising or storytelling. Training models to generate long videos is typically unfeasible due to the high computational cost and the risk of quality loss when extending short video models. Without addressing these issues, generated videos may suffer from temporal inconsistencies and degradation over time.
To generate videos with consistent quality and scene transitions, the system employs conditioning techniques that consider both short and long-term dependencies. Conditioning involves using similarity scores between data sets to influence the generation process. Attention techniques are used to compute these scores, ensuring the generated frames align well with the intended content. By reducing inconsistencies in scene transitions, the system mitigates issues like video stagnation and object appearance changes.
The Auto-Regressive User Input-to-Video System (AUI2VS) incorporates several modules to improve video generation. The Conditional Attention Module (CAM) leverages previous frame content to generate new frames without restricting motion. The Appearance Preservation Module (APM) maintains object and scene consistency by using initial anchor frame information. Together, these modules enhance the quality and resolution of long videos, employing techniques like randomized blending for seamless transitions between frames.
The AUI2VS system includes pre-trained modules for text-to-video or image-to-video generation. These modules have been trained independently and are used to enhance the quality of video generation within the system. By utilizing pre-trained models, the system can efficiently generate high-resolution videos while maintaining the flexibility to adapt to various user inputs. This approach ensures that the generated videos are both high quality and aligned with the user's original content request.