US20260330733
2026-10-22
Physics
G06T13/00
Point correspondence-guided motion transfer is a technique in video generation that leverages machine-learning models to transfer motion from one object to another in digital videos. It utilizes a first digital video depicting a reference movement of a first object and a prompt indicating a second object. The machine-learning model tracks point trajectories across frames of the first video to guide the movement of the second object, ensuring consistent motion transfer without appearance drift.
Traditional diffusion-based video generators often face challenges like appearance drift, where visual elements change or degrade over time. The described system addresses these issues by employing an optical flow procedure to establish point trajectories that maintain spatial correspondence across video frames. This ensures that generated videos have consistent and believable motion, overcoming limitations of conventional models in precise motion control.
The process begins with receiving a first digital video and a prompt, which can be text or an image. The machine-learning model, using an optical flow procedure, determines point trajectories that match points across frames. A denoiser, such as a U-Net, uses correspondence loss to update the latent representation of the first object. This updated representation guides the movement of the second object during the inference stage, resulting in a second video with transferred motion.
The machine-learning model is trained to minimize both diffusion loss and correspondence loss. By focusing on these losses, the model is able to generate a second digital video that accurately reflects the reference movement of the first object. The generated video is then presented to the user via a display device, showcasing the successful motion transfer from one object to another.
The described techniques are implemented in a digital medium environment that includes a service provider system and a computing device connected via a network. The service provider system supports digital services, such as content creation and streaming, using scalable hardware and software resources. Users can access these services through a communication module, enabling the generation and viewing of videos with complex motion transfer.