Invention Title:

SYSTEM AND METHOD FOR OBJECT INSERTION IN IMAGE

Publication number:

US20260320865

Publication date:
Section:

Physics

Class:

G06T11/60

Inventors:

Assignee:

Applicant:

Smart overview of the Invention

The patent application introduces a method for inserting objects into images using an advanced artificial intelligence (AI) system. The process begins with detecting a user-provided stroke input, which indicates a request to insert objects. The system identifies the shape and color of this stroke and utilizes a generative AI model to create multiple candidate objects based on these characteristics. The system then selects the most suitable objects from these candidates by considering the context of the image and the stroke's location. Finally, the image is updated by inserting these selected objects, taking into account depth information and the stroke's position.

Background

Generative AI models, particularly diffusion models, have significantly advanced image manipulation capabilities. These models can create realistic images from minimal input, yet they often struggle with precise object attributes like color and depth. Existing models such as text-to-image and scribble diffusion face challenges in accurately capturing user intentions, especially in terms of spatial properties and context. Current solutions often fail to integrate new objects into existing images effectively, as they lack mechanisms to handle depth and context information adequately.

Technical Challenges

Traditional image generation models encounter several issues, such as object hallucination, where artifacts create unrealistic objects. They also struggle with maintaining the appearance details of generated objects and understanding the image's context. Occlusion problems occur when strokes overlap existing objects, and existing models may inadvertently generate inappropriate content. Moreover, conventional methods typically place new objects over existing images rather than integrating them seamlessly with depth consideration, requiring manual adjustments for accurate placement.

Proposed Solution

The proposed method addresses these challenges by incorporating a sophisticated AI approach. It involves determining the depth information of objects to be inserted, based on the stroke's position and surrounding objects. The system identifies the scene's context and the stroke's location to select the most contextually appropriate objects. This selection process uses correlation scores to ensure the chosen objects fit the scene accurately. The image is then updated by inserting the selected objects, guided by a depth map and binary mask to ensure proper integration.

Implementation Details

The system generates candidate objects using a diffusion model that leverages color and shape feature maps derived from the stroke input. Low-pass and high-pass filters are used to extract color-based and shape-based features, respectively. The selection of candidate objects involves calculating correlation scores to match the scene's context and stroke location. The image update process ensures proper depth alignment by determining a depth map, assessing the similarity and proximity of candidate objects, and filtering based on these criteria to achieve a seamless integration.