PROMPT-BASED IMAGE RELIGHTING AND EDITING

Smart overview of the Invention

Techniques are introduced for automatic image relighting and editing using prompts that specify lighting and background conditions. A processing device receives a digital image depicting an object, along with a lighting and background prompt. It generates a relit digital object by applying the specified lighting condition and creates a background that incorporates features from the background prompt. The final output combines the relit object with the background, maintaining high fidelity and realistic effects.

Traditional image editing often requires manual effort, which is time-consuming and demands expertise. While machine learning has been employed to automate such tasks, these methods often struggle with maintaining image fidelity and realism, especially in lighting consistency between foreground and background elements. This invention addresses these challenges by leveraging advanced AI models to ensure cohesive and realistic image edits.

The process begins by extracting the digital object from its background and applying the desired lighting condition using a diffusion model. This model adjusts visual factors like brightness and color temperature to match the lighting prompt. Despite its effectiveness, the diffusion model may introduce visual artifacts. To counter this, a histogram matching operation restores content details, ensuring the relit object retains its original high-quality attributes.

Once the relit object is prepared, a generative fill model creates a background that aligns with the background prompt while respecting the lighting conditions of the relit object. This approach prevents lighting disparities between the object and its background, a common issue in conventional techniques. The system also employs a shadow synthesis model to enhance realism by accurately editing shadows in the final image.

The result is an edited digital image where the relit object is seamlessly integrated into the new background, preserving fine details like object shape and text. This method supports the creation of high-fidelity images with realistic lighting, background, and shadow features, offering a significant advancement over traditional and existing AI-based image editing techniques.