SCRIBBLE-TO-VECTOR IMAGE GENERATION

Smart overview of the Invention

The patent application details a method and system for generating images from sketch inputs, utilizing advanced machine learning models. This system processes informal sketches or scribbles to produce synthesized images that accurately depict the intended object from the sketch. The innovation lies in its ability to transform rudimentary sketches into detailed digital images, enhancing creativity and efficiency in design processes.

Background

Scribbles, ranging from simple doodles to complex sketches, serve as foundational elements in digital art and design. Traditional image generation models have limitations in interpreting such informal inputs. The proposed system addresses these limitations by employing a sketch encoder that processes these inputs, offering a more intuitive and flexible approach to digital design.

Technical Summary

The system includes a sketch encoder that processes sketch inputs to generate sketch guidance. This guidance is then used by an image generation model to create synthesized images. The sketch encoder is initially based on a pre-trained image generation model and is further trained with sketch input data to enhance its accuracy. This approach allows for the creation of synthetic images that can be converted into vector graphics, maintaining the integrity of the original sketch.

Innovative Features

• Utilizes diffusion models to refine sketches into detailed images.
• Accommodates diverse scribble styles and complexities through a specialized training data pipeline.
• Employs adherence factors to ensure the synthesized image closely matches the sketch input.
• Facilitates the transition from informal sketches to precise digital outputs, enhancing artistic workflows.

Application Example

An example provided illustrates the system's use: a user sketches a cute dog on their device, which is processed by an image processing apparatus equipped with a sketch encoder. This apparatus analyzes the sketch, focusing on key features like form and cuteness, and generates a detailed synthetic image. The final output is transmitted back to the user, demonstrating the system's capability to transform simple sketches into high-quality visual representations.