PERSONALIZED SKIN TONE ADAPTATION FOR IMAGES AND VIDEO

Smart overview of the Invention

An innovative apparatus is designed to enhance skin tone adaptation in images and video by determining an initial skin tone group distribution for a user in an input image. It calculates several skin tone measurements linked to various skin tone groups based on this input image. The apparatus then outputs a revised distribution using these measurements, the initial distribution, and a transition model, which collectively improve the accuracy of skin tone representation.

In video conferencing and creation, image signal processors (ISPs) are crucial for producing visually appealing images through automatic exposure, white balance correction, and color correction. These processes are important for accurate skin tone reproduction but often struggle in complex lighting scenarios. The narrow tonal range of human skin within the Monk Scale is used to approximate correct settings, although limitations exist in challenging conditions where exact color determination is difficult.

The disclosed methods utilize machine learning to learn a primary user's skin tone under favorable conditions and apply this information to refine white balance, exposure control, and skin tone reproduction in more complex scenes. This approach ensures accurate skin tone representation while maintaining lightweight processing by focusing on the primary user's specific skin tone rather than reproducing all possible tones.

The system involves a combination of skin tone adaptation circuitry with ISP circuitry. This setup features a switch that routes input frames either for normal operation or training purposes. By detecting the user's face and identifying the primary user, it estimates their skin tone using predefined groups like the Monk Skin Tone Scale, which categorizes tones from lightest to darkest. This estimation assists in enhancing ISP functions such as automatic white balance (AWB) and automatic exposure (AE).

AWB circuitry determines white balance coefficients by resolving ambiguities using additional face information, while AE circuitry adjusts exposure to optimize face visibility. Different Monk Skin Tone groups have varied target luminance values, allowing tailored exposure settings for diverse users. In challenging lighting conditions, the apparatus accurately identifies the user's skin tone group, enabling precise adjustments that result in improved visual output.