ConsistentID is a groundbreaking method designed for diverse identity-preserving portrait generation using fine-grained multimodal facial prompts and a single reference image. This innovative approach addresses the limitations of existing diffusion-based technologies, which struggle to achieve high-fidelity and detailed identity consistency due to insufficient fine-grained control over facial areas and the absence of a comprehensive strategy for identity preservation.
The method consists of two primary components: a multimodal facial prompt generator and an identity-preservation network. The multimodal facial prompt generator combines facial features, corresponding facial descriptions, and the overall facial context to enhance precision in facial details. The identity-preservation network, optimized through the facial attention localization strategy, aims to preserve identity consistency in facial regions. The synergy between these components significantly improves the accuracy of identity preservation by introducing fine-grained multimodal identity information from facial regions.
To support the training of ConsistentID, the authors introduce FGID, a fine-grained portrait dataset containing over 500,000 facial images. This dataset offers greater diversity and comprehensiveness compared to existing public facial datasets, enabling the model to learn from a wide range of facial variations.
Experimental results demonstrate that ConsistentID achieves exceptional precision and diversity in personalized facial generation, outperforming existing methods on the MyStyle dataset. Moreover, ConsistentID maintains a fast inference speed during generation, despite introducing more multimodal identity information. This breakthrough in identity-preserving portrait generation has the potential to revolutionize applications in various domains, such as virtual avatars, gaming, and personalized marketing.
