• Author(s) : Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Yuyin Zhou, Cihang Xie

A new dataset named HQ-Edit, which has transformed the field of instruction-based image editing through a recent research breakthrough, has been introduced. This dataset contains approximately 200,000 high quality edits falling in the new class of image editing datasets.

Unlike previous approaches that relied on attribute guidance or human feedback for dataset creation, the authors of this study have developed a scalable data collection pipeline. This pipeline leverages advanced foundation models, specifically GPT-4V and DALL-E 3, to create a dataset that is both diverse and high-quality.

The process begins with the collection of diverse examples from online sources. These examples are then expanded and used to create high-quality diptychs. Each diptych features an input image, an output image, and a detailed text prompt describing the editing process. To ensure precise alignment between the images and text prompts, the authors employ post-processing techniques. In addition to the dataset, the authors propose two novel evaluation metrics: Alignment and Coherence. These metrics, used in conjunction with GPT-4V, allow for a quantitative assessment of the quality of image edit pairs.

The HQ-Edit dataset, with its high-resolution images and comprehensive editing prompts, significantly enhances the capabilities of existing image editing models. For instance, an InstructPix2Pix model fine tuned with HQ-Edit can achieve state-of-the-art image editing performance. Remarkably, this model even outperforms those fine tuned with human-annotated data, demonstrating the superiority of the HQ-Edit dataset.

In conclusion, the HQ-Edit dataset represents a significant advancement in the field of instruction-based image editing. Its scalable data collection pipeline, innovative evaluation metrics, and high-quality images make it an invaluable resource for researchers and developers alike. By providing a rich and diverse set of editing examples, HQ-Edit paves the way for the development of more sophisticated and capable image editing models.