Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation

作者：Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yan Yu, Jie Song · 发表于：Proceedings of the AAAI Conference on Artificial Intelligence · 年份：2025 · DOI：10.1609/aaai.v39i4.32386 · 被引用次数：7 · 研究领域：Medical Image Segmentation Techniques、Image Retrieval and Classification Techniques、Computer Graphics and Visualization Techniques

Personalized text-to-image generation methods can generate customized images based on the reference images, which have garnered wide research interest. Recent methods propose a finetuning-free approach with a decoupled cross-attention mechanism to generate personalized images requiring no test-time finetuning. However, when multiple reference images are provided, the current decoupled cross-attention mechanism encounters the object confusion problem and fails to map each reference image to its corresponding object, thereby seriously limiting its scope of application. To address the object confusion problem, in this work we investigate the relevance of different positions of the latent image features to the target object in diffusion model, and accordingly propose a weighted-merge method to merge multiple reference image features into the corresponding objects. Next, we integrate this weighted-merge method into existing pre-trained models and continue to train the model on a multi-object dataset constructed from the open-sourced SA-1B dataset. To mitigate object confusion and reduce training costs, we propose an object quality score to estimate the image quality for the selection of high-quality training samples. Furthermore, our weighted-merge training framework can be employed on single-object generation when a single object has multiple reference images. The experiments verify that our method achieves superior performance to the state-of-the-arts on the Concept101 dataset a...