End‐to‐End Multi‐Entity Customization

ABSTRACT Recent advancements in text‐to‐image (T2I) models have enabled the synthesis of personalized images that align closely with user‐specified prompts, especially through the use of modifiers. However, generating multiple detailed objects with distinct modifiers in a single image remains challenging due to concept‐mixing, resulting from the difficulty of capturing interactions among text tokens. This paper proposes a modifier‐based approach to mitigate concept‐mixing by addressing the interaction among text tokens. Our method enables practical multi‐personalization while preserving the original T2I model's straightforward inference pipeline. Without structural guidance, it ensures seamless object interaction with enhanced consistency. Through a loss‐based finetuning approach, our method is adaptable to various concept‐learning algorithms, enabling plug‐and‐play functionality. Through both qualitative and quantitative evaluations, we demonstrate that our method effectively resolves concept‐mixing issues to better preserve concepts' identities and outperforms recent baselines in both quantitative and qualitative results. Our code will be publicly available.