ANCHOR: LLM-driven Subject Conditioning for Text-to-Image Synthesis

Read full text
Authors: Aashish Anantha Ramakrishnan, Xiaolei Huang, Dongwon Lee
Cited by: 0

Abstract

Text-to-image (T2I) models have achieved remarkable progress in high-quality image synthesis, yet most benchmarks rely on simple, self-contained prompts, failing to capture the complexity of real-world captions. Human-written captions often involve multiple interacting subjects, rich contextual references, and abstractive phrasing, conditions under which current image-text encoders like CLIP struggle. To systematically study these deficiencies, we introduce ANCHOR, a large-scale dataset of 70K+ abstractive captions sourced from five major news media organizations. Analysis with ANCHOR reveals persistent failures in multi-subject understanding, context reasoning, and nuanced grounding. Motivated by these challenges, we propose Subject-Aware Fine-tuning (SAFE), which uses Large Language Models (LLMs) to extract key subjects and enhance their representation at the embedding-level. Experiments with contemporary models show that SAFE significantly improves image-caption consistency and human preference alignment, serving as a practical and scalable solution.

Fields of Study & Subjects

Image Retrieval and Classification Techniques Handwritten Text Recognition Techniques Generative Adversarial Networks and Image Synthesis

Publication details

Journal / source
Findings of the Association for Computational Linguistics: ACL 2026
Publisher
Association for Computational Linguistics
Pages
618–638
Publication date
1 January 2026
Language
EN