How Bias Binds: Measuring Hidden Associations for Bias Control in Text-to-Image Compositions

The rapid advancement of text-to-image generative models has catalyzed widespread applications. However, persistent model biases continue to pose significant challenges to their ethical and effective deployment, often resulting in adverse outcomes across many use cases. Previous research has primarily addressed bias in narrowly defined scenarios, typically involving single-subject generation with limited contextual variability. Such simplified tasks fall short of serving as meaningful model evaluations in more complex real-world settings. For example, the prompt ``an assistant wearing a pink hat'' may reflect female-inclined biases associated with a pink hat. The neglected joint effects of the semantic binding in the prompts cause significant failures in current debiasing approaches. This work investigates how bias manifests under semantic binding, where contextual associations between objects and attributes influence generative outcomes. We demonstrate that the underlying bias distribution can be amplified based on these associations. To address this, we introduce a bias adherence score that quantifies how specific object-attribute bindings activate bias. Using this score, we develop a training-free context-bias control framework that decouples the underlying bias from the semantic bindings, improving over 10% biases in compositional generation tasks. Our analysis of bias scores across various attribute-object bindings and token decorrelation highlights a fundamental challenge: reducing bias without disrupting essential semantic relationships. These findings expose critical limitations in current debiasing approaches when applied to semantically bound contexts, underscoring the need to reassess prevailing bias mitigation strategies.