ClassBooth: Boost Class Semantics With Bidirectional Feature Fusion in Text-to-Image Diffusion Models

Text-to-image (T2I) diffusion models aim to generate images that are both visually realistic and aligned with open-domain textual prompts. However, the leading T2I models often fall short in capturing semantic details, especially for fine-grained object categories. We find that simply expanding original prompts cannot effectively guide the model to generate the desired content, since T2I diffusion models pre-trained on generic datasets lack a mechanism to boost class semantics. In this work, we present ClassBooth, a flexible framework that improves pre-trained T2I diffusion model in rendering class-specific content, while preserving the open-domain generation capability. Toward this end, we introduce an auxiliary class-specific semantic booster conditioned on learnable prompts, which are associated with specific classes to encode fine-grained conditioning information. To enable the T2I model to synthesize fine-grained details of specific categories, we perform bidirectional fusion on the features conditioned on different information, and this design is beneficial for class-specific semantic expression, thereby synthesizing high-fidelity data encapsulating precise class-specific details. We validate ClassBooth across multiple benchmarks, demonstrating its superiority over existing methods through comprehensive quantitative and qualitative evaluations.