Diffusion-Based Multi-Agent With Reinforcement Learning for Multimodal-Based Recommendation

Multimodal-based recommendation integrates visual, textual, and acoustic modalities of items to comprehensively capture user preferences, thereby playing a crucial role in modern multimedia online platforms. In this paper, we propose aDiffusion-based Multi-agent withReinforcement Learning forMultimodal-basedRecommendation (DRMRec). Specifically, our DRMRec first leverages a hierarchicalDiffusion-basedMulti-Agent (DMA) to reconstruct the user-item interaction graph. Subsequently, these reconstructed graphs are fused with multimodal information to form contrastive views. During the reconstruction, each agent is injected with multimodal information through aCross-ModalAligner (CMA), thereby bringing cross-modal information closer to interaction embeddings and facilitating alignment across modalities. Then, we introduce aMetric-AwareDiffusionReinforcer (MADR), a reinforcement learning framework that leverages validation-set recommendation metrics as reward signals to enable dynamic and individual fine-tuning of each agent, thereby actively aligning model optimization with recommendation tasks. Next, we apply cross-modal graph contrastive learning to contrastive views, alleviating data sparsity while further enhancing cross-modal alignment. Extensive experiments on three real-world multimedia platform datasets demonstrate that DRMRec consistently outperforms state-of-the-art approaches in multimodal-based recommendation. It is noteworthy that the performance improvements across all three metrics on both the TikTok and Sports datasets exceed 10%.