BiDPO: Bidirectional Preference Optimization with Token-level Supervision for Vision-Language Models

Recent advancements in vision-language models (VLMs) have achieved significant progress in image-text reasoning and human-centered multimodal interaction. However, these models often heavily rely on language priors and exhibit limited sensitivity to fine-grained visual details, leading to hallucinations and unreliable predictions that compromise user trust and hinder human-AI collaboration. To address these limitations, we propose BiDPO, a semantics-driven BiDirectional Preference Optimization framework for VLMs to enhance both the reliability and interpretability of VLMs in multimodal systems. BiDPO automatically identifies the semantic focus of each question and then constructs minimal yet contrastive image pairs with targeted visual modifications, which are guided by CLIPbased similarity filtering to ensure semantic consistency and localized variation. Our bidirectional optimization mechanism makes the model distinguish between correct and incorrect image-answer pairs in both forward and reverse directions. This effectively mitigates hallucinations, significantly enhances model attention to salient visual cues, and enables more transparent and trustworthy human-VLM interactions. Furthermore, we integrate explicit token-level supervision and regularization, guiding the model to ground each answer token in the corresponding visual evidence. Extensive experiments on benchmarks, including AMBER, MMHalBench, and ObjectHalBench, demonstrate that BiDPO significantly reduces hallucination rates, lowering the MMHalBench rate from 57.0% to 31.2% for 7B-scale models. And we also evaluate our method on the more comprehensive benchmarks, including VQAv2, GQA, SQA, etc., and the results outperform the performances of existing state-of-the-art approaches.