Calibrated Query Augmentation and Dual Preference Alignment for Retrieval-Augmented Generation

Even though Retrieval-Augmented Generation (RAG) models have shown promise in integrating outside knowledge into Large Language Models, there is still a fundamental problem in this field that needs to be solved. Retrieval models that were created with traditional IR metrics at their core have a tendency to perform poorly on LLM tasks. However, there is a fundamental performance bottleneck because documents that are reranked using conventional relevance metrics frequently lack the supporting context needed to produce the right answer. In this paper, we propose a preference-based learning paradigm, CQADPA-RAG, to rectify this misalignment. We perform a multidimensional query augmentation with the following structure: We start by rephrasing the queries to make them more clear on one end, adding causal dependencies on the other, taking into account several constraints in the middle, and using scoring to verify the citation requirements. We employ confidence-weighted labeling across multiple models to guarantee high-quality annotations. Our solution’s two-stage training mechanism is its key component. Multi-granular preference learning is used in the first stage to take advantage of classification while introducing contrastive losses within and between queries for retrieval at various scales. In the second stage, we incorporate bidirectional LLM preference alignment, a dual ranking mechanism that not only identifies beneficial documents but also blocks harmful ones. Our analysis of five QA benchmarks demonstrates that our model consistently outperforms the current models in terms of support inference and answer accuracy. We demonstrate through systematic ablation studies that every design decision is crucial to attaining such a high performance, particularly for complex reasoning.