Tool-Augmented Multi-Turn Academic Paper Recommendation via Reinforcement Learning

Recent progress in large language models (LLMs) has reshaped the landscape of academic literature retrieval. However, most existing systems still rely on rigid, stage-wise pipelines with limited reasoning depth, making them ill-suited to capture evolving user intents or maintain precision during iterative exploration. We propose a multi-turn, tool-augmented agent that unifies clarification, retrieval, and filtering within a single decision loop. The agent learns the "clarify–retrieve–filter" process end-to-end, enabling dynamic refinement of search results through dialogue. We first employ an LLM-based user simulator to generate realistic multi-turn trajectories for supervised finetuning, and then introduce a Group Relative Policy Optimization (GRPO) algorithm with process reward masking to address the long-horizon credit assignment problem. To support training and evaluation, we construct AnchorScholar, a 30k-instance dataset in which each sample uses an anchor paper’s references as the ground-truth target set. Built on Qwen3-8B, Qwen2.5-7B, and LLaMA3.1-8B, our agent consistently outperforms strong baselines—including Google, GPT-4o, PaSa-7B, PaperFinder, and SPAR—demonstrating superior adaptability and retrieval efficiency in multi-turn academic recommendation.