REINFORCEMENT LEARNING-DRIVEN INTERNAL BACKTRACKING FOR EFFICIENT SEARCH AND REASONING IN LARGE LANGUAGE MODELS

Read full text Summarise with VIRAT
Authors: Leadership and Sciences International Journal of Technology
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Recent advancements in Large Language Models (LLMs) have demonstrated their capabilities not only in reasoning but also in invoking external tools, particularly search engines. However, teaching models to discern when to invoke search and when to rely on their internal knowledge remains a significant challenge. Existing reinforcement learning approaches often lead to redundant search behaviors, resulting in inefficiencies and increased costs. In this paper, we propose SEM, a novel post-training reinforcement learning framework designed to explicitly train LLMs to optimize search usage. By constructing a balanced dataset combining MuSiQue and MMLU, we create scenarios where the model must learn to distinguish between questions it can answer directly and those requiring external retrieval. We design a structured reasoning template and employ Group Relative Policy Optimization (GRPO) to post-train the model’s search behaviors. Our reward function encourages accurate answering while minimizing unnecessary search and promotes effective retrieval when needed. Experimental results demonstrate that our method significantly reduces redundant search operations while maintaining or improving answer accuracy across multiple challenging benchmarks. This framework advances the model’s reasoning efficiency and extends its capability to judiciously leverage external knowledge (Sha et al., 2025).

Fields of Study & Subjects

Topic Modeling Information Retrieval and Search Behavior Natural Language Processing Techniques

Publication details

Journal / source
IJTLS
Publication date
20 March 2026
Language
EN