A MONOTONIC PROGRESS CONSTRAINT FOR LONG-HORIZON REASONING IN LARGE LANGUAGE MODELS.
Abstract
Large language models frequently lose coherence and accumulate logical errors over extended reasoning sequences. This work introduces a monotonic progress constraint,The method is training-free, model-agnostic, and adds negligible computational overhead. On long-chain deductive tasks in mid-sized models, it achieves up to 13× improvement in stability and coherence compared to standard autoregressive decoding, with near-zero error accumulation over thousands of steps. This simple structural prior provides a practical solution for reliable multi-step reasoning in LLMs.
Fields of Study & Subjects
Topic Modeling
Natural Language Processing Techniques
Logic
Reasoning
and Knowledge
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 18 January 2026