COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context

Long-horizon tasks that require sustained reasoning and multiple tool interactions remain challenging for LLM agents: small errors compound across steps, and even state-of-the-art models often hallucinate or lose coherence.We identify context management as the central bottleneck-extended histories cause agents to overlook critical evidence or become distracted by irrelevant information, thus failing to replan or reflect from previous mistakes.To address this, we propose COMPASS (Context-Organized Multi-Agent Planning and Strategy System), a lightweight hierarchical framework that separates tactical execution, strategic oversight, and context organization into three specialized components: (1) a Main Agent that performs reasoning and tool use, (2) a Meta-Thinker that monitors progress and issues strategic interventions, and (3) a Context Manager that maintains concise, relevant progress briefs for different reasoning stages.Across three challenging benchmarks-GAIA, BrowseComp, and Humanity's Last Exam-COMPASS improves accuracy by up to 20% relative to both single-and multi-agent baselines.We further introduce a test-time scaling extension that elevates performance to match established DeepResearch agents, and a posttraining pipeline that delegates context management to smaller models for enhanced efficiency.