Coherence Over Completion: Why Task Length Is the Wrong Metric for AI Capability
Abstract
This paper argues that the field of AI capability measurement is optimising for the wrong variable. Recent work by METR (Kwa et al., 2025) proposes task length as the primary metric for AI capability, finding exponential improvement with a doubling time of approximately seven months. This paper does not dispute the measurement - it disputes what it measures. Task completion and coupling fidelity are not the same metric. A model that completes a week-long task in an hour and produces output that does not integrate with the receiving context has not succeeded. The paper extends this argument into two adjacent debates: whether Retrieval-Augmented Generation is obsolete (reframed as an empirical routing problem rather than a retrieval problem); and whether the current race toward faster inference and larger compute is building the right thing (argued it is not). Introduces computational patience - slower approach geometry, longer positional resolution - as the variable that produces higher coupling fidelity than throughput optimisation. Proposes coupling fidelity as the missing axis in AI capability evaluation.
Fields of Study & Subjects
Artificial Intelligence Applications
Topic Modeling
Natural Language Processing Techniques
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 12 March 2026
- Language
- EN
- Document type
- Preprint