Coherence Over Completion: Why Task Length Is the Wrong Metric for AI Capability

Read full text
Authors: James E. Dunn
Cited by: 0

Abstract

This paper argues that the field of AI capability measurement is optimising for the wrong variable. Recent work by METR (Kwa et al., 2025) proposes task length as the primary metric for AI capability, finding exponential improvement with a doubling time of approximately seven months. This paper does not dispute the measurement - it disputes what it measures. Task completion and coupling fidelity are not the same metric. A model that completes a week-long task in an hour and produces output that does not integrate with the receiving context has not succeeded. The paper extends this argument into two adjacent debates: whether Retrieval-Augmented Generation is obsolete (reframed as an empirical routing problem rather than a retrieval problem); and whether the current race toward faster inference and larger compute is building the right thing (argued it is not). Introduces computational patience - slower approach geometry, longer positional resolution - as the variable that produces higher coupling fidelity than throughput optimisation. Proposes coupling fidelity as the missing axis in AI capability evaluation.

Fields of Study & Subjects

Artificial Intelligence Applications Topic Modeling Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
12 March 2026
Language
EN
Document type
Preprint