Correlations without causation do not support claims of human–LLM reasoning alignment
Abstract
de Varda et al. (1) ask whether large language reasoning models (LRMs) can serve as models of human reasoning.To address this question, they compared LRM and human performance across seven tasks that varied in difficulty and found that the number of "reasoning" tokens generated by LRMs predicted human reaction times.The authors concluded that "LRMs exhibit strong alignment with human reasoning behavior, not only in terms of which problems are solved correctly but also in the processing cost associated with solving them."They also took these findings to challenge the view that symbolic architectures are required to support human-like thinking.Central to their argument is the assumption that the correlations were causal, with successful predictions reflecting similar reasoning processes.We tested this assumption by manipulating the length of the reasoning trace generated by the LRM gpt-oss-120b on six tasks used by de Varda et al.These manipulations had minimal effect on accuracy in 5 of 6 tasks (Fig. 1).Furthermore, the LRM generated a similar number of tokens when adding single-and double-digit numbers (Fig. 2A).If the observed correlations reflected LRM-Human alignment, then increasing/decreasing the number of tokens output by the LRM should have improved/impaired its performance, and the LRM should have generated many more tokens when adding 2-digit numbers.An inspection of a sample LRM reasoning trace illustrates the problem (Fig. 2B).
Fields of Study & Subjects
Child and Animal Learning Development
Biomedical Text Mining and Ontologies
Genomics and Rare Diseases
Publication details
- Journal / source
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Volume
- 123
- Issue
- 12
- Pages
- e2536362123–e2536362123
- Publication date
- 16 March 2026
- Language
- EN