Correlations without causation do not support claims of human–LLM reasoning alignment

Read full text
Authors: Ivan Vankov, Federico Adolfi, Rachel F. Heaton, Guillermo Puebla, Jeffrey S. Bowers
Cited by: 3
References: 5

Abstract

de Varda et al. (1) ask whether large language reasoning models (LRMs) can serve as models of human reasoning.To address this question, they compared LRM and human performance across seven tasks that varied in difficulty and found that the number of "reasoning" tokens generated by LRMs predicted human reaction times.The authors concluded that "LRMs exhibit strong alignment with human reasoning behavior, not only in terms of which problems are solved correctly but also in the processing cost associated with solving them."They also took these findings to challenge the view that symbolic architectures are required to support human-like thinking.Central to their argument is the assumption that the correlations were causal, with successful predictions reflecting similar reasoning processes.We tested this assumption by manipulating the length of the reasoning trace generated by the LRM gpt-oss-120b on six tasks used by de Varda et al.These manipulations had minimal effect on accuracy in 5 of 6 tasks (Fig. 1).Furthermore, the LRM generated a similar number of tokens when adding single-and double-digit numbers (Fig. 2A).If the observed correlations reflected LRM-Human alignment, then increasing/decreasing the number of tokens output by the LRM should have improved/impaired its performance, and the LRM should have generated many more tokens when adding 2-digit numbers.An inspection of a sample LRM reasoning trace illustrates the problem (Fig. 2B).

Fields of Study & Subjects

Child and Animal Learning Development Biomedical Text Mining and Ontologies Genomics and Rare Diseases

Publication details

Journal / source
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Volume
123
Issue
12
Pages
e2536362123–e2536362123
Publication date
16 March 2026
Language
EN