Are Large Language Models Intentional? The Limits of Referential Grounding

This paper reassesses Searle’s Chinese Room argument in light of Large Language Models (LLMs). While Searle criticized computational systems for being mere “syntax manipulators,” transformer models of the kind exemplified by LLMs complicate this characterization. Earlier computational systems relied on a more rigid logical foundation, based largely on pattern recognition and mechanical substitution. By tracking patterns of use, maintaining rich contextual dependencies, and exploiting implicit regularities in large datasets, LLMs show an unprecedented capacity for semantic modelling. And yet, LLMs remain limited in several respects. For instance, even the most advanced AI model suffers from impaired referential grounding and a good model of the external world. I argue that this is because they fail to successfully reproduce the spontaneous, nonsymbolic, biologically entrenched forms of semiosis that characterize the simplest biological organizations. After examining two plausible strategies to cope with these deficits, I conclude that the algorithmic implementation of formal symbol systems diverges in fundamental ways from experience-based intentional representations. This, in turn, warrants skepticism toward bald attributions of intentionality to LLMs and substantiates the core of Searle’s original argument against strong AI.