317 Attention Is All You Need in the Brain
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
INTRODUCTION: Large language models (LLMs) are remarkably adroit at using language. Their expressive power derives, in part, from mechanisms of self-attention. Given similarity in their capabilities to human brains, it is possible that these two vastly different networks share some underlying computations. METHODS: Native English-speaking patients undergoing neural monitoring for epilepsy listened to 47 minutes of English speech (six monologues taken from the Moth podcast). We collected responses of isolated single neurons in hippocampus (HPC, n = 356 neurons) from 10 patients. RESULTS: Hippocampal neurons (n = 356 neuron from 10 patients) represented three key computations. First, word position was not only encoded via linear index within a phrase construct, but also in Fourier space, consistent with some LLM architectures. Second, semantic encoding was linearly modulated by the representations of prior words, with lexical pairing strength correlated to both standard language models and LLM-derived estimates of self-attention weighting. Finally, neurons encoded the potential space of future words (i.e. not just the true next word but also a list of most-likely next words). CONCLUSIONS: Together these results support the idea that the principles of self-attention used in transformer models of language have at least some overlap with the brain mechanisms of language processing, possibly due to similar computational goals.
Fields of Study & Subjects
Neurobiology of Language and Bilingualism
Neural dynamics and brain function
Memory and Neural Mechanisms
Publication details
- Journal / source
- Neurosurgery
- Volume
- 72
- Issue
- Supplement_1
- Pages
- 82–82
- Publication date
- 26 March 2026
- Language
- EN