Bags of Words

Chapter 5 addresses yet another aspect of word meanings. Back in the mid-twentieth century, the linguist J. R. Firth (1957, p. 11) stated that “you shall know a word by the company it keeps.” More recently, this idea has been supported by distributional semantic models (DSMs), which come from computational linguistics and demonstrate that a word’s meaning can in fact be derived partly from its statistical co-occurrence patterns with other words. For instance, part of the meaning of scissors can be derived from its tendency to be used together with certain other words like sharp, pointy, cut, snip, paper, hair, etc. DSMs are surprisingly good at predicting people’s performance on many (although not all) conceptual tasks, and they are now so sophisticated that they constitute the engines of many chatbots and AI systems. What’s more, by combining DSMs with brain mapping methods, a rapidly growing line of research has been accumulating evidence that the distributionally based properties of word meanings are not only captured by purely verbal representations in the core language network, but enable a “quick and dirty” shortcut to comprehension.