What does Surprisal have to do with Information Status?

Read full text Summarise with VIRAT
Authors: Andrew Thomas Dyer
Cited by: 0
References: 11

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

It is common in cognitive computational linguistics to use language model surprisal as a measure of the information content of units in language production.From here, it is tempting to then apply this to information structure and status, considering surprising mentions to be new and unsurprising ones to be given, providing us with a ready-made continuous metric of information givenness/newness.To see if this conflation is appropriate, we perform regression experiments to see if surprisal is actually well predicted by information status as manually annotated, and if so, if this effect is separable from more trivial linguistic information such as parts of speech and word frequency.We find that information status alone is at best a very weak predictor of surprisal, and that surprisal can be much better predicted by the effect of parts of speech, which are highly correlated with both information status and surprisal; and word frequency.We conclude that surprisal should not be used as a continuous representation of information status by itself.

Fields of Study & Subjects

Syntax Semantics Linguistic Variation Natural Language Processing Techniques Neurobiology of Language and Bilingualism

Publication details

Journal / source
Workshop on Research in Computational Linguistic Typology and Multilingual NLP (SIGTYP)
Pages
26–31
Publication date
1 January 2026
Language
EN