Interpreting the Statistical Significance of Pointwise Mutual Information Using Probability Models of Collocation

Collocation analysis is a widespread method in corpus linguistics. A key metric used for collocation discovery is pointwise mutual information (PMI), determined by how frequently a collocation occurs relative to its expected frequency under the assumption of independence. However, PMI suffers from several limitations, especially its well-known bias for collocations involving low-frequency words. In this paper, we propose a method to determine the significance of the PMI statistic by calculating its p-value, following two probability models for collocations involving the binomial distribution and the Poisson distribution. We demonstrate the effectiveness of this method by investigating collocations involving the Greek word θεóς , ‘god’, in ancient historiography. This example illustrates that the PMI statistic alone does not reveal the significance of a collocation, but rather that p-values provide a consistent threshold of statistical significance and thereby overcome many of the well-known limitations of PMI.