The Size of an Effect

Effect sizes are traditionally conceived of as descriptive measures that complement significance testing by quantifying the magnitude of observed relationships in linguistic data. This chapter examines two principal categories relevant to corpus linguistics: independence effect sizes measuring association between categorical variables, and goodness of fit effect sizes evaluating how well observed distributions match expected ones. For independence relationships, the chapter focuses on Cramér’s ϕ, a probabilistic measure extending beyond simple difference measures to handle larger contingency tables. Unlike directional measures, ϕ captures bidirectional association between variables, ranging from 0 (independence) to 1 (perfect predictability). Confidence intervals can be constructed using Newcombe-Wilson methods for https://www.w3.org/1998/Math/MathML" display="inline"> 2 × 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq1950.tif "/> tables or Bishop’s variance estimates for larger tables. The chapter illustrates this with a linguistic interaction transmission experiment investigating grammatical priming in the ICE-GB corpus, showing how coordination structures facilitate significantly stronger priming effects than non-coordinated constructions. For goodness of fit scenarios, two complementary measures are presented: unweighted https://www.w3.org/1998/Math/MathML" display="inline"> ϕ p https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq1951.tif "/> for mutual comparison between distributions, and variance-weighted https://www.w3.org/1998/Math/MathML" display="inline"> ϕ e https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq1952.tif "/> for evaluating subset fit against baseline supersets. Both include confidence interval construction methods. The chapter notes that robust confidence intervals applied to effect sizes can permit the same statistical rigour as significance tests, permitting robust and meaningful comparisons to be made.