Meta-Tests for Comparing Tables of Results

Researchers frequently need to compare experimental results across different datasets or experimental designs. However, comparison of significance test statistics ( https://www.w3.org/1998/Math/MathML" display="inline"> χ 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq2267.tif "/> , ‘ p values’) is fundamentally flawed, as these measures conflate effect size with sample size and cannot determine whether results are statistically separable. This chapter introduces rigorous statistical methods for comparing contingency table results through three classes of meta-tests. Point tests compare specific cell proportions or rows of data between tables using Newcombe-Wilson difference intervals or https://www.w3.org/1998/Math/MathML" display="inline"> χ 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq2268.tif "/> homogeneity tests, enabling researchers to determine whether observed frequencies differ significantly across studies. Gradient tests evaluate whether the magnitude or pattern of relationships differs between tables. For https://www.w3.org/1998/Math/MathML" display="inline"> 2 × 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq2269.tif "/> tables, Wilson-based gradient tests compare differences in proportions using generalised difference-of-difference intervals. Extensions to larger tables use https://www.w3.org/1998/Math/MathML" display="inline"> χ 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq2270.tif "/> , though direct gradient comparisons are less conservative than Cramér’s ϕ intervals. Goodness of fit gradient tests perform ‘separability of fit’ evaluations, comparing how well observed distributions match expected ones using modified variance estimates and continuity corrections. The chapter also addresses subset tests for datasets where one represents a subset of another. Practical applications include comparing modal verb distributions across corpora, evaluating grammatical priming effects, and assessing probability patterns between language varieties. The methods accommodate standard statistical corrections and emphasise transparent reporting for meaningful cross-study comparisons.