Choosing the Right Test

This chapter provides a comprehensive framework for selecting appropriate statistical tests in corpus linguistics research, moving beyond traditional decision trees to emphasise proper experimental design and data understanding. Rather than focusing solely on test mechanics, the author addresses how researchers should approach statistical analysis by first establishing sound research design principles. The chapter examines the crucial distinction between dependent and independent variable selection, emphasising choice-based research questions in corpus linguistics. It critiques the widespread use of word-based frequency baselines (per million words) that violate Binomial model assumptions, noting problems of linguistic implausibility, confounding variation, and exaggerated potential variation. The solution requires methodologically sound baselines reflecting genuine linguistic alternation patterns. The chapter provides detailed guidance on categorical data analysis, distinguishing between goodness of fit tests and independence tests. It emphasises benefits of simple https://www.w3.org/1998/Math/MathML" display="inline"> 2 × 2 https://www.w3.org/1999/xlink" xlink:href=" https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781003622697/595a86a4-dce3-49d4-a444-4b7b0b7a7b7d/content/ieq1858.tif "/> tests over complex multidimensional analyses, showing how visualisation through confidence intervals enhances interpretability. Beyond categorical analysis, the chapter discusses tests for ranked, interval, and ratio data, including Student’s and Welch’s t tests. Throughout, it advocates for a visualisation-first approach using confidence intervals to understand data patterns before selecting tests. It emphasises effect size estimation over simple significance testing, warning against ‘ p -hacking’ and misuse of complex methods without considering underlying assumptions.