The Idea of Corpus Experiments

This chapter establishes the theoretical foundation for applying scientific methodology to corpus linguistics research. The author argues that corpus linguistics is fundamentally an observational science that can employ rigorous experimental methods analogous to other scientific disciplines, while acknowledging its distinctive characteristics and limitations. The chapter distinguishes corpus experiments from laboratory experiments: corpus research samples and analyses naturally occurring language data in heterogeneous contexts, rather than constraining data collection and manipulating experimental conditions. It demonstrates how research questions must be transformed into testable hypotheses with clearly defined dependent and independent variables, populations and controls. Using the who/whom alternation in 1990s British English as a case study, the chapter describes the complete experimental process from hypothesis formation through chi-square statistical testing. Key methodological principles include eliminating confounding variables (such as removing subjective cases where speakers lack genuine choice), distinguishing correlation from causation, and conducting linguistic interaction experiments examining local textual factors. Experimental refinement reveals nuanced patterns, such as discovering that correlations between subordinate clause subject types and relative pronoun choice differ significantly between spoken and written data. The chapter addresses epistemological questions about scientific proof in linguistics, emphasising that experiments provide evidence rather than definitive proof, and that auxiliary assumptions – including linguistic frameworks and annotation schemes – shape experimental outcomes. It advocates a focus on richly annotated parsed corpora potentially in combination with larger tagged corpora, while maintaining a primacy of linguistic knowledge.