A Replication Framework for Observational Research: From Specification Restriction to Specification Expansion

Sociology has remained relatively insulated from debates about the "replication crisis." Heeding calls to consider replication more deeply, we introduce a distinction between two types of research reforms that have emerged in the wake of the crisis: specification-restricting and specification-expanding reforms. Specification restriction is often promoted as a way to increase the repeatability of research findings by controlling false positives. We argue that this statistical justification is less compelling in observational contexts---where the assumptions licensing false-positive control are harder to sustain and rigid ex ante analytical commitments can instead enshrine fragile specifications---than in experimental contexts. We further argue that apparently low replication rates do not necessarily imply a high prevalence of false positive findings. Instead, we propose a replication framework centered on specification-expanding reforms, stronger incentives for confirmatory research, and meta-analysis. This approach equips sociologists to assess the repeatability of findings and build a more cumulative discipline.