Defining Target Populations for Impact Studies of Education Interventions: Balancing Generalizability and Feasibility in the Real World

This paper provides guidance to researchers conducting rigorous impact evaluations in education aiming to produce evidence that generalizes to a policy-relevant target population, but who must exclude certain types of districts, schools, teachers, or students from the sample to make the study feasible. This guidance includes how to define the broadest population of potential policy interest, narrow the study's target population based on substantive or policy considerations, exclude segments of this population from the study for pragmatic reasons, and assess the generalizability implications of those exclusions. The paper illustrates each step using examples of randomized controlled trials or regression discontinuity designs that aim to measure impacts of educational policies or interventions. The ultimate purpose of the paper is to improve the generalizability of rigorous impact evaluations and make the limitations of their generalizability more transparent.