Guide to recent advances in difference-in-differences methodology for population health studies

BACKGROUND: Difference-in-differences is a method commonly used in population health research. It is based on assumptions that are well-known and well-understood in the classic textbook example of two groups (exposed and unexposed) and two periods (before and after). However, several papers have emerged recently highlighting further complications, for instance, when there are multiple groups and/or multiple periods and/or variation in timing of exposure across groups. Many ways to adjust the classic difference-in-differences methodology have been proposed. METHODS: We explain the challenges for difference-in-differences methodology in real-world settings. We summarise alternative methods that have been proposed to deal with deviations from the classic difference-in-differences situation and set out some considerations for choosing between estimators. RESULTS: The straightforward reasoning of the difference-in-differences method cannot be assumed when the effects of an exposure vary between groups and/or vary over time. These so-called 'heterogeneous treatment effects' require explicit consideration because typical ways to estimate difference-in-differences make implicit assumptions about how to create a summary measure of impact from multiple comparisons. We explain how proposed alternative methods deal with the aggregation of these heterogeneous treatment effects. CONCLUSIONS: Real-world applications of the difference-in-differences method typically require the analyst to allow the effects of an exposure to vary across treatment groups and over time. Summary assessment of impact requires an explicit weighting of heterogeneous impacts. The preferred approach to difference-in-differences depends on how exposure is assigned and the availability of data on exposed and unexposed groups before and after exposure begins.