Impact of strategies for handling overlapping data on effect estimates in overviews of systematic reviews of acupuncture: a protocol for a methodological meta-research study

Systematic reviews (SRs) have proliferated exponentially in recent decades, with estimates suggesting that nearly 80 SRs were published daily between 2000 and 2019.[1] As the number of SRs accumulates, redundancy has become a pervasive issue, where identical or highly similar clinical questions are repeatedly addressed.[2] This overlapping evidence makes it increasingly difficult for clinicians to keep up to date and effectively inform their practice.[2] Consequently, overviews of SRs (OoSRs) have emerged as a vital method to collate broad information, synthesise diverse findings, and provide a comprehensive summary of evidence.[3][4] Ideally, OoSRs serve to resolve discordant conclusions (which usually arise from differing selection criteria, heterogeneity, potential risks of bias, and variable methodological quality) across multiple SRs on the same topic.[3][4] Acupuncture therapy, given its extensive application and global integration, has been the subject of numerous SRs.[5] Accordingly, OoSRs are increasingly conducted to synthesise data from these SRs to guide health decision-making.[6][7][8] Conducting OoSRs in this field faces a critical methodological challenge, which is the overlap of primary studies.[2] Due to the similarity of the Population, Intervention, Comparison, Outcomes, and Study design (PICOS) frameworks across included SRs, the same primary trials frequently appear in multiple reviews. Because the unit of analysis for OoSRs is the SRs, conducting new meta-analyses for outcomes may be time- and resource- intensive.[9] However, when compared with new meta-analyses, the choice of different strategies will lead to varied effect estimates of acupuncture and be limited by incomplete evidence coverage (e.g., selecting the best quality SR) or double-counting of trials (e.g., second-order meta-analysis).[10][11] Methodological surveys indicate that only 30% to 63% of overview authors reported that they used specific strategies to manage overlapping primary studies.[11][12][13] Inappropriate handling of the overlap may artificially inflate statistical power and narrow confidence intervals, leading to incorrect understanding of acupuncture’s efficacy.[9] Such distortions in effect estimates could mislead clinical decision-making and guideline recommendations, potentially leading to the over-recommendation of ineffective interventions or the underestimation of beneficial ones, thereby allocating healthcare resources inefficiently.[10][12][13] Therefore, a systematic assessment of how different strategies for handling overlapping data impact outcomes is essential. To fill this knowledge gap, we will conduct a methodological meta-research study to inform evidence-based clinical practice and support healthcare decision-making. Objective and research question The primary aim of this study is to investigate the frequency and characterise the specific strategies used to handle overlapping data from primary studies within OoSRs regarding the effectiveness of acupuncture. Furthermore, this study aims to compare the impact of these strategies on estimated treatment effects and to analyse their variation across OoSRs. This study addresses the following research questions: (1) How frequently are strategies for handling overlapping data applied in OoSRs? (2) What are the characteristics of the strategies used (visualisation, quantification, resolution)? (3) What is the magnitude of the divergence in estimated treatment effects yielded by different handling strategies compared to outcomes from new meta-analyses?