How Good Is Good Enough? A Simulation Study of Adaptive Health Technology Assessment

OBJECTIVES: This study evaluated how uncertainty in adaptive health technology assessment (aHTA) affects reimbursement decisions and identified factors contributing to this uncertainty. METHODS: A simulation-based approach was used to generate a distribution of aHTA incremental cost-effectiveness ratios (ICERs). ICERs sampled from systematic reviews of 7 technologies were adjusted using (1) USD adjustment for currency and inflation and (2) technology price adjustment. Uncertainty in aHTA was quantified by estimating the probability of wrong reimbursement decisions using a willingness-to-pay (WTP)-threshold-based decision rule. This involved comparing decisions based on the simulated aHTA ICER with a decision based on a known ICER from Thailand, serving as the "true" reference ICER. Financial risk of wrong reimbursement decisions from the aHTA approach was also quantified. RESULTS: The probability of wrong decisions from aHTA decreased when aHTA ICERs were clearly above or below the WTP threshold. Low variability among published ICERs, particularly when studies shared a similar methodological framework, improved confidence in aHTA. Simple adjustments to ICERs, such as technology price adjustments, showed potential in reducing variability across studies. Technologies with modest disease burden and lower cost were associated with smaller financial risks, even under uncertain evidence. CONCLUSIONS: The aHTA approach is likely suitable under 3 conditions: (1) when the aHTA ICER is clearly positioned far from a country's WTP threshold, (2) when published ICERs exhibit low variability, and (3) when disease burden and financial risks are modest.