Making decisions based on uncertain evidence: an examination of health technology assessments for rare disease treatments in England and Germany

Understanding how health technology assessment (HTA) agencies deal with uncertainty in decision-making is important because greater tolerance of uncertainty may result in suboptimal decisions. Consideration of uncertainty is a particular challenge for rare disease treatments (RDTs) because of the different clinical and economic uncertainties surrounding them. This issue is becoming ever more relevant as an increasing number of RDTs is being approved for treatment, leading to a higher volume of reimbursement requests to which HTA agencies have to respond, weighing limited evidence against the urgent and varied needs of rare disease patients. RDT appraisals therefore provide a useful lens to investigate how countries deal with uncertainty in HTA decision-making. Existing research has shown that how uncertainty is identified and managed varies between HTA agencies, as do appraisal processes and evaluation methods. HTA agencies also differ in the ways they consider and interpret different types of evidence. Taken together, these differences have been shown to result in variation in HTA outcomes between countries. However, previous studies have mainly relied on case analyses or a comparison of a small number of selected RDTs across countries. Systematic, comparative, empirical analyses of how uncertainty is considered in HTA decision-making for RDTs are lacking. This thesis sought to contribute to closing this evidence gap and to improve our understanding of how uncertainty is considered in HTA decision-making for RDTs. In particular, this thesis investigated how process-specific, product-specific and clinical evidence quality characteristics were considered when evaluating RDTs for public funding in England and Germany. I systematically analyse RDT appraisal documents issued by HTA agencies in England and Germany between 2011 and 2023. The thesis is broadly divided into three parts. First, I present the results of comparative analyses of process-specific, product-specific, and clinical evidence quality characteristics of RDT appraisals. I extract data from RDT appraisal documents using a specially developed protocol, present descriptive statistics and case studies to illustrate the findings and report the results of associations between characteristics and the HTA outcome using logistic regression. The second and third parts of the thesis explore country-specific elements of the HTA process in Germany and England, respectively. First, since the HTA process in Germany specifically allows for the reappraisal of RDTs, I examine reappraisal practices for RDTs in Germany in more detail. Second, I consider aspects of economic modelling, which is central to the HTA methodology in England. Specifically, I explore two issues related to uncertainty in the economic modelling evidence in RDT appraisals: approaches used to model patient and carer health-related quality of life, and the use of automated text classification to analyse evidentiary uncertainty in appraisal documents. This thesis provides empirical contributions to the literature on how uncertainty is considered in HTA decision-making and on cross-country variation in HTA outcomes. Considering appropriate country- and process-specific modifications, insights from this thesis could inform similar analyses of how issues affecting evidence quality and uncertainty are considered in HTA processes in other countries that employ comparative clinical effectiveness or cost-effectiveness approaches.