Risk averse: assessing risk as a surrogate measure in pulmonary arterial hypertension

The design of clinical trials investigating novel therapies for pulmonary arterial hypertension (PAH) has evolved over the years, in both scale and scope. Initial studies employed the 6-minute walk test (6MWT) as the primary outcome measure; these trials were relatively small and short in duration (mostly 12-16 weeks). However, subsequent studies failed to validate the 6MWT as a surrogate measure for clinical worsening or mortality, prompting experts to recommend the use of time to clinical worsening (TTCW), a composite endpoint that encompasses death, morbidity events, and clinical worsening.1,2,3 Subsequently, most of the recent clinical trials of novel agents in PAH have used TTCW as the primary outcome. However, given the low rate of events fulfilling the TTCW definition in these trials, the necessary sample size and study duration increased significantly, impacting the feasibility of conducting such studies in the future. Further, more recent ­studies have called into question the validity of TTCW as a surrogate measure for mortality in PAH, although it may remain a clinically meaningful endpoint.3,4 To address these challenges, investigators have sought to identify and validate other surrogate measures. Multidimensional risk scores offer an assessment of clinical status and have been shown to predict risk of death in various cohorts of patients with PAH. Further, secondary analyses of clinical trials demonstrate the responsiveness of various risk scores and associations with clinical outcomes.5,6 As such, they are commonly used in clinical practice and are incorporated into current guideline treatment algorithms, making them an attractive potential surrogate outcome measure.7,8 Still, controversy exists about the utility of risk scores as outcome measures; a study using patient-level data from 3 randomized controlled trials of novel therapy in PAH found that achieving low-risk status was not a suitable surrogate, based on failure to achieve the generally accepted threshold for mediation in their mediation analyses.9 Others have argued that the findings of Blette and colleagues9 were impacted by the large proportion of participants who were already low risk; a mediation analysis of the FREEDOM-EV trial excluding low-risk participants demonstrated that change in risk score accounted for significantly more of the treatment effect, approaching the threshold for surrogacy.6,10 With this as a backdrop, Liu and colleagues11 report in this issue of the Journal their findings from an examination of risk assessment scores as surrogate outcome measures in PAH. They included 2 event-driven studies: FREEDOM-EV and AMBITION, in which assessments at weeks 12 and 16, respectively, were sufficiently comprehensive to calculate risk scores. Other event-driven studies were excluded due to the absence of data (eg, brain natriuretic peptide was not captured), longer time interval to repeat assessment (26 weeks vs 12 or 16 weeks), and early roll-over to open-label extension that truncated exposure to randomization. Events, defined as death, hospitalization, or disease progression, that occurred during the first year alone were tabulated to reflect the inherent properties of the risk scores, which are currently calibrated to reflect risk at 1 year. The authors then performed a mediation analysis to determine if the change in risk score at early follow-up mediates the effect of treatment on clinical worsening events at 1 year. Further, they compared the proportion of mediation attributable to the change in risk scores to the proportion attributed to the change in the 6MWT. Through these analyses, they found that (1) the change in each risk score studied demonstrated greater percentages of mediation effect than the change in the 6MWT in the overall cohort, (2) the change in each risk score demonstrated greater mediation effect than the change in the 6MWT in intermediate- and high-risk patients, and (3) the greatest mediation effect was seen with the REVEAL Lite 2 and European Four Strata Score. While these findings are of clinical interest and offer insight into the utility of risk scores in PAH, the mediation effect for each is less than 30% for any risk score in the entire cohort (similar to the estimates from the Blette et al9 study), only slightly improving to 38% among the intermediate- and high-risk patients. Thus, each falls short of validation of risk scores as surrogates in PAH, which is generally considered achieved at a threshold of 50% to 75% of the treatment effect.12 The authors further note a statistically significant difference in the percentage mediated by the change in risk scores compared to the change in the 6MWT, interpreting the 4% to 10% difference in the mediation effect as evidence supporting the use of risk scores over the 6MWT as a surrogate outcome. Two major caveats to this interpretation should be considered. First, an increase in the mediation effect of a risk score, which includes the 6MWT and functional class, compared to the 6MWT alone in predicting an outcome that also incorporates the 6MWT and functional class, is wholly expected. Second, and perhaps most importantly, the clinical relevance of a 4% to 10% difference in the mediation effect is unclear, particularly when the total mediation effect is below the aforementioned threshold. It is just as valid to interpret these results to suggest that most of the mediation ­effect for risk scores comes from 6MWT. So why might this be? Risk scores are predictive models, not events, and as such, they do not measure the actual occurrence of the outcome. Rather, risk scores assess the likelihood of an event based on clinical features that may not be directly related to the event of interest at the individual level. Similarly, risk scores can be miscalibrated in certain circumstances, such as in specific patient subgroups (eg, scleroderma-associated PAH) or geography (Framingham Risk Score overestimates risk in European populations).13,14 Further, the impact of newer and potentially more efficacious therapies may affect the accuracy of prediction and ­calibration of current tools (eg, impact of sotatercept as ­background therapy on risk scores at the population level). These and other limitations have precluded the use of other risk scores commonly used in clinical practice as surrogate measures in ­clinical trials of cardiovascular disease. So how do we place these findings into clinical context and ­clinical practice? Risk scores are useful to predict the risk of death at 1 year. They identify features associated with poor outcomes better than our clinical gestalt and, as such, may inform treatment strategies as recommended by our current consensus statements and guidelines.15 A recent study of sotatercept that enrolled ­patients at high risk of death based on risk score utilizing TTCW as the primary outcome was stopped early at the predefined interim analysis due to efficacy; this suggests that stratified enrollment based on risk successfully identifies patients with higher event rates.16 Employment of such strategies may lead to more efficient clinical trials. However, their utility in clinical trials may not be as broad as anticipated or hoped. The study by Liu and colleagues11 is important as it addresses a significant gap in knowledge regarding the strength of association between risk scores and clinical outcomes. Their work highlights the ongoing need for thoughtful inclusion of potential surrogate measures as secondary outcomes in clinical trials to allow for thorough examination of their usefulness and validity. Only through such evaluation and analyses can we identify appropriate surrogate measures for efficient clinical trial design in PAH.17 Supplementary material is available at American Journal of ­Respiratory and Critical Care Medicine online. Please see the ICMJE disclosure forms, which have been provided as supplementary material. No artificial intelligence tools were used in writing this manuscript.