Imprecision in tuberculosis infection outcomes: implications for non-inferiority vaccine trials

BACKGROUND: Randomized trials comparing new vaccines against tuberculosis for use in neonates and infants, for whom Bacille Calmette-Guérin vaccination is established practice, are using tuberculosis infection as the primary endpoint in a non-inferiority design. Markers of tuberculosis infection have imperfect sensitivity and specificity. Flaws in the non-inferiority trial design typically bias towards non-inferiority, which may result in falsely declaring non-inferiority. METHODS: We conducted a statistical simulation study to assess the impact of imperfect markers of tuberculosis infection on the interpretation of tuberculosis vaccine trials testing a non-inferiority hypothesis of an infection primary outcome in a two-arm randomized comparison. Data were generated in three 2-year cumulative risk of tuberculosis infection scenarios (2%, 5%, and 8%). The specificity of tests of tuberculosis infection was assumed to range from 100% to 85%, while the sensitivity was assumed to range from 100% to 64%. Log-binomial regression was used to estimate the relative risk of tuberculosis infection. RESULTS: With 100% sensitivity and specificity, type I error and power were both approximately equal to the expected values (2.5% and 80%, respectively) in all three cumulative tuberculosis risk scenarios. With modest deviations from perfect sensitivity and specificity (95% for both), the risk of falsely declaring non-inferiority was 96.8%, 53.2%, and 27.8% in the 2%, 5%, and 8% cumulative tuberculosis risk infection scenarios, respectively. DISCUSSION: Tuberculosis vaccine non-inferiority trials using an infection primary outcome must be designed and interpreted accounting for the specificity of the tools used to measure infection, otherwise they risk declaring non-inferiority by default.