Clinical Validation of a Cell-Free DNA Fragmentome Assay for Augmentation of Lung Cancer Early Detection—Letter

Dr. Mazzone and colleagues report on a cell-free DNA fragmentome assay (DELFIscore) for the classification of lung cancer (1). We have concerns and recommendations.In the current study, there are differences between cases and controls in the distributions of multiple risk factors, and potential exists for confounding although the authors assert, “No appreciable confounding by subject characteristics … was observed (Fig. 3a).” Consider that the age in cases is substantially higher than in controls (median 70 vs. 64 years). Seventy-three percent of cases and fifty percent of controls were ≥65 years [OR = 2.75; 95% confidence interval (CI), 2.07–3.65; P < 0.0001]. Calculated by the well-validated PLCOm2012 prediction model (2), the average 6-year lung cancer risk for 70-year-old individuals with other predictor values set to control levels (article Table S1) is 2.90% versus 1.84% for similar 64-year-olds (absolute △ = 1.1%, relative △ = 1.6). The case–control risk difference due to age alone is substantial. Article Figure 4a demonstrates that for similar sensitivities, specificities for age < 65 years versus age ≥ 65 years differ by about 25%, and 95% CIs do not overlap (for a difference of 40% vs. 60%, P < 0.001). Thus, an important, significant age difference in predictive performance for DELFIscore exists. The associations between DELFIscore and age and DELFIscore and lung cancer after adjustment for age (and other risk factors) were not presented. Age was excluded from the provided study dataset.Study participants were enrolled from three separate studies with different protocols. Protocol-specific site effects and the impact of clustering were not presented.In study data (Table S1 and provided dataset), current smoking is strongly protective against lung cancer (OR = 0.58; 95% CI, 0.44–0.75; P < 0.0001). This is contrary to established knowledge. The test sample included individuals who were United States Preventive Services Task Force (USPSTF) ineligible and had quit-times >15 years (68/382 = 17.8%). Lung cancer risk in those sampled with ≤15 years of quit-years declines with increasing quit-years, as expected (Fig. 1A, below). Lung cancer risk for individuals with quit-times >15 years is consistently much higher. The quit-time_X_≤15 years quit-time–status interaction is significant (P = 0.033). The risk observed in individuals with quit-times >15 years is unrepresentative of target population patterns and was strong enough to flip the effect for current smoking from being harmful to being protective. The risk in those with quit-times >15 years is higher than observed in many clinical settings and is not representative of the USPSTF eligible target population, which is the basis for the DELFI simulation modeling and study conclusions.The authors reported a strong improvement in positive predictive value (PPV) for DELFIscore over the USPSTF2021 criteria. PPV is strongly driven by the prevalence of the disease under study. We applied the DELFIscore validation sensitivity (84%) and specificity (53%; reported in the article’s Clinical Validation and Contextualization of Test Performance) to the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial (PLCO) participants (nonsynthetic data) who were USPSTF2021-positive, whose annual incidence rate averaged over 5 years was 0.54%. In PLCO USPSTF2021-eligible individuals, the DELFIscore 1-year PPV is 0.96%. The authors report “the PPV of the positive blood test (1.3%) is approximately twice that of LDCT eligibility criteria alone (0.7%).” The PLCO data demonstrate a more modest increase in PPV than presented for the DELFI analysis. The PLCO-based PPV estimate would lead to different simulation results; more people will be directed to screening that will end up being false positives. However, if the authors’ estimate of lung cancer prevalence is correct, PPV improvements of around 90% may be possible.We produced a logistic lung cancer risk prediction model using the DELFI training data and available PLCOm2012 model predictors (education, body mass index, chronic obstructive pulmonary disease, personal history of cancer, and smoking status, intensity, pack-years, and quit-time). The model was validated in DELFI test data. Two important predictors, age and family history of lung cancer, were unavailable for modeling. Analysis used Stata/MP 18.1. In DELFI test data, the PLCOm2012 risk factor model AUC is 0.772 (bootstrap bias-corrected 95% CI, 0.704–0.832), and at a model threshold for positivity of ≥1.98%, the sensitivity is 80.3%, and specificity is 54.1% (Fig. 1B). Even though missing important predictors, the risk model accuracy statistics are comparable with DELFIscore (84% and 53%, respectively). If a risk model can classify as well as or better than the DELFI assay, which requires an office visit, blood draw, and laboratory assay and is relatively expensive, why not alternatively use or incorporate a quality risk model as described in reference (3)?No lung cancer–specific evidence was presented to support the idea that applying the DELFI assay will benefit screening uptake. This idea is speculative. Although colorectal cancer screening uptake is improved using biomarker assays, smoking is generally associated with lower screening participation rates (4). The lack of lung cancer screening uptake is multifactorial, including challenges for patients, providers, and the healthcare system that are unlikely to be resolved by the use of one expensive blood test. Design, analysis, and interpretation problems of this study cast doubt on expectations.M.C. Tammemagi reports that he developed the PLCOm2012 lung cancer risk prediction models, which are referred to in this letter. The model is open access and is available free of charge to noncommercial users. To date, M.C. Tammemagi has not received any money for the use of the PLCOm2012 model, nor does he anticipate any payments in the future.