Quantitative Optimization of Sensitivity and Specificity in Targeted and Whole-Exome Sequencing Using Reference-Standard DNA Mixtures

ABSTRACT Background We previously developed a benchmarking strategy using mixtures of homozygote and heterozygote DNAs as reference standards to simultaneously assess sensitivity and false positive (FP) error rates in targeted next-generation sequencing (T-NGS) and whole-exome sequencing (WES), revealing substantial variability across commercial platforms. However, optimal analytic conditions for clinical application remain undefined. Methods We systematically evaluated multiple sequencing kits and bioinformatics pipelines across various variant allele fraction (VAF) thresholds to identify conditions that maximize both sensitivity and specificity. Recurrent error-prone alleles were defined and filtered to enhance specificity. Results Optimal performance was achieved using the DRAGEN pipeline with recurrent FP allele filtering. For T-NGS, a 1% VAF cutoff yielded a 95% detection threshold of 2.99% and 1.21 FPs per megabase (FP/Mb); for WES, a 2% cutoff yielded a 95% threshold of 5.02% and 1.15 FP/Mb. These settings improved sensitivity >3-fold and reduced FP rates >96% versus suboptimal pipelines. Notably, VAF thresholds flattened sensitivity differences across platforms, obscuring key performance disparities—challenging assumptions that T-NGS is inherently more sensitive than WES. In-house and conventional pipelines undercalled up to 10% of true variants. Restricting reporting of 1–4% VAF variants to ∼1,000 predefined actionable sites enabled recovery of clinically relevant mutations while reducing FP risk >99%. Conclusions This study provides a quantitative framework for optimizing NGS performance. Our findings support actionable strategies to improve diagnostic accuracy in clinical genomics through tailored pipeline selection, VAF thresholding, and artifact filtering.