Partial-label metric ceilings for evaluating gene regulatory networks inferred from single-cell foundation models
Abstract
Gene regulatory network (GRN) benchmarks are typically interpreted as if curated references were complete, yet they are not. We formalize observed-metric ceilings under partial positive labels and reanalyze existing benchmark outputs across 15 methods and 5 references. The methods include six edge sets derived from a single-cell foundation model (scGPT) via attention and gradient attribution probes, evaluated alongside classical statistical and tree-based inferers and a random control. Under a missing-at-random (MAR) label model, we derive explicit ceilings for observed F1 and AUPR, propagate coverage uncertainty via Beta posteriors, and stress-test assumption violations. Crucially, because real curated references concentrate on extensively studied regulators, we replace stylized non-MAR tests with study-biased missingness grounded in two annotation databases: transcription factors assayed by ChIP-seq (CistromeDB/ChIP-Atlas) and genes carrying Reactome pathway annotations. Across 39 AUPR-evaluable rows, the best normalized F1 and AUPR ratios are 0.137 and 0.014, median normalized AUPR is , and only 3 of 39 rows exceed the observed random baseline; the foundation-model probes occupy the top of the ranking but still operate far below the observable ceiling. Restricting ground-truth edges to pathway-annotated genes leaves only 60% of edges observable and, at the empirically measured coupling between model scores and studied status, inflates observed AUPR by above the MAR ceiling—intermediate between MAR ( 0) and the stylized worst case ( ). Missing labels therefore explain only part of the performance gap; substantial model-to-biology mismatch persists after ceiling correction, and evaluation claims should be narrowed accordingly.
Fields of Study & Subjects
Gene Regulatory Network Analysis
Single-cell and spatial transcriptomics
Bioinformatics and Genomic Networks
Publication details
- Journal / source
- Biosystems
- Publisher
- Elsevier BV
- Volume
- 266
- Pages
- 105864–105864
- Publication date
- 30 June 2026
- Language
- EN