Partial-label metric ceilings for evaluating gene regulatory networks inferred from single-cell foundation models

Read full text
Authors: Ihor Kendiukhov
Cited by: 1
References: 15

Abstract

Gene regulatory network (GRN) benchmarks are typically interpreted as if curated references were complete, yet they are not. We formalize observed-metric ceilings under partial positive labels and reanalyze existing benchmark outputs across 15 methods and 5 references. The methods include six edge sets derived from a single-cell foundation model (scGPT) via attention and gradient attribution probes, evaluated alongside classical statistical and tree-based inferers and a random control. Under a missing-at-random (MAR) label model, we derive explicit ceilings for observed F1 and AUPR, propagate coverage uncertainty via Beta posteriors, and stress-test assumption violations. Crucially, because real curated references concentrate on extensively studied regulators, we replace stylized non-MAR tests with study-biased missingness grounded in two annotation databases: transcription factors assayed by ChIP-seq (CistromeDB/ChIP-Atlas) and genes carrying Reactome pathway annotations. Across 39 AUPR-evaluable rows, the best normalized F1 and AUPR ratios are 0.137 and 0.014, median normalized AUPR is , and only 3 of 39 rows exceed the observed random baseline; the foundation-model probes occupy the top of the ranking but still operate far below the observable ceiling. Restricting ground-truth edges to pathway-annotated genes leaves only 60% of edges observable and, at the empirically measured coupling between model scores and studied status, inflates observed AUPR by above the MAR ceiling—intermediate between MAR ( 0) and the stylized worst case ( ). Missing labels therefore explain only part of the performance gap; substantial model-to-biology mismatch persists after ceiling correction, and evaluation claims should be narrowed accordingly.

Fields of Study & Subjects

Gene Regulatory Network Analysis Single-cell and spatial transcriptomics Bioinformatics and Genomic Networks

Publication details

Journal / source
Biosystems
Publisher
Elsevier BV
Volume
266
Pages
105864–105864
Publication date
30 June 2026
Language
EN