Equity Without Utility: An Audit of Whether Synthetic Clinical Data Repairs False-Negative Inequity or Manufactures It

Read full text
Authors: Omar Mohammed
Cited by: 2

Abstract

Synthetic data generated by deep generative models is increasingly proposed as afairness intervention for clinical machine learning: if minority-group patients areunderrepresented or atypical in real training data, generate more of them. This studyaudits that claim empirically. Using a preregistered protocol, a real-only baselineclassifier was trained to predict 30-day readmission in 71,515 unique patients from theUCI Diabetes 130-US hospitals dataset (race, gender, and age labels present; overall testAUC 0.652; labeled-race false-negative-rate gap 7.9 percentage points at a frozenoperating point). Three generators (CTGAN, TVAE, Gaussian copula) and one equity-targeted per-race CTGAN ensemble were trained on the training split only, andevaluated across augmentation (ratios 0.25-1.0), replacement, and targeted arms, withlogistic regression, gradient boosting, and an MLP (150 fits over three seeds), all scoredon real held-out patients never seen by any generator. Results: (1) synthetic-onlyreplacement destroyed clinical utility entirely - every replacement model scored atchance on real patients (AUC 0.48-0.54) while some simultaneously displayed thesmallest group FNR gaps in the study (1.6 pp), a direct demonstration that fairnessmetrics without a utility context can be satisfied by useless models; (2) syntheticaugmentation produced no repair - zero of 90 augmentation runs produced astatistically reliable gap reduction, the mean effect was +1.7 pp (slightly worse), andhigher CTGAN doses significantly worsened the gap for the gradient-boosting learner;(3) the equity-targeted generator showed the largest hallucination index (+0.19) - thegap between its apparent fairness gain inside its own synthetic distribution and its real-data effect - meaning the arm explicitly designed for equity flattered itself most; (4) allgenerators passed naive fidelity and privacy audits while transferring none of theoutcome signal (train-on-synthetic AUC 0.486-0.514 vs 0.652 train-on-real), and TVAEmode-collapsed on the outcome entirely (zero positive labels among the 28,580 rowsgenerated across its two independent training runs), a failure invisible to marginal-fidelity checks and unrepaired by a sanctioned tuning pass. The conclusion is not thatsynthetic data cannot help, but that the burden of proof is on the generator: fidelitymetrics do not measure clinical signal, fairness gains measured on synthetic oraggregate data do not transfer to real sick patients, and equity-targeted generation iscurrently the least trustworthy evaluation surface for its own claims.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Machine Learning in Healthcare Ethics and Social Impacts of AI

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
23 August 2026
Document type
Preprint