A hyperparameter benchmark of VAE-based methods for scRNA-seq batch integration

Read full text
Authors: Mohamad Kassab, Luiz Henrique de Santana Maniero, Eduardo da Veiga Beltrame
Cited by: 1
References: 44

Abstract

We present the first systematic benchmark of model-architecture hyperparameters for variational autoencoder (VAE) methods for single-cell RNA-seq batch integration within scvi-tools, comparing scVI, MrVI, and LDVAE across four heterogeneous datasets under two feature regimes (all genes vs highly variable genes (HVGs)). We investigated 960 trainings (120 configurations) varying latent size and network depth/width, and evaluated with a standardized scIB metric suite covering batch removal and biological conservation (Batch ASW, PCR-batch, iLISI, graph connectivity, NMI, ARI, label ASW, isolated-label F1/ASW, cLISI, trajectory conservation), plus qualitative UMAP/t-SNE and PCA, random projection, and unintegrated baselines. Results show dataset-dependent trade-offs: scVI performs best overall via stronger batch correction; LDVAE can better preserve biological structure in some datasets; MrVI is stable and excels at batch correction in multi-protocol settings but is more resource-intensive. HVG-only training generally outperforms full-gene training for all models. Hyperparameter analysis suggests moderate-to-high latent dimensionality (>30) often gives the best balance; sensitivity to latent size tracks dataset heterogeneity (tissues, labs, chemistries, gene coverage), with larger latents improving batch mixing but sometimes reducing biological conservation. We provide model- and dataset-specific guidelines for practical defaults and tuning of VAE-based integration in single-cell studies. Reproducibility code is available on GitHub at: https://github.com/Kassab11/A-hyperparameter-benchmark-of-VAE-based-methods-for-scRNA-seq-batch-integration

Fields of Study & Subjects

Single-cell and spatial transcriptomics Cell Image Analysis Techniques Gene expression and cancer classification

Publication details

Journal / source
bioRxiv (Cold Spring Harbor Laboratory)
Publisher
openRxiv
Publication date
12 February 2026
Document type
Preprint