A Multimodal Foundation Model of Spatial Transcriptomics and Histology for Biological Discovery and Clinical Prediction

Read full text
Authors: Jinxi Xiang, Siyu Hou, Yuchen Li, Ryan J. Quinton, Xiaoming Zhang, Feyisope R. Eweje, Xiangde Luo, Yijiang Chen, Zhe Quan LI, Colin P. Bergstrom, Ted Kim, Sierra Willens
+8 more Francesca Olguin, Matthew Abikenari, Andrew Heider, Sanjeeth Rajaram, Joel W. Neal, Maximilian Diehn, Xiang Zhou, Ruijiang Li
Cited by: 1

Abstract

Spatial transcriptomics (ST) enables gene expression mapping within anatomical context but remains costly and low-throughput. Hematoxylin and eosin (H\&E) staining offers rich morphology yet lacks molecular resolution. We present \textbf{\ours} (\textbf{S}patial \textbf{T}ranscriptomics and hist\textbf{O}logy \textbf{R}epresentation \textbf{M}odel), a foundation model trained on 1.2 million spatially resolved transcriptomic profiles with matched histology across 18 organs. Using a hierarchical architecture integrating morphological features, gene expression, and spatial context, STORM bridges imaging and omics through robust molecular--morphological representations. STORM enhances spatial domain discovery, producing biologically coherent tissue maps, and outperforms existing methods in predicting spatial gene expression from H\&E images across 11 tumor types. The model is platform-agnostic, performing consistently across Visium, Xenium, Visium HD, and CosMx. Applied to 23 independent cohorts comprising 7,245 patients, STORM significantly improves immunotherapy response prediction and prognostication over established biomarkers, providing a scalable framework for spatially informed discovery and clinical precision medicine.

Fields of Study & Subjects

Single-cell and spatial transcriptomics Cell Image Analysis Techniques Gene expression and cancer classification

Publication details

Journal / source
arXiv (Cornell University)
Publication date
4 April 2026
Document type
Preprint