Representation learning for multi-modal spatially resolved transcriptomics data

MOTIVATION: Spatial transcriptomics enables in-depth molecular characterization of samples on a morphology and RNA level while preserving spatial location. Integrating the resulting multi-modal data is an unsolved problem, and developing new solutions in precision medicine depends on improved methodologies. RESULTS: We introduce AESTETIK, a convolutional deep learning model that jointly integrates spatial, transcriptomics, and morphology information to learn accurate spot representations. AESTETIK yielded substantially improved cluster assignments on widely adopted technology platforms (e.g. 10x Genomics™, NanoString™) across multiple datasets. We achieved performance enhancement on structured tissues (e.g. brain) with a 21% increase in median ARI over previous state-of-the-art methods. Notably, AESTETIK also demonstrated superior performance on cancer tissues with heterogeneous cell populations, showing a 2-fold increase in breast cancer, 79% in melanoma, and 21% in liver cancer. We expect that these advances will enable a multi-modal understanding of key biological processes. AVAILABILITY AND IMPLEMENTATION: AESTETIK is implemented in Python 3 and is available as open source software at http://www.github.com/ratschlab/aestetik. The Snakemake pipeline for reproducing the results is available at http://www.github.com/ratschlab/st-rep.