Exploring Pediatric Brain Tumor Segmentation Using Deep Learning Approaches

Brain tumors pose significant diagnostic and therapeutic challenges, particularly in the pediatric pop- ulation, where tumor types are rare, heterogeneous, and anatomically variable. Accurate segmentation of tumor subregions from magnetic resonance imaging (MRI) is crucial for surgical planning, radiotherapy targeting, and longitudinal response assessment, yet remains a labor-intensive and subjective task when performed manually. Recent advances in deep learning (DL), particularly convolutional neural network (CNN)–based architectures such as U-Net and transformer-based hybrids, have achieved state-of-the-art segmentation performance in adult cohorts. However, the translation of these methods to pediatric neuro- oncology is limited by small, heterogeneous datasets, domain shifts in anatomy and tumor appearance, and institutional variability in MRI acquisition protocols. This thesis addresses these challenges through four major contributions. First, a systematic evaluation of transfer learning (TL) strategies is conducted for three state-of-the-art architectures to determine op- timal layer fine-tuning regimes for CNN- and transformer-based segmentation networks trained on adult data but applied to pediatric gliomas. The study compares decoder-only fine-tuning, full-network fine- tuning, and progressive layer unfreezing, providing empirical guidance for model adaptation under data scarcity and domain shift. Second, a novel hybrid architecture, DSATransU-Net, is proposed. By inte- grating attention gates with transformer-based global self-attention under a deeply supervised training regime, DSATransU-Net improves segmentation of under-represented and morphologically complex tu- mor subregions, achieving competitive performance on the BraTS-PEDs challenge. Third, an exploratory analysis investigates whether case-level segmentation performance can be explained by similarity metrics derived from a Siamese neural network trained on adult MRI data. Although the network successfully captured domain differences and qualitative similarity patterns, statistical modeling revealed limited predictive power for pediatric segmentation quality, underscoring the complexity of cross-domain perfor- mance estimation. Finally, this work introduces Aesclepian Vision, an end-to-end neuroimaging platform that unifies preprocessing, automated segmentation, post-processing, and treatment response assessment into a modular, accessible pipeline. The platform lowers the barrier to clinical adoption, standardizes workflows, and enables multi-center studies, with initial deployment at Alder Hey Children’s Hospital. Together, these contributions advance the methodological, analytical, and translational landscape of pediatric brain tumor imaging, offering practical recommendations for model adaptation, introducing a new state-of-the-art segmentation architecture, and providing infrastructure to bridge the gap between algorithm development and clinical implementation.