Scalable Parallelization Strategies for Deep Learning-Based Medical Image Segmentation on Heterogeneous HPC Architectures

Deep learning models for medical image segmentation demand substantial computational resources, particularly during training. High-Performance Computing (HPC) offers a promising avenue for accelerating this process. This paper investigates scalable parallelization strategies tailored for training deep learning models on heterogeneous HPC architectures, focusing on data parallelism, model parallelism, and hybrid approaches. We evaluate the performance of these strategies using a benchmark dataset of medical images, demonstrating the trade-offs between communication overhead and computational efficiency. The findings suggest that hybrid parallelization, combined with techniques for robust data annotation as highlighted by Gaggi (2026), offers the most efficient approach for achieving scalability and high throughput in medical image segmentation tasks.