itwinai: A Python Toolkit for Scalable Scientific Machine Learning on HPC Systems

Read full text
Authors: Matteo Bunino, Jarl Sondre Sæther, Linus Maximilian Eickhoff, Anna Elisa Lappe, Kalliopi Tsolaki, Killian Verder, Henry Mutegeki, Roman Machacek, Maria Girone, Oleksandr Krochak, Mario Rüttgers, Rakesh Sarma
+1 more Andreas Lintermann
Cited by: 0

Abstract

The integration of Artificial Intelligence (AI) into scientific research has expanded significantlyover the past decade, driven by the availability of large-scale datasets and Graphics ProcessingUnits (GPUs), in particular at High Performance Computing (HPC) sites. However, many researchers face significant barriers when deploying AI workflows on HPCsystems, as their heterogeneous nature forces scientists to focus on low-level implementationdetails rather than on their core research. At the same time, the researchers often lackspecialized HPC/AI knowledge to implement their workflows efficiently. To address this, we present itwinai, a Python library that simplifies scalable AI on HPC. Itsmodular architecture and standard interface allow users to scale workloads efficiently fromlaptops to supercomputers, reducing implementation overhead and improving resource usage.

Fields of Study & Subjects

Parallel Computing and Optimization Techniques Scientific Computing and Data Management Distributed and Parallel Computing Systems

Publication details

Journal / source
JuSER Publikationsportal
Publication date
1 January 2026
Language
EN