itwinai: A Python Toolkit for Scalable Scientific Machine Learning on HPC Systems
Abstract
The integration of Artificial Intelligence (AI) into scientific research has expanded significantly over the past decade, driven by the availability of large-scale datasets and Graphics Processing Units (GPUs), in particular at High Performance Computing (HPC) sites.However, many researchers face significant barriers when deploying AI workflows on HPC systems, as their heterogeneous nature forces scientists to focus on low-level implementation details rather than on their core research.At the same time, the researchers often lack specialized HPC/AI knowledge to implement their workflows efficiently.To address this, we present itwinai, a Python library that simplifies scalable AI on HPC.Its modular architecture and standard interface allow users to scale workloads efficiently from laptops to supercomputers, reducing implementation overhead and improving resource usage. Statement of needIntegrating machine learning into scientific workflows on HPC systems remains complex.Researchers must often invest substantial effort to configure distributed training, manage hyperparameter optimization, and analyze performance, while adapting to varied system architectures.itwinai is a Python library that streamlines this process by providing a unified framework for scalable AI workflows.It offers consistent interfaces for distributed training, supports large-scale hyperparameter optimization, and includes tools for profiling and code scalability analysis.
Fields of Study & Subjects
Scientific Computing and Data Management
Parallel Computing and Optimization Techniques
Distributed and Parallel Computing Systems
Publication details
- Journal / source
- The Journal of Open Source Software
- Volume
- 11
- Issue
- 117
- Pages
- 9409–9409
- Publication date
- 20 January 2026