Discovering the SUPER in computing - dagster-slurm for reproducible research on HPC
Abstract
Dagster-Slurm is an open-source execution layer that natively integrates the Dagster data orchestration platform with the Slurm workload manager. It enables data engineering and scientific computing teams to seamlessly execute Dagster Software-Defined Assets (SDAs) on remote High-Performance Computing (HPC) clusters with minimal configuration changes.
Fields of Study & Subjects
Scientific Computing and Data Management
Distributed and Parallel Computing Systems
Cloud Computing and Resource Management
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 4 March 2026
- Language
- EN