Towards Workload‐Tailored Optimization of Job Scheduling Policies in HPC Environments

ABSTRACT Supercomputers play a pivotal role in advancing research and development across diverse scientific and engineering domains. However, configuring job scheduling in these systems to ensure maximum productivity and cost‐effectiveness is a challenge. Workload simulation emerges as a crucial tool in this context, offering a mechanism to explore job scheduling configurations in the presence of expected user behaviors. In this paper, we focus on simulation‐based optimization applied to tuning job scheduling configurations. We introduce a discrete‐event simulator that utilizes two strategies to accommodate real workload traces under varying job scheduling policies: Job shaping and job splitting. Our findings from evaluating the proposed strategies on a real‐world case study suggest that they allow the effective accommodation of the real workload traces used as input to the simulation of incompatible policies. By plugging the simulator into an evolutionary optimization algorithm, we also demonstrate the flexibility of the proposed strategies in helping with the proper exploration of the job scheduling configuration space.