Fine-Tuned Sentence-BERT for HPC Job Outcome Prediction via Textual Feature Embedding

High Performance Computing (HPC) systems are strained by emerging workloads such as large language models, which drive up resource use and energy consumption. Accurately predicting job runtime, energy usage, and outcomes at submission time is crucial for efficient, energy-aware scheduling. However, key textual metadata are often anonymized or one-hot encoded in logs, losing valuable semantic information. We propose HPCBERT+, a framework that fine-tunes a Sentence-BERT model on HPC job log data to embed these textual attributes into a compact semantic vector. By converting job fields into natural-language sentences and training the embedding model with supervised signals from historical job outcomes, HPCBERT+ produces representations where semantically or behaviorally similar jobs lie close in vector space. These embeddings are combined with standard numerical features and fed into downstream machine learning predictors to forecast each job’s runtime, energy consumption, or performance class. We evaluate HPCBERT+ on logs from two production supercomputers, comparing against baselines using pre-trained SBERT or simple ID encodings. Results show that HPCBERT+ yields significantly lower error in runtime and energy predictions and achieves comparable or better accuracy in job classification, even with fully anonymized text. The fine-tuned model effectively captures HPC-specific usage patterns in the embedding space, enabling more accurate and generalizable predictions. This demonstrates that domain-specific fine-tuning of SBERT yields embeddings that better capture HPC-specific semantics and outcome patterns, leading to more generalizable and accurate models. Our work highlights the importance of leveraging textual features in HPC job records through NLP techniques and opens a new research direction for integrating language models into modern energy-aware HPC scheduling techniques.