SpectraBench: A three-stage evolution framework for intelligent large language model evaluation

Evaluating Large Language Models (LLMs) is one of the most resource-intensive steps in the AI pipeline. Conventional sequential execution often results in long runtimes, frequent out-of-memory (OOM) failures, and poor GPU utilization. Existing frameworks underuse modern GPU capabilities and provide limited failure prevention for large models. We present SpectraBench, an adaptive scheduling framework that optimizes LLM benchmarking through machine learning (ML)-driven resource orchestration. The framework follows a three-stage process: (1) Foundation Scheduling for cold-starts using heuristic rules, (2) Hybrid Scheduling that fuses heuristics with ML predictions via confidence-weighted decisions, and (3) Autonomous Scheduling using Random Forest ensembles for automated optimization. SpectraBench integrates continuous monitoring, predictive memory management, and dynamic parameter tuning to prevent failures and maximize efficiency. A two-phase optimization combining Convergence Analysis with Threshold Optimization establishes validated transition points between stages. Across 13 LLMs (0.5B–32B) and 10 benchmarks, SpectraBench achieves 10.3% shorter execution time, 11.8% higher throughput, and 5.5% lower thermal output. Hybrid Scheduling consistently outperforms both heuristic-only and ML-only baselines. SpectraBench is production-ready, modular, and compatible with single- and multi-GPU systems, enabling seamless integration into existing evaluation workflows. By addressing the memory pressure, scheduling, and failure-prevention challenges specific to LLM evaluation, SpectraBench delivers substantial gains in performance, resource efficiency, and reliability.