Green AI at Scale: Efficient Model Selection for Production Ensembles

Production systems often rely on machine learning ensembles, as aggregating several models tends to enhance predictive consistency. However, running all models simultaneously often leads to unnecessary computational overhead and higher energy use. Prior research shows that carefully choosing a smaller subset of models can reduce energy costs, but many existing approaches depend on evaluating every possible combination, which quickly becomes computationally infeasible as the model pool grows. This paper presents a greedy forward selection algorithm that delivers predictive performance comparable to exhaustive search while reducing computation time by 6-8 times. Furthermore, an adaptive energy-priority parameter is used to systematically adjust the trade-off between accuracy and energypreferences based on operational needs. Experimental validation on the CUAD legal document dataset using five state-of-the-art language models demonstrates that the proposed greedy approach achieves F1 scores of 0.540 (SD 0.016) for static selection and 0.571 (SD 0.013) for dynamic selection across 5-fold cross-validation. These outcomes are statistically equivalent to those from exhaustive search; however, the greedy approach completes in 5-10 minutes, while exhaustive search takes 30-40 minutes, representing a 6-8 times computational speedup. These results show that the proposed framework offers a viable and computationally efficient approach for deploying energy-aware ensemble methods in a real-world production ML environment.