ScalerEval: Automated and Consistent Evaluation Testbed for Auto-scalers in Microservices

Auto-scaling is an automated approach that dynamically provisions resources for microservices to accommodate fluctuating workloads. Despite the introduction of many sophisticated auto-scaling algorithms, evaluating auto-scalers remains time-consuming and labor-intensive, as it requires the implementation of numerous fundamental interfaces, complex manual operations, and in-depth domain knowledge. Besides, frequent human intervention can inevitably introduce operational errors, leading to inconsistencies in the evaluation of different auto-scalers. To solve these issues, we present ScalerEval, an end-to-end automated and consistent testbed for auto-scalers. ScalerEval integrates essential fundamental interfaces for implementation of auto-scalers and further orchestrates a complete evaluation workflow for researchers. The source code is publicly available at https://github.com/WHU-AISE/ScalerEval and the demo video is available at https://youtu.be/HFxuXxNBTAM.