PROPHET: Efficient and Intelligent Orchestrator for Microservices Scheduling and Scaling

Microservices are popular and widely used in the cloud. However, realizing cost-effective and high-performance microservice orchestration is challenging for Cloud Service Providers (CSPs). Current orchestration mechanisms have limited flexibility and resource efficiency in scheduling and would cause sluggishness in scaling, which brings unnecessary costs to CSP. This paper presents PROPHET, a microservice orchestrator for optimizing service scheduling and scaling. To improve scheduling flexibility and resource utilization, we propose aranking-based p-batch scheduling mechanism, which adopts a pairwise ranker to obtain resource-efficient scheduling plans for large-scale microservice applications rapidly. To advance the scaling agility, we design aproactive prediction-based scaling mechanism, which performs scaling in advance based on resource usage prediction. Our evaluations are conducted on a real-world cluster with the public Alibaba cluster dataset and datasets collected from the cluster. The results indicate that PROPHET can significantly reduce the number of nodes running in the cluster and improve scaling. This shows great potential in achieving cost-effective and high-performance microservice orchestration.