GCN-Transformer-Assisted Live SFC Migration With Hierarchical Reinforcement Learning in Mobile Edge Computing

Empowered by network function virtualization (NFV), mobile edge computing aims to provide low latency and ultra reliable network services to mobile end users, achieved as a service function chain (SFC) consisting of a series of ordered virtual network functions (VNFs). Due to user mobility, live SFC migration is imperative to avoid Quality of Service (QoS) degradation. Recent advances mainly make separate decisions on VNF node remapping and migration path routing in a heuristic manner, or implement both through reinforcement learning within a single agent of ill-defined policy and action space. In this paper, given next access node, we first formulate the live SFC migration problem as an integer linear programming (ILP) model to achieve optimal solutions. Then, we present HRL-QC, a hierarchical reinforcement learning framework that jointly optimizes VNF destination node remapping, migration path and post-migration service path selections for QoS-aware and cost-efficient live SFC migration. A GCN-Transformer block is introduced to capture long-range VNF-to-physical node dependencies, while a two-level actor-critic design couples the decision-makings through inter-level reward passing. Extensive evaluations show that HRL-QC outperforms the state-of-the-art in energy consumption, migration time, end-to-end service delay, and migration success rate, while remaining within a small margin of the optimal ILP solution.