Reinforcement Learning Based Model Adaptation and Partition for Dynamic DNN Empowered Mobile Vision

Most mobile vision systems adopt background subtraction or task offloading, but inference performance remains constrained by background dynamics. Dynamic deep neural networks (DNNs) scale computation without relying on historical information. However, a dynamic DNN tuned to a specific spatial granularity cannot excel across all images. For example, resolution level models perform well on images with sparse and large objects but poorly on images with dense and small objects. Changing system overheads make the choice of an optimal edge-assisted scheme for dynamic DNNs difficult. In this paper, we propose an efficient reinforcement-learning-based model adaptation and partition scheme for mobile vision empowered by dynamic DNNs, which jointly adapts the backbone network and the partition point of the inference model to accelerate inference without degrading inference accuracy. The scheme selects hyperparameters for dynamic DNNs, such as spatial granularity and complexity level, based on image characteristics and measured performance without relying on previous frames. A model partition algorithm is proposed to split the changing inference model in polynomial complexity using the Ford–Fulkerson algorithm. Experimental results based on object detection and image classification show that the proposed scheme reduces inference latency and energy consumption by over 71.2% and 31.8%, without degrading inference accuracy compared to the benchmarks.