Deep Learning for Video Summarization: Systematic Review, Challenges and Opportunities

The exponential growth of video content has driven significant advancements in video summarization techniques in recent years. Breakthroughs in deep learning have been particularly transformative, enabling more effective detection of key information and creating new possibilities for video synopsis. To summarize recent progress and accelerate research in this field, this paper provides a comprehensive review of deep learning-based video summarization methods developed over the past decade. We begin by examining the research landscape of video abstraction technologies and identifying core challenges in video summarization. Subsequently, we systematically analyze prevailing deep learning frameworks and methodologies employed in current video summarization systems, offering researchers a clear roadmap of the field's evelution. Unlike previous review works, we first classify research papers based on the structural hierarchy of the video (from frame-level to shot-level to video-level), then further categorize them according to the summary backbone model (feature extraction and spatiotemporal modeling). This approach provides a more systematic and hierarchical organization of the documents. Following this comprehensive review, we summarize the benchmark datasets and evaluation metrics commonly employed in the field. Finally, we analyze persistent challenges and propose insightful directions for future research, providing a forward-looking perspective on video summarization technologies. This systematic literature review is of great reference value to new researchers exploring the fields of deep learning and video summarization.