MedSum-MARL: Multi-Agent Reinforcement Learning with Centralized Training and Decentralized Execution for Context-Aware Medical Video Summarization

Medical video summarization is a critical task in healthcare, enabling efficient analysis of lengthy surgical procedures by extracting the most salient and context-rich frames. Traditional approaches often fail to address the multi-objective nature of summarization, particularly in capturing fine-grained gestures across complex surgical workflows. To address these limitations, we propose MedSum-MARL, a multi-agent reinforcement learning framework designed specifically for medical video summarization. It adopts a Centralized Training with Decentralized Execution (CTDE) architecture, where agents collaboratively learn to optimize different summarization objectivesin formativeness, temporal coverage, and redundancy minimizationthrough a coordinated reward mechanism. The system integrates Kernel Temporal Segmentation (KTS) for detecting coherent action segments and employs transformer-based attention to model contextual dependencies across frames. EfficientNet is used for extracting spatial features, while Farneback optical flow provides motion cues to enhance temporal representation of surgical actions. We evaluated MedSum-MARL on benchmark datasets including JIGSAWS and CATARACTS, achieving significant improvements in F1-Score, Coverage Ratio (CR), and Temporal Smoothness (TS) over state-of-the-art methods. Notably, our approach improves average F1-Score by over 18.5%, with final scores of 85.2% on JIGSAWS and 83.1% on CATARACTS. MedSum-MARL outperforms all baselines with a substantial average F1-score improvement: +24.1% over A2C, +17.1% over PPO, and +12.85% over Single-Agent PPO. These results position MedSum-MARL as a robust solution for medical video summarization with broad applicability across healthcare and other video-intensive domains. Unlike prior methods, the proposed MedSum-MARL framework is built exclusively on a multi-agent reinforcement learning paradigm, while A2C and PPO are implemented solely as single-agent baselines for comparative evaluation.