ProtoFlow: interpretable and robust surgical workflow modeling with learned dynamic scene graph prototypes

Read full text
Authors: Felix Holm, Ghazal Ghazaei, Nassir Navab
Cited by: 0
References: 19

Abstract

PURPOSE: Detailed surgical recognition is critical for advancing AI-assisted surgery, yet progress is hampered by high annotation costs, data scarcity, and a lack of interpretable models. While scene graphs offer a structured abstraction of surgical events, their full potential remains untapped. In this work, we introduce ProtoFlow, a novel framework that learns dynamic scene graph prototypes to model complex surgical workflows in an interpretable and robust manner. METHODS: ProtoFlow leverages a graph neural network (GNN) encoder-decoder architecture that combines self-supervised pretraining for rich representation learning with a prototype-based fine-tuning stage. This process discovers and refines core prototypes that encapsulate recurring, clinically meaningful patterns of surgical interaction, forming an explainable foundation for workflow analysis. RESULTS: We evaluate our approach on the fine-grained CAT-SG dataset. ProtoFlow not only outperforms standard GNN baselines in overall accuracy but also demonstrates exceptional robustness in limited-data, few-shot scenarios, maintaining strong performance when trained on as few as one surgical video. Our qualitative analyses further show that the learned prototypes successfully identify distinct surgical sub-techniques and provide clear, interpretable insights into workflow deviations and rare complications. CONCLUSION: By uniting robust representation learning with inherent explainability, ProtoFlow represents a significant step toward developing more transparent, reliable, and data-efficient AI systems, accelerating their potential for clinical adoption in surgical training, real-time decision support, and workflow optimization.

Fields of Study & Subjects

Surgical Simulation and Training Artificial Intelligence in Healthcare and Education Machine Learning in Healthcare

Publication details

Journal / source
International Journal of Computer Assisted Radiology and Surgery
Publisher
Springer Science and Business Media LLC
Volume
21
Issue
7
Pages
1475–1484
Publication date
28 May 2026
Language
EN