TempoFew: Few-Shot Procedural Activity Recognition via Temporal Alignment Pretraining and Prototype Refinement

Recognizing procedural activities from video is essential for applications such as manufacturing quality control, surgical workflow analysis, and intelligent tutoring systems. However, obtaining large-scale annotated datasets for every new procedural domain is prohibitively expensive. In this paper, we propose TempoFew, a framework for few-shot procedural activity recognition that leverages temporal alignment pretraining to build transferable frame-level representations. Our approach consists of two stages: (1) a self-supervised pretraining stage that learns temporally structured embeddings by aligning pairs of demonstration videos using a combination of Soft Dynamic Time Warping and contrastive temporal regularization [8], and (2) a few-shot adaptation stage that constructs temporal prototype sequences from a handful of labeled support videos and classifies query videos via a novel prototype refinement mechanism. We introduce Temporal Prototype Refinement (TPR), which iteratively adjusts class prototypes by exploiting the temporal coherence of the learned embedding space. Extensive experiments on four procedural activity benchmarks-IKEA ASM, 50Salads, Breakfast, and a new dataset we collect called LabProceduresdemonstrate that TempoFew outperforms existing few-shot action recognition methods by 4.7-8.3% in accuracy under 1-shot and 5-shot settings. Our results also show that temporal alignment pretraining provides significantly better initialization than contrastive or predictive pretraining strategies, particularly for long, multi-step procedural activities.