Reasoning-Enhanced Few-Shot Human Activity Recognition via Visual Reinforcement Learning

Most existing human activity recognition (HAR) models rely on large-scale labeled datasets for training, which makes it difficult for HAR models to recognize activity classes unseen during training when only a few samples are available. Moreover, they struggle to capture contextual information, such as semantic relationships among activities, which further limits their robustness in real-world scenarios. In this work, we introduce FAVOR++, a reinforcement learning (RL)-based framework that enables Multimodal Large Language Models (MLLMs) to perform structured and interpretable reasoning directly over human-activity image inputs and task prompts, thereby improving generalization to novel actions. FAVOR++ first constructs a reasoning-enhanced chain-of-thought dataset by sampling informative plan–reason trajectories, which are used to perform chain-of-thought supervised fine-tuning. Building on this, FAVOR++ applies Group Relative Policy Optimization (GRPO) for post-training, using an outcome-based reward to support classification. To further stabilize training and enhance downstream generalization, FAVOR++ incorporates rectified ratio clipping and dynamic sampling, which suppress unstable updates and reduce variance during policy optimization. Extensive experiments across five few-shot HAR datasets and five distinct few-shot settings demonstrate that FAVOR++ consistently outperforms state-of-the-art methods. Notably, under the 1-shot setting on UCF-101, the proposed method improves the classification accuracy from 38.41% to 72.35%. Furthermore, in the cross-domain generalization scenario, FAVOR++ boosts the average 1-shot accuracy across five in-domain and out-of-domain datasets from 36.18% to 51.99%. In addition to these performance gains, the model also provides more interpretable reasoning and exhibits stronger generalization capability.