A Cross-attention Fusion Network for Human Action Recognition in Still Images

The recognition of human actions (HAR) in static images represents a crucial yet difficult challenge in the field of computer vision, primarily because of the lack of temporal motion indicators.Although hybrid Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures have demonstrated potential, they frequently depend on basic late-fusion methods, like score averaging, to integrate global scene context with individual-specific data.This approach limits the model's ability to learn the intricate interplay between a person and their environment.To overcome this limitation, we propose ConViT-CrossAttention, a novel end-to-end architecture that introduces a deep fusion mechanism.Our model utilizes a CNN backbone to extract a global feature map, while a Region of Interest Align (RoIAlign) layer extracts person-centric features guided by bounding box annotations.A cross-attention module then deeply integrates these two feature streams, allowing the global context to query and be enriched by personspecific details.We conduct extensive experiments on the benchmark Stanford40 Actions dataset and demonstrate the critical role of combining our advanced architecture with strong regularization techniques like mixup.Our final model achieves a competitive mean Average Precision (mAP) of 91.58%.These findings confirm that our deep fusion method offers a more robust and cohesive representation for comprehending human actions in static images when compared to more basic fusion techniques.