VisAULa: Advancing Facial Action Unit Recognition With Token-Level Alignment

The emergence of multimodal large language models has demonstrated remarkable success in general visual understanding tasks. However, directly applying them to facial Action Unit(AU) recognition still faces two major challenges. First, designing a visual module that efficiently learns highly discriminative AU representations remains difficult. Second, achieving effective vision–language semantic alignment specifically for AUs is non-trivial. To address these issues, we first introduce the concept of AU tokens, which are extracted from the original global image-patch features using a cross-attention mechanism guided by learnable query tokens. This enables the model to focus on AU-specific visual cues while maintaining a compact representation. Next, we propose a novel training strategy called “RandReAU”, which encourages the model to respond correctly to AU tokens presented in arbitrary orders and varying lengths, enhancing its robustness and generalization. Building upon these techniques, we present VisAULa, a novel Vision AUR MLLM that significantly improves both the speed and precision of AU recognition. Extensive evaluations on the BP4D and DISFA datasets demonstrate that VisAULa achieves superior AU understanding while significantly reducing computational cost and accelerating inference, achieving a 2.14×speedup in inference and a 67.9% reduction in FLOPs, highlighting its practical potential for real-world applications.