Enhanced Attention Mechanism for Joint Intent Detection and Slot Filling: Integrating Order Preservation and Dynamic Scaling

Intent detection and slot filling are central to Natural Language Understanding (NLU), enabling dialogue systems to interpret user inputs. Existing joint learning models often struggle to preserve token order, capture contextual dependencies, and maintain stable convergence. This paper proposes an Enhanced Attention-Based Joint Learning Model (EABJLM) that introduces a learnable association matrix to strengthen sequential dependency modeling and a scaling factor to stabilize attention distributions during training. In addition, the model employs a multi-embedding strategy that integrates BERT-based contextual embeddings, Word2Vec semantic embeddings, and part-of-speech (POS) embeddings to encode complementary contextual, semantic, and syntactic information jointly. Evaluated on the ATIS and SNIPS benchmarks, EABJLM achieves 97.85% accuracy and 98.15% F1-score on ATIS, and 98.82% accuracy and 97.72% F1-score on SNIPS. Ablation studies confirm that each component contributes to these improvements. Additionally, an analysis of attention channels reveals that three channels provide the best balance of accuracy and F1-score, with extra channels introducing redundancy. These findings demonstrate that EABJLM achieves state-of-the-art performance with reduced training variance and efficient inference, making it suitable for deployment in real-world dialogue systems.