Self Attention K Folded With Xgboost

Sarcasm detection is a challenging task in Natural Language Processing (NLP) due to its reliance on contextual, contrastive, and figurative cues. While deep learning models such as BERT have advanced contextual representation learning, they often rely on computationally expensive end-to-end fine-tuning. This paper proposes a hybrid framework that integrates BERT embeddings with a custom self-attention mechanism for feature extraction, combined with XGBoost classification for efficient decision making. The self-attention layer highlights sarcasm-relevant tokens, producing contextualized, weighted sentence representations, which are then fed into XGBoost to establish robust decision boundaries without neural classification overhead. We further employ Stratified 6-fold cross validation to ensure balanced evaluation and reliable performance reporting. Experiments on benchmark sarcasm datasets demonstrate that the proposed approach achieves superior F1-scores compared to traditional baselines, validating the effectiveness of combining contextual embeddings with gradient-boosted classification. This work highlights the potential of attention-augmented embeddings with ensemble learning for advancing sarcasm detection in social media text.