A Hidden Markov Model Framework for POS Tagging of English–Punjabi Code-Mixed Social Media Text

Part-of-Speech (POS) tagging for code-mixed text is notably challenging due to frequent language switching, non-standard orthography, transliteration issues, and the prevalence of informal syntactic structures in user-generated content. This study presents a Hidden Markov Model (HMM)-based approach tailored to English–Punjabi code-mixed text, specifically addressing bilingual interactions in which Punjabi is written in Romanized script. A code-mixed corpus was strategically compiled from platforms such as YouTube, Facebook, and WhatsApp, and meticulously annotated at the token level. The resulting dataset comprises 900 sentences totalling 10,117 words, showcasing diverse mixing patterns and typical social media artefacts, including abbreviations, emojis, and irregular punctuation. The proposed framework conceptualizes POS tagging as a sequence labelling problem. It estimates emission and transition probabilities through the annotated corpus and employs the Viterbi algorithm to decode the most probable tag sequences. Experimental evaluations yield an overall tagging accuracy of 71.52%, establishing a probabilistic baseline for English–Punjabi code-mixed POS tagging. This work lays the groundwork for future research to integrate richer feature sets and leverage neural architectures to enhance performance.