A laser embedding frame work for cross-lingual hate speech detection

Detection of hate speech in multilingual as well as code-switched settings continues to remain a major challenge for online content moderation. This paper presents a new method that employs Language-Agnostic SEntence Representations (LASER) embeddings with token-level language identification along with adapter layers for efficient hate speech identification in 13 languages. Our method provides three contributions: (1) token-level language identification to analyze code-switching behavior, (2) light-weight adapter layers for per-language few-shot learning with no loss of multilingual representation, as well as (3) an explanation module with clear decision-making. We experiment on a multilingual collection of 219,981 samples, which shows that our new model attains an F1-score of 0.7337 as well as AUC-ROC of 0.8802 with a 0.65 percentage point improvement over a baseline. Token-level analysis shows that 60.78% of text samples display code-switching behavior, indicating a need for fine-grain language identification. Our explanation study shows that LASER embeddings contribute 3.5 times more than language features towards final prediction. The generalizability of the model extends over a large range of languages that cover European, Asian, as well as low-resource languages, with good cross-lingual performance with hate detection rates between 20.10% as well as 48.71% across test languages.