A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs

Read full text Summarise with VIRAT
Authors: Uri Z. Kialy, Avi Shtarkberg, Ayal Klein
Cited by: 0
References: 33

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

While multilingual language models successfully transfer factual and syntactic knowledge across languages, it remains unclear whether they process culture-specific pragmatic registers, such as slang, as isolated languagespecific memorizations or as unified, abstract concepts.We study this by probing the internal representations of Gemma-2-9B-IT using Sparse Autoencoders (SAEs) across three typologically diverse source languages: English, Hebrew, and Russian.To isolate pragmatic register processing from trivial lexical sensitivity, we introduce a novel dataset in which every target term is polysemous, appearing in both literal and informal contexts.We find that while much of the informal-register signal is distributed across language-specific features, a small but highly robust cross-linguistic core consistently emerges.This shared core forms a geometrically coherent "informal register subspace", with greater geometric concentration at the deeper layer.Crucially, these shared representations are not merely correlational: activation steering with these features causally shifts output formality across all source languages and transfers zero-shot to six held-out languages spanning diverse language families and scripts.Together, these results provide the first mechanistic evidence that multilingual LLMs internalize informal register not just as surface-level heuristics, but as a portable, language-agnostic pragmatic abstraction.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Explainable Artificial Intelligence (XAI)

Publication details

Journal / source
Conference on Empirical Methods in Natural Language Processing (EMNLP)
Pages
50015–50032
Publication date
1 January 2026
Language
EN