GeoLAN: Geometric Learning of Latent Explanatory Directions in Large Language Models

Read full text Summarise with VIRAT
Authors: Tianyu Bell Pan, Damon L. Woodard
Cited by: 0
References: 34

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Large language models (LLMs) demonstrate strong performance, but they often lack transparency. We introduce GeoLAN, a training framework that treats token representations as geometric trajectories and applies stickiness conditions inspired by recent developments related to the Kakeya Conjecture. We have developed two differentiable regularizers, Katz-Tao Convex Wolff (KT-CW) and Katz-Tao Attention (KT-Attn), that promote isotropy and encourage diverse attention. Our experiments with Gemma-3 (1B, 4B, 12B) and Llama-3-8B show that GeoLAN frequently maintains task accuracy while improving geometric metrics and reducing certain fairness biases. These benefits are most significant in mid-sized models. Our findings reveal scale-dependent trade-offs between geometric precision and performance, suggesting that geometry-aware training is a promising approach to enhance mechanistic interpretability.

Fields of Study & Subjects

Explainable Artificial Intelligence (XAI) Topic Modeling Adversarial Robustness in Machine Learning

Publication details

Journal / source
arXiv (Cornell University)
Publication date
19 March 2026
Language
EN
Document type
Preprint