Deep Layer Expansion: Expert Prompts Counteract Dimensional Collapse in Large Language Models
Abstract
Large Language Models (LLMs) exhibit systematic performance improvements when prompts contain expert-level domain signals. We investigate the geometric mechanism underlying this phenomenon through controlled experiments on two mainstream 70B-class open-source instruction-tuned models. Contrary to the conventional understanding that deep layers compress representations toward deterministic outputs, we discover a striking universal phenomenon: expert signals induce "Deep Layer Expansion" in the representation space. Specifically, expert-level prompts increase the Effective Intrinsic Dimension (EID) in deep layers (Layer 60+) by 60-100% compared to standard prompts. We formalize this as Manifold Teleportation: expert signals act as high-dimensional navigators that counteract the model's tendency toward dimensional collapse during reasoning, maintaining activation trajectories in manifold regions with higher semantic density. Our findings provide a geometric foundation for prompt engineering and offer a new quantitative tool for LLM interpretability research -- understanding how prompts affect internal model computation by tracing EID trajectories.
Fields of Study & Subjects
Topic Modeling
Explainable Artificial Intelligence (XAI)
Domain Adaptation and Few-Shot Learning
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 29 January 2026
- Language
- EN
- Document type
- Preprint