Deep Layer Expansion: Expert Prompts Counteract Dimensional Collapse in Large Language Models

Read full text
Authors: Lei Zhao
Cited by: 0

Abstract

Large Language Models (LLMs) exhibit systematic performance improvements when prompts contain expert-level domain signals. We investigate the geometric mechanism underlying this phenomenon through controlled experiments on two mainstream 70B-class open-source instruction-tuned models. Contrary to the conventional understanding that deep layers compress representations toward deterministic outputs, we discover a striking universal phenomenon: expert signals induce "Deep Layer Expansion" in the representation space. Specifically, expert-level prompts increase the Effective Intrinsic Dimension (EID) in deep layers (Layer 60+) by 60-100% compared to standard prompts. We formalize this as Manifold Teleportation: expert signals act as high-dimensional navigators that counteract the model's tendency toward dimensional collapse during reasoning, maintaining activation trajectories in manifold regions with higher semantic density. Our findings provide a geometric foundation for prompt engineering and offer a new quantitative tool for LLM interpretability research -- understanding how prompts affect internal model computation by tracing EID trajectories.

Fields of Study & Subjects

Topic Modeling Explainable Artificial Intelligence (XAI) Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
29 January 2026
Language
EN
Document type
Preprint