Granular Knowledge Mapping in Sparse Mixture of Experts

While large language models (LLMs) have scaled significantly through Sparse Mixture-of-Experts (MoE) architectures, interpretability remains a critical bottleneck. Unlike dense models, where all parameters are active for every token, MoE models like Mixtral 8x7B rely on dynamic routing mechanisms that select specific "experts" for specific tasks. This introduces a new layer of opacity: not only how a model reasons, but which sub-network is responsible for that reasoning. This paper introduces an enhanced Knowledge Provenance Mapping system specifically designed for MoE architectures. We present a novel methodology that: (1) tracks granular gradients across the Router, Attention, and Expert layers (w1, w2, w3) during Low-Rank Adaptation (LoRA) fine-tuning; (2) aggregates these signals into a 3D "Ribcage" visualization that topologically separates general reasoning (Attention Spine) from specialized knowledge (Expert Wings); and (3) introduces new metrics for Expert Utilization and Gini Concentration. This approach provides the first holistic, visual audit trail of how an MoE model routes domain-specific knowledge, distinguishing between generalist reasoning and expert specialization.