Softmax Attention Collapse via Leakage Bounds in Stochastic Block Models

This technical note presents a leakage-based analysis of attention collapse in softmax mechanisms operating over structured neighborhoods modeled as stochastic block models (SBMs). Leakage is defined as the total softmax mass assigned to non-dominant blocks. Under a mean-gap dominance condition exceeding sub-Gaussian noise fluctuations and a logarithmic block-size correction, total leakage decays exponentially. As a result, attention concentrates on a dominant block with near-uniform intra-block weights, effectively reducing attention to a block-averaging operator. The analysis relies only on order-statistics stability and log-sum-exp dominance and does not invoke training dynamics, metric entropy, or chaining arguments. Empirical simulations demonstrate a sharp phase transition when the hypothesis holds and smooth pre-collapse behavior under partially aligned competitors. Explicit scope conditions and failure modes are identified.