Nonlinear Attention via Information Phase Transition on Semantic Topology A Topological Collapse-Based Alternative to Quadratic Self-Attention

This paper proposes a nonlinear attention mechanism based on semantic topology and information phase transition to address the fundamental problems of O(N²) computational complexity and high power consumption inherent in existing self-attention mechanisms typified by Transformers. Instead of treating input information through numerical similarity, the method models it as a topological structure with semantic connectivity, enabling the collective extraction of important information via density and curvature analysis without exhaustive pairwise comparisons. As a result, computational complexity is reduced to linear or sub-linear time, allowing contextual convergence without reliance on gradient-based learning. This approach enables low-power large-context understanding and opens a pathway toward on-device AI.