Lorentz-Manifold Transformers: A Geometric–Dynamical Framework for Hierarchical Representation Learning

Standard Transformer architectures operating in Euclidean space suffer from a fundamental Geometric Mismatch when modeling data with intrinsic hierarchical or cyclic structures, leading to suboptimal Out-of-Distribution (OOD) generalization. Embedding exponentially growing hierarchies into polynomial Euclidean volumes results in severe distortion and "representation crowding." In this work, we propose the Lorentz-Manifold Transformer, a neural architecture designed to maximize Manifold Capacity through three geometric inductive biases: Lorentzian Hyperbolic Embeddings: Utilizing the Lorentz (Hyperboloid) model ($\mathbb{L}^n$) to accommodate exponential hierarchical growth with superior numerical stability compared to the Poincaré ball. Oscillatory Gating mechanism: Introducing Artificial Kuramoto Oscillatory Neurons (AKORN) to enforce temporal feature coherence and filter incoherent noise via synchronization dynamics. Riemannian Optimization: employing Structured K-FAC (SK-FAC) to align learning trajectories with the manifold's intrinsic curvature. Furthermore, we introduce a novel training objective based on the Sliced Gromov-Wasserstein (SGW) distance, which aligns the neural state space with a target topological prior without requiring paired coordinates. Empirical results on synthetic topology recovery tasks (e.g., the "Color Ring" problem) demonstrate that the Lorentz-Manifold Transformer recovers persistent topological features ($\beta_1$ cycles) where Euclidean baselines fail, offering a rigorous framework for structurally aligned Artificial Intelligence.