Geodesic Attention Engine (GAE): Minimum-Energy Path Through Transformer Attention

GAE (Geodesic Attention Engine) is a fused attention implementation that finds the minimum-energy path through transformer attention computation. Standard attention implementations perform 12 separate HBM (High-Bandwidth Memory) round-trips per attention operation. GAE fuses the entire operation—Q, K, V projection, softmax, and output—into a single kernel that touches HBM only twice: once to load, once to store. Key results on NVIDIA H100: - 83% reduction in HBM round-trips (12 → 2) - O(N) memory complexity vs O(N²) standard - 23-37% improvement in Tokens per Joule - Bit-exact deterministic output across runs Technical approach: - Online Softmax for single-pass computation - Welford Streaming Statistics for numerical stability - Register-Level Fusion keeping all intermediates in GPU registers GAE computes exact attention—not an approximation, not sparse. Every query attends to every key, just with fewer memory operations. CUDA and Rust implementations included.