T³: Toroidal Tesseract Transformer - Geometric Optimization for Deep Learning
Abstract
We present T3 (Toroidal Tesseract Transformer), a framework that respects the geomet- ric structure of neural network weight space. Standard optimizers treat weight space as Euclidean, but we demonstrate that emergent structure follows geometric invariants, and respecting this geometry yields significant improvements. We introduce three components: a geodesic optimizer that navigates a 3-torus man- ifold (32–38% perplexity improvement, p < 0.001), a chain architecture that preserves geometry at scale (7–33% improvement, p < 0.0001), and adaptive attention via E,I,F,V primitives. We validate our approach across 75 experimental conditions, showing that the nearest-neighbor (NN) invariant holds at NN ≈ 1.0 (CV = 12.5%). This invariant caught a critical +0.1 bug that was degrading performance by 30%. We report both successes and the training recipe that enables them: primitives require staged training (freeze base → train heads → fine-tune). With staged training, primitives achieve +22–30% perplexity improvement (p < 0.02) on WikiText and C4. End-to- end training from scratch fails catastrophically. The same mechanism yields opposite results depending on training procedure
Fields of Study & Subjects
Topic Modeling
Stochastic Gradient Optimization Techniques
Neural Networks and Applications
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 31 January 2026
- Document type
- Preprint