Rethinking Foundation Model Compute: The Jarvis Architecture for Infinite-Context Spiking Ternary Mixture-of-Experts
Abstract
The scaling laws of contemporary deep learning dictate an unsustainable trajectory of hardware consumption. Foundation architectures require massive clusters of high-bandwidth memory (HBM), constrained primarily by the $\mathcal{O}(N^{2})$ algorithmic complexity of standard softmax attention and the heavy memory footprint of continuous FLOAT32 parameterization. We propose the Jarvis Engine, a radically divergent architectural paradigm designed to democratize foundation-scale training on consumer hardware. By orchestrating Spiking Neural Networks (SNNs), extreme discrete Ternary Weight Quantization {-1,0,1}, sparse Mixture-of-Experts (MoE) routing, and an $\mathcal{O}(N)$ Infinite Associative Attention mechanism, the Jarvis Engine mathematically shatters current VRAM bottlenecks. We detail the resolution of nine critical failure modes inherent to uniting discontinuous ternary spaces with biological spiking dynamics. We present formal mathematical proofs for our discrete Straight-Through Estimators (STE) and Orthogonal Reflective Penalties. Theoretical bounds demonstrate a 16x compression in parameter volume and infinite sequence scaling without catastrophic gradient degradation or expert collapse.
Fields of Study & Subjects
Advanced Neural Network Applications
Advanced Memory and Neural Computing
Neural Networks and Applications
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 24 February 2026
- Language
- EN
- Document type
- Preprint