Rethinking Foundation Model Compute: The Jarvis Architecture for Infinite-Context Spiking Ternary Mixture-of-Experts

Read full text
Authors: Parth Patil
Cited by: 0

Abstract

The scaling laws of contemporary deep learning dictate an unsustainable trajectory of hardware consumption. Foundation architectures require massive clusters of high-bandwidth memory (HBM), constrained primarily by the $\mathcal{O}(N^{2})$ algorithmic complexity of standard softmax attention and the heavy memory footprint of continuous FLOAT32 parameterization. We propose the Jarvis Engine, a radically divergent architectural paradigm designed to democratize foundation-scale training on consumer hardware. By orchestrating Spiking Neural Networks (SNNs), extreme discrete Ternary Weight Quantization {-1,0,1}, sparse Mixture-of-Experts (MoE) routing, and an $\mathcal{O}(N)$ Infinite Associative Attention mechanism, the Jarvis Engine mathematically shatters current VRAM bottlenecks. We detail the resolution of nine critical failure modes inherent to uniting discontinuous ternary spaces with biological spiking dynamics. We present formal mathematical proofs for our discrete Straight-Through Estimators (STE) and Orthogonal Reflective Penalties. Theoretical bounds demonstrate a 16x compression in parameter volume and infinite sequence scaling without catastrophic gradient degradation or expert collapse.

Fields of Study & Subjects

Advanced Neural Network Applications Advanced Memory and Neural Computing Neural Networks and Applications

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
24 February 2026
Language
EN
Document type
Preprint