ALTAMBA: Alternating Mamba with Peristaltic Normalization
Abstract
ALTAMBA is a dual-path Transformer+SSM architecture that achieves 8-9% validation loss improvement over parameter-matched Jamba baselines through learnable destructive interference and alternating Post-LN/Pre-LN normalization.
Fields of Study & Subjects
Advanced Neural Network Applications
Domain Adaptation and Few-Shot Learning
Topic Modeling
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 8 February 2026
- Language
- EN
- Document type
- Preprint