Wave-Attractor-Tree: A Hierarchical Binary Tree Reduction Architecture for Efficient Sequence Modeling
Abstract
Work introduces a hierarchical binary tree-based reduction that replaces standard self-attention. The core idea is to use a recursive Gated Linear Unit merge operation, achieving O(n) total merge operations O(log n) parallel depth O(n d^2) total work and O(n) space complexity. In these experiments, the model significantly outperforms standard Transformers in both convergence speed and accuracy on long-range structural dependencies, specifically where hierarchical inductive bias is critical.
Fields of Study & Subjects
Topic Modeling
Advanced Neural Network Applications
Natural Language Processing Techniques
Publication details
- Journal / source
- arXiv (Cornell University)
- Publication date
- 28 February 2026
- Language
- EN
- Document type
- Preprint