LaminarNet: Linear-Time Multi-Scale State Propagation for Efficient Language Modeling

Read full text Summarise with VIRAT
Authors: Ugurhan Colak
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Transformer architectures dominate modern language modeling but incur O(N²) computational and memory costs with respect to sequence length, limiting scalability. We introduce LaminarNet, a linear-time architecture that replaces quadratic self-attention with two key mechanisms: Geometric Drift Field (GDF) for selective state propagation and Cross-Stratum Routing (CSR) for multi-scale hierarchical token interaction. LaminarNet achieves significant improvements in efficiency and performance. In a parameter-matched benchmark (49M parameters), it reduces perplexity by 38.3%, increases throughput by 63.3%, and reduces VRAM usage by 38.8% compared to Transformers. Additionally, a 437M-parameter model trained on 9.56B Turkish tokens demonstrates strong scaling behavior, achieving a perplexity of 11.38. Code: https://github.com/Uunan/LaminarNet Model: https://huggingface.co/Uunan/LaminarNet_437M_Turkish_Base

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Advanced Neural Network Applications

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
18 March 2026
Language
EN
Document type
Preprint