FourierNAT: a Fourier-mixing-based non-autoregressive transformer for parallel sequence generation
Abstract
We present FourierNAT, a novel non-autoregressive transformer (NAT) architecture that leverages Fourier-based mixing in the decoder to generate output sequences in parallel. While traditional NAT approaches often face challenges in capturing global dependencies, our method uses a discrete Fourier transform with learned frequency-domain gating to mix token embeddings across the entire sequence dimension. This design enables efficient propagation of context without explicit autoregressive steps. Empirically, FOURIERNAT achieves competitive results on WMT14 En-De and CNN/DailyMail benchmarks, highlighting that frequency-domain operations can mitigate coherence gaps often associated with NAT generation. Our approach underscores the potential of integrating spectral-domain operations to accelerate and improve parallel text generation.
Fields of Study & Subjects
Topic Modeling
Natural Language Processing Techniques
Digital Humanities and Scholarship
Publication details
- Journal / source
- International Journal of Complexity in Applied Science and Technology
- Publisher
- Inderscience Publishers
- Volume
- 2
- Issue
- 1
- Pages
- 63–75
- Publication date
- 1 January 2026
- Language
- EN