A Geometric Attention Transformer via the Leech Lattice: Leech-LILA (Lie Lattice LM)

We introduce Leech-LILA, a Transformer architecture that replaces the standard learnable query/key projections with a fixed orthogonal matrix derived from the Leech lattice — the densest sphere packing in 24 dimensions. This frozen geometric core acts as a high-dimensional symmetry filter, guiding hidden representations toward lattice nodes and preventing attention collapse. A quantized resonance loss further enforces alignment with the lattice basis, acting as an anti-hallucination regularizer. The model is designed with a block-diagonal structure, partitioning the latent space into independent 24-dimensional “semantic cells”. The approach is universal and can be applied to any modality; the source code is released under AGPLv3.