Fractal Tensor Compression for Large Language Models: A Theoretical and Experimental Proposal
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This paper proposes a framework for fractal compression of tensors in large language models(LLMs), targeting both offline weight storage and online key–value (KV) cache compressionunder strict quality constraints. The method generalizes classical fractal image compression—based on iterated function systems (IFS), domain–range block matching, and contractive affinetransforms—to high-dimensional tensors used in transformer architectures.[1, 2] Unlike vectorquantization schemes that focus on near-optimal scalar or vector quantization of activations andKV caches, the fractal approach explicitly exploits self-similarity across positions and features,enabling potentially very high compression ratios at the cost of more expensive encoding.[3] Weformalize the tensor setting, define a family of contractive mappings on tensor blocks, describefixed-point reconstruction, and analyze compression ratio, approximation error, and compu-tational complexity at a conceptual level. We then report preliminary empirical results onQwen 3.5–0.8B that demonstrate promising offline compression ratios and KV-cache speedupsunder tight quality guardrails.
Fields of Study & Subjects
Advanced Data Compression Techniques
Advanced Neural Network Applications
Algorithms and Data Compression
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 28 March 2026
- Language
- EN