CL5D Phase Dynamics for KV Cache Token Importance Scoring: A Theoretical Framework and Proof-of-Concept

Read full text Summarise with VIRAT
Authors: Mrinmoy Chakraborty
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

We apply the CL5D Hybrid Model to token-level compression in transformer languagemodels. Using the Cn score (At × Ab) as a measure of token importance, we prune 15/16of the tokens (Phase II) and then inject the information from the pruned tokens into theretained ones via a weighted sum (Phase IV). The injection strength is controlled by aglobal catalyst strength λ. On a single 48-token prompt with GPT-2 large, we compress to3 tokens (16× compression). The original perplexity is 19.46. At λ = 0.5, the compressedperplexity reaches 4146.91, a 213× increase. The U-shaped curve confirms the existence ofa conjugate balance point, where the injected information optimally compensates for thepruned tokens. This proof-of-concept demonstrates that CL5D phase dynamics can serveas a principled framework for KV cache compression, with the optimal injection strengthreducing perplexity by 22% compared to no injection.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Advanced Neural Network Applications

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
30 March 2026
Language
EN