Constellation-Indexed Model Composition: Query-Driven Parameter Mixing via Activation Fingerprints
Abstract
Dynamically composes specialist language models at the parameter level using activation fingerprints and orthogonal decomposition. Generalist-space indexing eliminates alignment failures (20.6% to 98.1% win rate). Gram-Schmidt orthogonalization of domain centroids reduces raw cosine similarity of 0.91–0.97 to ~0. Stochastic resonance at sigma=0.020 rescues composition at 7B (+90.7 points). Architecture-invariant across GPT-2 and Qwen families.
Fields of Study & Subjects
Topic Modeling
Domain Adaptation and Few-Shot Learning
Advanced Neural Network Applications
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 1 March 2026
- Language
- EN