Accidental Alignment Windows in Small Language Models: A Case Study of Creative Emergence in Gemma-7B
Abstract
This study documents and formalizes an unexpected empirical phenomenon observed in a locally deployed, alignment-constrained small language model (Gemma-7B). Despite exhibiting near-total conversational collapse and refusal-dominant behavior under ordinary dialogue, the model demonstrated coherent, genre-appropriate creative competence when reframed into bounded aesthetic tasks, including industrial metal lyric generation and evocative sonic world-building. Through direct interaction logs obtained via local inference (Ollama, Ubuntu 24.04), this paper introduces the concept of accidental alignment windows—task framings in which safety-driven refusal behaviors are deprioritized, allowing latent generative capacity to surface. A qualitative comparative appendix contrasts Gemma-7B with LLaMA-2 7B and Mistral-7B under identical creative framing, alongside a taxonomy of high-entropy aesthetic domains and a technical note reframing prompt engineering as alignment-aware interface design. The findings challenge conversational benchmarks as a proxy for general model capability and propose an alternative framework for evaluating and interfacing with small, heavily aligned language models in creative and exploratory contexts.
Fields of Study & Subjects
Topic Modeling
Speech and dialogue systems
Natural Language Processing Techniques
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 16 January 2026
- Document type
- Preprint