TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks

Lightweight Vision-Language Models (VLMs) are indispensable for resource-constrained applications.The prevailing lightweight recipe freezes both the vision encoder and the language model while training only a small connector.While efficient, this strategy relies heavily on the intrinsic representational capacity of the language model and can become suboptimal for compact backbones.In this work, we study this alignment bottleneck through an information-theoretic lens and use Effective Mutual Information (EMI) as a guiding intuition for the amount of multimodal information that a frozen lightweight language model can effectively exploit.Motivated by this perspective, we propose TinyAlign, a retrieval-augmented framework that retrieves relevant context from a memory bank constructed from training data and injects compressed multimodal cues to ease alignment.Extensive experiments show that TinyAlign consistently reduces training loss, accelerates convergence, and improves downstream performance with negligible computational overhead.TinyAlign is also highly dataefficient, reaching baseline-level performance with only 40% of the fine-tuning data.These results provide both a practical recipe for stronger lightweight VLMs and an informative perspective on alignment bottlenecks in constrained multimodal systems. Vision Encoder Embedding LayerPerceiver