Enhancing Translation with Multimodal LLMs: Is Visual Information Still Beneficial?

Machine translation (MT) for Indic languages faces two persistent challenges-insufficient parallel data and semantic ambiguity. This work investigates whether multimodal large language models (LLMs) can alleviate both by incorporating image context alongside text. We fine-tune LLaVA-NeXT (Mistral-7B), a Vision language instruction-tuned model, on minimal multilingual data to translate English captions into Hindi and Bengali. Two setups are compared: multimodal fine-tuning (image + text input) and text-only fine-tuning under identical configurations. Evaluation using cross-entropy loss, BLEU, and chrF shows that even with only two training examples per language, multimodal fine-tuning consistently outperforms its text-only counterpart. Visual cues help disambiguate polysemous words such as bank and bat, improving contextual fidelity and lexical precision. These results demonstrate that multimodal grounding even with microscale data-enhances translation robustness and generalization in Indic language settings. Our study contributes a reproducible framework for vision augmented translation and insights into the emerging role of multimodal LLMs in multilingual NLP.