Transformers for Vision-to-Language: A Comparative Evaluation of Image Captioning Architectures, Datasets, and Metrics
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Image Captioning (IC) is defined as automatic generation of semantically and contextually coherent descriptions of image. Early CNN-RNN approaches captured multimodality features but failed to capture context and longrange dependencies. This study explores Image Captioning (IC) and Visual Question Answering (VQA) tasks through implementations. We considered seven transformer based models for the study and their performance was assessed. BLEU, ROUGE, METEOR, F1 Score and VQA accuracy are the evaluation metrics used across the benchmarking datasets and custom datasets. We studied the response on two scenarios - with ground truth and without ground truth. For without ground truth scenario, we obtained user feedback to evaluate the generated captions. Initially, we discussed primary structural and architectural components like encoderdecoder, attention techniques and model parameters of the selected models, as they directly influence the performance of the model. Subsequently, we investigated the operational dynamics through IC and VQA as they impact the image-text grounding and reasoning ability of the models. Finally, performance metric along with error analysis offered insights on how architectural choices affect reasoning and visionlanguage alignment. This survey provides pathway for the development and analysis of transformer-based IC and VQA models across datasets.
Fields of Study & Subjects
Multimodal Machine Learning Applications
Advanced Neural Network Applications
Image Retrieval and Classification Techniques
Publication details
- Journal / source
- International Conference on Intelligent Systems for Communication, IoT and Security (ICISCoIS)
- Pages
- 1–6
- Publication date
- 8 January 2026
- Language
- EN