Transformers for Vision-to-Language: A Comparative Evaluation of Image Captioning Architectures, Datasets, and Metrics

View at publisher Summarise with VIRAT
Authors: Dharani R, Kavin Karthic M, Kanishka D, A Rithanya, Subiksha MV, A. Baskar
Cited by: 0
References: 19

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Image Captioning (IC) is defined as automatic generation of semantically and contextually coherent descriptions of image. Early CNN-RNN approaches captured multimodality features but failed to capture context and longrange dependencies. This study explores Image Captioning (IC) and Visual Question Answering (VQA) tasks through implementations. We considered seven transformer based models for the study and their performance was assessed. BLEU, ROUGE, METEOR, F1 Score and VQA accuracy are the evaluation metrics used across the benchmarking datasets and custom datasets. We studied the response on two scenarios - with ground truth and without ground truth. For without ground truth scenario, we obtained user feedback to evaluate the generated captions. Initially, we discussed primary structural and architectural components like encoderdecoder, attention techniques and model parameters of the selected models, as they directly influence the performance of the model. Subsequently, we investigated the operational dynamics through IC and VQA as they impact the image-text grounding and reasoning ability of the models. Finally, performance metric along with error analysis offered insights on how architectural choices affect reasoning and visionlanguage alignment. This survey provides pathway for the development and analysis of transformer-based IC and VQA models across datasets.

Fields of Study & Subjects

Multimodal Machine Learning Applications Advanced Neural Network Applications Image Retrieval and Classification Techniques

Publication details

Journal / source
International Conference on Intelligent Systems for Communication, IoT and Security (ICISCoIS)
Pages
1–6
Publication date
8 January 2026
Language
EN