Improving the Reliability and Accuracy of Image Captioning Systems Using Ensemble of FC, Softmax, and LSTM Deep Decoders

Read full text Summarise with VIRAT
Authors: Ghadeer Abdulrasool Mohammed, Raidah Salim Khudeyer, Maytham Alabbas
Cited by: 0
References: 32

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

In this work, a deep system for automatic image description is presented, which aims to produce fluent, meaningful, and structurally coherent sentences for input images. The proposed architecture is based on an encoder-decoder framework, in which high-level image features are first extracted by an Inception-v3 deep convolutional network and then fed as a compressed image representation to an LSTM-based language decoder to produce a word-by-word sentence. On this basic structure, a voting-based ensemble learning framework is designed, in which three deep paths, including a fully connected (FC) network, a Softmax linear model, and a sequence-oriented LSTM decoder, are trained independently, and the word probability vectors at the output level are combined with a maximum voting mechanism. The evaluation is performed on the standard Flickr8k database and using BLEU-1 to BLEU-4, METEOR, and ROUGE-L metrics. The results show that the best single LSTM model achieves values ​​of 0.64, 0.39, 0.23, and 0.16 for BLEU-1 to BLEU-4, and 0.22 and 0.50 for METEOR and ROUGE-L, respectively, while the Ensemble model improves the values ​​to 0.74, 0.50, 0.35, and 0.22 for BLEU-1 to BLEU-4, 0.475 for METEOR, and 0.55 for ROUGE-L; such that the relative improvements in BLEU-3 and BLEU-4 are 54% and 41%, respectively. The paired t-test also shows that the difference in Ensemble performance with single models is significant at the 95% confidence level, and compared to the existing methods on Flickr8k, competitive results are obtained and, in some measures, superior.

Fields of Study & Subjects

Multimodal Machine Learning Applications Image Retrieval and Classification Techniques Advanced Neural Network Applications

Publication details

Journal / source
Journal of Al-Qadisiyah for Computer Science and Mathematics
Volume
18
Issue
1
Pages
51–66
Publication date
30 March 2026
Language
EN