Indonesian Journal of Electrical Engineering and Computer Science
Vol 21, No 2: February 2021

Bangla language textual image description by hybrid neural network model

Md. Asifuzzaman Jishan (Department of Computer Science and Engineering, University of Liberal Arts Bangladesh (ULAB))
Khan Raqib Mahmud (Department of Computer Science and Engineering, University of Liberal Arts Bangladesh (ULAB))
Abul Kalam Al Azad (Department of Computer Science and Engineering, University of Liberal Arts Bangladesh (ULAB))
Mohammad Rifat Ahmmad Rashid (Department of Computer Science and Engineering, University of Liberal Arts Bangladesh (ULAB))
Bijan Paul (Department of Computer Science and Engineering, University of Liberal Arts Bangladesh (ULAB))
Md. Shahabub Alam (Department of Computer Science and Engineering, Ahsanullah University of Science and Technology)



Article Info

Publish Date
01 Feb 2021

Abstract

Automatic image captioning task in different language is a challenging task which has not been well investigated yet due to the lack of dataset and effective models. It also requires good understanding of scene and contextual embedding for robust semantic interpretation of images for natural language image descriptor. To generate image descriptor in Bangla, we created a new Bangla dataset of images paired with target language label, named as Bangla Natural Language Image to Text (BNLIT) dataset. To deal with the image understanding, we propose a hybrid encoder-decoder model based on encoder-decoder architecture and the model is evaluated on our newly created dataset. This proposed approach achieves significance performance improvement on task of semantic retrieval of images. Our hybrid model uses the Convolutional Neural Network as an encoder whereas the Bidirectional Long Short Term Memory is used for the sentence representation that decreases the computational complexities without trading off the exactness of the descriptor. The model yielded benchmark accuracy in recovering Bangla natural language and we also conducted a thorough numerical analysis of the model performance on the BNLIT dataset.

Copyrights © 2021