Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control
Vol. 7, No. 4, November 2022

Image Captioning using Hybrid of VGG16 and Bidirectional LSTM Model

Yufis Azhar (Universitas Muhammadiyah Malang)
M. Randy Anugerah (Universitas Muhammadiyah Malang)
Muhammad Al Reza Fahlopy (Universitas Muhammadiyah Malang)
Alfin Yusriansyah (Universitas Muhammadiyah Malang)



Article Info

Publish Date
30 Nov 2022

Abstract

Image captioning is one of the biggest challenges in the fields of computer vision and natural language processing. Many other studies have raised the topic of image captioning. However, the evaluation results from other studies are still low. Thus, this study focuses on improving the evaluation results from previous studies. In this study, we used the Flickr8k dataset and the VGG16 Convolutional Neural Networks (CNN) model as an encoder to generate feature extraction from images. Recurrent Neural Network (RNN) uses the Bidirectional Long-Short Term Memory (BiLSTM) method as a decoder. The results of the image feature extraction process in the form of feature vectors are then forwarded to Bidirectional LSTM to produce descriptions that match the input image or visual content. The captions provide information on the object’s name, location, color, size, features of an object, and surroundings. A greedy Search algorithm with Argmax function and Beam-Search algorithm are used to calculate Bilingual Evaluation Understudy (BLEU) scores. The results of the evaluation of the best BLEU scores obtained from this study are the VGG16 model with Bidirectional LSTM using Beam Search with parameter K = 3 and the BLEU-1 score is 0.60593, so this score is superior to previous studies.

Copyrights © 2022






Journal Info

Abbrev

kinetik

Publisher

Subject

Computer Science & IT Control & Systems Engineering Electrical & Electronics Engineering Energy Engineering

Description

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control was published by Universitas Muhammadiyah Malang. journal is open access journal in the field of Informatics and Electrical Engineering. This journal is available for researchers who want to improve ...