IJISTECH (International Journal Of Information System & Technology)
Vol 5, No 3 (2021): October

Image Captioning Approach for Household Environment Visual Understanding

Dhomas Hatta Fudholi (Department of Informatics, Universitas Islam Indonesia, Yogyakarta, Indonesia)



Article Info

Publish Date
30 Oct 2021

Abstract

Image captioning task generates a description from an image in a form of a sentence. It holds various critical usage in different applications and domains, such as indexing in retrieval system, by capturing semantic information within the images. In the area of live quality support system, an image captioning model may be used by visual impaired people to achieve visual understanding of their surroundings. In this paper, we present a novel image captioning model that aims to give visual understanding on household environment. To develop the model, we use five different household objects (sinks, chairs, tables, beds, couches) from MS COCO datasets. We create three new captions, in Bahasa Indonesia, for the selected data. The captions describe the name, the color, the position/location, the size, and the type/characteristic of the related object and its close surrounding. InceptionV3 and LSTM architecture is used to train the model with GloVe as the word embedding. In this study, our developed image captioning model can generate caption well and achieved BLEU-1 score of 0.502033, BLEU-2 score of 0.312539, BLEU-3 score of 0.193333, BLEU-4 score of 0.106111, METEOR score of 0.183193, ROUGE-L score of 0.358339, and CIDEr score of 0.348903.

Copyrights © 2021






Journal Info

Abbrev

ijistech

Publisher

Subject

Computer Science & IT

Description

IJISTECH (International Journal Of Information System & Technology) is published with both online and print versions. The journal covers the frontier issues in the computer science and their applications in business, industry and other subjects. The computer science is a branch of engineering ...