Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
Edward Ferdian
Telkom University
Computer Science & IT Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Leveraging Assistive Image Captioning on VizWiz through Multi-Caption Supervision Fauzan Firdaus; Ema Rachmawati; Angel Metanosa Afinda; Edward Ferdian; Muhammad Maulana Yusuf
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 10 No 5 (2026): October 2026 (In progress)
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v10i5.7472

Abstract

Most image captioning datasets provide several human captions for the same image, yet model training is frequently carried out in a way that effectively treats each image as having only one caption. Our study revisits this mismatch by examining the impact of training with multiple captions per image on caption generation performance. For this purpose, we utilized the VizWiz-Captions dataset, which reflects a real assistive scenario with images taken by blind or visually impaired users with five reference captions per image. We evaluated two conventional CNN-based encoder–decoder models, namely CNN–LSTM and CNN–GRU. In both models, the visual encoder was fixed using InceptionV3 image features, while the recurrent decoder was varied between LSTM and GRU. We performed experiments by using training data by varying the number of captions used per image from one to five (k = 1–5), while keeping the overall pipeline unchanged. The models were evaluated using standard multi-reference metrics (BLEU-1 to BLEU-4, CIDEr, and SPICE). Across both baselines, performance generally improves when more captions are included in training, although the gains are not perfectly monotonic for every metric. The strongest results are observed at k = 5. For CNN–LSTM, the best setting achieved BLEU-4 = 0.175, CIDEr = 0.206, and SPICE = 0.120. On the other hand, CNN–GRU achieved BLEU-4 = 0.192, CIDEr = 0.226, and SPICE = 0.119. These findings suggest that, even without architectural changes, simply leveraging the available caption diversity during training can provide a practical boost for VizWiz-style captioning tasks.
Co-Authors Angel Metanosa Afinda Ema Rachmawati Fauzan Firdaus Muhammad Maulana Yusuf