Regi Fernando Najoan
Universitas Klabat, Minahasa Utara

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Sistem Lost and Found Berbasis Vision-Language Retrieval dengan Explainable AI untuk Lingkungan Smart Campus Enrico Samuel Djimesha; Regi Fernando Najoan; Argha Orion Silitonga; Wilsen Grivin Mokodaser; George Morris William Tangka
Journal of Information System Research (JOSH) Vol 8 No 1 (2026): October 2026
Publisher : Forum Kerjasama Pendidikan Tinggi (FKPT)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47065/josh.v8i1.10990

Abstract

Evidence for combined image-text retrieval in campus lost-and-found systems remains limited, and zero-shot CLIP's ability to distinguish item attributes and identities is uncertain. This study evaluates CLIP ViT-B/32 on an item-image collection covering eight categories. The controlled multimodal evaluation uses 404 images, 85 metadata-defined relevance groups as proxy ground truth, and 237 leave-one-out queries. Separate image-only validation uses 387 images from 79 provenance-based Object IDs after duplicate removal and quality control; each Object ID contains multiple images of the same physical item. In Object-ID validation, CLIP achieved Recall@1 of 0.674, Recall@10 of 0.935, and mAP of 0.419 (95% object-cluster bootstrap CI: 0.369–0.469). In multimodal evaluation, image-only achieved mAP 0.624, text-only 0.409, and image+text reached the best mAP of 0.732 at α=0.5, selected in four of five group cross-validation folds. Ablation showed that the single fused-index outperformed the three-index configuration (mAP 0.715 versus 0.692); therefore, three-index is not claimed as an accuracy improvement. Direct Indonesian input to English CLIP achieved mAP 0.163, compared with 0.363 using multilingual CLIP and 0.409 through translation. Attention Rollout is positioned only as a supporting visualization because its deletion-curve AUC of 0.871 lacks a random-occlusion baseline. These results indicate the benefit of image-text fusion under controlled evaluation but do not establish performance for free-form user descriptions or production readiness.