Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
Raihan Atsal Hafizh
Telkom University
Computer Science & IT Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
DETR-CLIP-BLIP Integrated Framework for Context-Guided Multimodal Knowledge Graph Retrieval Raihan Atsal Hafizh; Kemas Rahmat Saleh Wiharja
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 10 No 5 (2026): October 2026 (In progress)
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v10i5.7647

Abstract

Image search systems that rely on metadata tags or basic captions often miss the semantic relationships and visual details needed for complex queries. This study presents a multimodal knowledge graph framework that combines DETR for object detection, CLIP for attribute classification across 188 dimensions, and BLIP for caption generation. On the COCO 2017 validation set (5,000 images), the pipeline processed 4,906 images successfully, of which 4,243 contained the multiple objects required for relationship extraction. The resulting knowledge graph reached 99.98% relational coverage among multi-object images and 2.94 relationships per image, with 117 of the 188 attributes appearing in the data. Retrieval was evaluated under a self-consistency protocol relevance defined by the system's own top-ranked matches yielding an MRR of 0.96 for keyword-based queries and 0.29 for relationship-based queries, with Recall@50 of 88% for the latter. The framework runs on pretrained models without task-specific training and produces both structured triples and human-readable descriptions, supporting applications such as visual question answering, scene understanding, and context-aware image retrieval.
Co-Authors Kemas Rahmat Saleh Wiharja