Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : Journal of Intelligent Computing and Health Informatics (JICHI)

Evaluation of a Semantic Representation-Based Retrieval Model on a Text Dataset Generated from Image Transformation Firmansyah, Muhammad; Marutho, Dhendra; Ilham, Ahmad; Saputra, Irwansyah
Journal of Intelligent Computing & Health Informatics Vol 6, No 2 (2025): September
Publisher : Universitas Muhammadiyah Semarang Press

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.26714/jichi.v6i2.19240

Abstract

The increasing demand for efficient multimodal information retrieval has driven significant research into bridging visual and textual data. While sophisticated models like CLIP offer state-of-the-art semantic alignment, their substantial computational requirements present challenges for deployment in resource-constrained environments. This study introduces a lightweight retrieval framework that leverages the BLIP image captioning model to transform image data into rich textual descriptions, effectively reframing cross-modal retrieval as a text-to-text task. We systematically evaluated three retrieval models BM25, SBERT, and T5 on caption-transformed MSCOCO and Flickr30K datasets, utilizing both classical metrics (Recall@5, mAP) and semantic-aware metrics (SAR@5, Semantic mAP). Experimental results demonstrate that T5 achieves superior semantic performance (SAR@5 = 0.561, Semantic mAP = 0.524), surpassing SBERT (SAR@5 = 0.524) and outperforming the lexical BM25 baseline (SAR@5 = 0.312). Notably, the proposed BLIP+T5 pipeline attains 88% of CLIP’s semantic accuracy while reducing inference latency by approximately 60% and decreasing GPU memory consumption by over 60%. These findings underscore the potential of caption-based retrieval frameworks as scalable, cost-effective alternatives to computationally intensive multimodal systems, especially in latency-sensitive and resource-limited scenarios. Future work will explore fine-tuning strategies, domain-adapted semantic metrics, and robustness under real-world conditions to further advance retrieval effectiveness.
Co-Authors A, Oky Yudi Abdurrahman, Muhammad Farhan Abiyah, Indra Adib Adhitya Bagus Singandaru Adibrata, Sudirman Ainul Yaqin Ali Rahman Amri Gunasti Amrillah Apriani, Isna Apriyandi, Putra Arif Maulana, Arif Aslimah, Aslimah Aulia, Nur Afti Az Zahra, Alifia Azizah, Nur Jannah Bagus, Wahyu Bhaskara, Renaldi Surya Chairul Abdi Dhendra Marutho Dyo Arga Nata, Muhamad EMI SALMAH Eri Sasmita Susanto Farizki, Daniel Achmad Fikri, Sultoni Firdausy, Muhammad Abrar Firmansyah, Esa Frans Simangunsong Gunawan, Bambang Ady Haq, Ahmad Khoir Al hartanto hartanto Hendarti, Hendarti Herdinov, Adam Hidayat, Ali Akbar Hidayat, Lutfi Alvian Hikmawati, Inayah Himawan Sutanto, Himawan Ika Kartika Ilham, Ahmad Imroatul Khasanah Irhash, Irhash Irwan Irwan Ismaili, Ismaili Iwan Harsono Jati, Dian Rahayu Karim, M. Ainul KARISMAWAN, PUTU Khairunisa, Assavira L, Adinda Fitriyah Laily Purnamasari, Dewi M. Irwan M. Irwan, M. Irwan Mahmudi, Hadi Maria, Linda Masrun Masrun masrun Maulana Al Aby, Theo Mauliddin, Rizky Akbar Maulidia, Vera Mizwar, Andi Mochammad Machlul Alamin MUAIDY YASIN, MUAIDY Muftiadi, M. Rizza Muhammad Agreindra Helmiawan Muhammad Aria Wahyudi Muhammad Sri Wahyudi Suliswanto Muin, Suriati Abd Musyoliha, Tuti Muzakar, Muzakar Nasir Hamzah Nilawati, Indah Nirtha Nofitasari, Solehati Nopi Stiyati Prihatini Nova, Kiky Rizky Nugraha, Nanda Nugraha, Thomas Fajar Nur Tjendani, Endah Nur, Alqadri Nuraulia, Nasyah Octavia, Azella Nosih Paramita Sari, Paramita Paratama, Faradillah Perangin-angin, Robet Prabowo, Ien Yus Rizal Prawira, Nurul Adi Prayitno Basuki, Prayitno Purnama, Isnan Purwanti, Irna Putra, Surya Dwi Qurrotul Aini Rahman, Ali Rasyidin, Mukhammad Hafid Rianto, Yan Rijal, Muhammad Nor Rishan Adha Riza Miftahul Khair Rizqi Puteri Mahyudin S, Raja Eko Dwi SABINA, VINA Sadiq, Ahmad Safitri, Nur Aprilia Sahri Sahri Sahri Saputra, Irwansyah Sari, Nova Puspita Savira, Aora Septiyani, Santi Sikin, Muhamad St. Maryam, St. Subhan Purwadinata, Subhan Supardi, Sudarman Suphia, Suphia T, Thoyyibah Tanjung , Yanto Titi Yuniarti, Titi Ul Walidaien, Syifa Ulfa, Yakut Nahdiana Ulumuddin, Mochammad Fatih Wahidin Wahidin Wahyuni , Hera Watono Widari, Baiq Widyanor, Julian Yahya, Helmi Zara Tania Rahmadi