Saifu Rohman
Informatics Engineering Study Program, Faculty of Engineering and Computer Science, Universitas Sains Al Qur’an Jawa Tengah di Wonosobo, Indonesia

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

A RAG and Reciprocal Rank Fusion-Based Chatbot for New Student Admission (SPMB) Information Services at SMK Muhammadiyah 1 Wonosobo Iqbal Ali Ar-Ridho; Nahar Mardiyantoro; Saifu Rohman
Jurnal Teknik Indonesia Vol. 5 No. 01 (2026): Jurnal Teknik Indonesia (JU-TI) 2026
Publisher : SEAN Institute

Show Abstract | Download Original | Original Source | Check in Google Scholar

Abstract

The New Student Admission (SPMB) information service at SMK Muhammadiyah 1 Wonosobo faces a high volume of inquiries—987 registrants were recorded in 2025—handled through seven official channels limited to working hours, resulting in repetitive questions and inconsistent answers. This study aims to develop a Natural Language Processing (NLP)-based chatbot using the Retrieval-Augmented Generation (RAG) and Reciprocal Rank Fusion (RRF) methods to automate the SPMB information service on the school’s official website. The system was built using the Prototype development method. The designed RAG pipeline adds three retrieval-enhancement mechanisms on top of naive RAG, namely query expansion, RRF with an asymmetric 4:1 weighting, and cross-encoder reranking, with Gemini 3 Flash Preview as the answer generator and a knowledge base of 44 official school text documents. Testing was conducted through Black Box Testing and RAGAS evaluation measuring faithfulness, answer relevancy, and context precision. Black Box Testing of 17 scenarios achieved a 100% functional success rate. RAGAS evaluation on the full (Enhanced) configuration obtained faithfulness 0.9812, answer relevancy 0.9146, and context precision 0.7991. Compared to the Baseline configuration, answer relevancy increased by 12.40% and context precision by 14.16%, while faithfulness remained in the “Very Good” category. This study concludes that applying these three techniques positively contributes to document relevance and the answer quality of the SPMB information chatbot.