Muhammad Rafi Ramzi
Telkom University

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

Academic Chatbot for Campus Information Services Using Retrieval-Augmented Generation Haddad Alwi Yafie; Achmad Udin Zailani; Widang Muttaqin; Muhammad Sheva Atallah Daffansyah; Muhammad Rafi Ramzi
Sinkron : jurnal dan penelitian teknik informatika Vol. 10 No. 3 (2026): Article Research July 2026
Publisher : Politeknik Ganesha Medan

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33395/sinkron.v10i3.16088

Abstract

University service centers handle many repetitive queries about academic schedules, registration, and policies stored in internal documents. Manual lookup is inefficient, and answers given by staff can be inconsistent. Rule-based chatbots only handle limited question patterns, while large language models are hard to update and may produce unsupported answers (hallucinations). This research designs an academic chatbot that combines document retrieval with answer generation so that each answer remains traceable to its source. The system extracts text from campus documents, segments it, encodes it using a multilingual embedding model, and stores it in a vector index for context retrieval. A response is generated through an instruction template that confines the output to the retrieved information and includes page references. Evaluation followed a mixed-method design: a quantitative layer measured retrieval quality (Precision@5, Recall@5) and generation quality using the four RAGAS sub-metrics (faithfulness, answer_relevancy, context_precision, context_recall) on a 100-question test set, while a qualitative layer applied thematic analysis to open-ended user comments. Statistical testing used McNemar's test for accuracy and a paired bootstrap (10,000 resamples) for retrieval metrics; 95% confidence intervals are reported. Results: the proposed RAG system achieved 84% answer accuracy (95% CI 76–90%), Precision@5 = 0.80 and Recall@5 = 0.72, with a System Usability Scale (SUS) score of 78 and a Net Promoter Score (NPS) of +32 from 30 participants. Differences in accuracy versus the lexical and LLM-only baselines were statistically significant (McNemar p < 0.05). The system offers a replicable instantiation of RAG for transparent, citation-backed campus information services in Indonesian.