<![CDATA[Academic service information is often stored in lengthy, formal, and distributed university documents, making accurate access difficult. Generative chatbots can improve access but may answer unsupported questions unless retrieval and refusal are controlled. This study proposes a reliability-oriented Retrieval-Augmented Generation (RAG) chatbot for Indonesian academic services integrating BGE-M3, FAISS, adaptive refusal thresholding, rule-based guardrails, Qwen2.5, and retry-on-refusal. A controlled experiment used 12 official documents and 160 Indonesian questions across eight categories. The final configuration achieved Precision@5 of 0.7250, Recall@5 of 0.9125, MRR of 0.8940, nDCG@5 of 0.8888, and mean semantic similarity of 0.8516. All 16 expected-refusal questions were refused and none of the 144 answerable questions was falsely refused, yielding observed refusal accuracy of 1.0000 and a false refusal rate of 0.0000. Retry-on-refusal reduced the false refusal rate from 0.0556 to 0.0000 and increased answerable-only similarity from 0.8150 to 0.8392. These results provide initial evidence that combining retrieval confidence, guardrails, and retry supports more reliable behavior within the controlled benchmark. Because the refusal subset was small and institution-specific, larger evaluations using naturally occurring student queries are required before inferring operational reliability.]]>
Copyrights © 2026
