ILKOM Jurnal Ilmiah
Vol 18, No 1 (2026)

A Hybrid BERT–RAG Model for Developing Knowledge-Validated Conversational Systems

Anggreani, Desi (Unknown)
Ismawati, Ismawati (Unknown)
Auliyah, A. Inayah (Unknown)
Lukman, Lukman (Unknown)
Rahman, Aedah Abd (Unknown)
Nurmisba, Nurmisba (Unknown)
Akbar, Muh Ilham (Unknown)



Article Info

Publish Date
20 Apr 2026

Abstract

The transition of freshmen into the university environment requires adaptive and responsive information support. This study develops a chatbot system based on a hybrid BERT–RAG architecture integrated with the FAISS Index to provide automated consultation services for new students. The novelty of this research lies in the implementation of a faculty-based hierarchical knowledge structure and an adaptive multi-domain context mechanism—an approach not previously found in studies involving BERT–RAG for university onboarding services. This design enables the chatbot to deliver more relevant, personalized, and faculty-specific responses. The dataset was derived from three primary sources of information: the Faculty of Economics and Business (FEB), the Faculty of Teacher Training and Education (FKIP), and the Faculty of Engineering (FT), which were structured into a validated knowledge base in documents.json format. System evaluation was conducted across ten interaction scenarios using performance metrics including BERT Similarity, BLEU Score, ROUGE-1, ROUGE-2, and ROUGE-L. The system achieved excellent results, with average scores of 0.905 (BERT Similarity), 0.844 (BLEU), 0.876 (ROUGE-1), 0.820 (ROUGE-2), and 0.871 (ROUGE-L) and standard deviations below 0.1 across all metrics. Strong metric correlations (0.85–0.99) further indicate consistency between semantic understanding and generated text quality. Furthermore, the system effectively minimizes hallucination through validated knowledge integration and faculty-based reranking strategies. Overall, this research provides a significant contribution to the development of institutionally contextual educational chatbots capable of delivering accurate, natural, and responsive communication to support new student orientation in higher education

Copyrights © 2026






Journal Info

Abbrev

ILKOM

Publisher

Subject

Computer Science & IT

Description

ILKOM Jurnal Ilmiah is an Indonesian scientific journal published by the Department of Information Technology, Faculty of Computer Science, Universitas Muslim Indonesia. ILKOM Jurnal Ilmiah covers all aspects of the latest outstanding research and developments in the field of Computer science, ...