Journal of Information Systems and Informatics
Vol 8 No 5 (2026): October

Design and Evaluation of a Route-Aware, Evidence-Grounded RAG Chatbot for Accessing Village Monographs and Census Data within Data Desa Presisi

Ikhlasul Amal (IPB University)
Irman Hermadi (IPB University)
Yani Nurhadryani (IPB University)



Article Info

Publish Date
24 Oct 2026

Abstract

Data Desa Presisi integrates narrative village monographs and structured census records, creating challenges for contextual, numerical, and cross-source information access. This study designed and evaluated a locally deployed, route-aware, evidence-grounded retrieval-augmented generation (RAG) chatbot that combines deterministic DuckDB queries, document retrieval, provenance, and privacy-aware routing. Evaluation used 100 ground-truth questions from Setu and Gunung Putri. Eighty-four database, monograph, and mixed-evidence questions were assessed using RAGAS and ROUGE-L, whereas 16 conversational and policy questions were evaluated through deterministic checks. Elasticsearch at a temperature of 0.1 produced the highest composite score (0.592018), although post hoc Wilcoxon tests found no statistically significant superiority over alternative configurations. The selected configuration achieved a 98.0% functional pass rate and a 97.92% numeric-match rate. Public-mode usability evaluation involving 21 village officials and staff yielded 96.03% effectiveness. Scalability testing covered pipeline construction and retrieval across 30 census inputs and 28 monographs, but did not assess concurrent users or full end-to-end production workloads. These findings provide preliminary evidence that route-aware RAG can improve transparent, evidence-grounded access to heterogeneous village data while preserving deterministic handling of numerical queries and privacy-sensitive information.

Copyrights © 2026






Journal Info

Abbrev

isi

Publisher

Subject

Computer Science & IT

Description

Journal-ISI is a scientific article journal that is the result of ideas, great and original thoughts about the latest research and technological developments covering the fields of information systems, information technology, informatics engineering, and computer science, and industrial engineering ...