Gusti Arsyad
Universitas Binawan

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

EVALUATING A LARGE LANGUAGE MODEL CHATBOT FOR ENHANCING HISTORICAL LITERACY ON PANGLIMA MAYU OF THE SUMBAWA SULTANATE Gusti Arsyad
CODEX: Journal of Software Engineering Vol 1, No 2 (2026): June 2026
Publisher : Nurul Jadid University, Paiton Probolinggo, East Java

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33650/codex.v1i2.15485

Abstract

The digital preservation of regional history in the modern era requires innovative approaches to bridge the gap between traditional archives and user accessibility. This study presents the development and evaluation of the Panglima Mayu ChatBot, a specialized Large Language Model (LLM) designed to enhance historical literacy regarding the 18th-century Sumbawa Sultanate. The chatbot was developed using a prompt engineering approach based on curated historical materials and a culturally grounded system prompt to ensure factual consistency and alignment with Adat Samawa values. Deployed on an autoscale server environment (2 vCPU / 4 GiB RAM), the system was evaluated through a quantitative survey involving 45 respondents, including general users, students, and educators. The results indicate positive technical and educational perceptions. the system achieved a 97.8% satisfaction rate for accessibility and speed, while 84.4% of users reported increased insight into Islamic historical narratives. Crucially, 77.8% of respondents perceived the chatbot's responses as accurate, indicating a high level of user confidence in the information provided. However, these findings reflect perceived accuracy rather than expert-verified historical accuracy, which requires further validation by subject-matter experts. Furthermore, the system showed strong practical utility, with 73.3% of respondents endorsing the ChatBot as a viable quick reference for tourism and educational purposes. These findings suggest that grounded AI personas can successfully preserve cultural integrity—specifically Adat Samawa values—while providing an interactive alternative to traditional textbooks. This research provides a scalable framework for digitizing regional heroic narratives across Indonesia