Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : JUITA : Jurnal Informatika

Assessment of Retrieval and Generative Chatbots in Tourism Information Service Handayani, Sharfina Febbi; Dairoh, Dairoh; Af'idah, Dwi Intan
JUITA: Jurnal Informatika JUITA Vol. 13 Issue 1, March 2025
Publisher : Department of Informatics Engineering, Universitas Muhammadiyah Purwokerto

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.30595/juita.v13i1.24182

Abstract

Chatbots are essential for improving the customer experience on tourism websites, especially when it comes to arranging travel and offering precise information. The purpose of this study is to evaluate the effectiveness of generative and retrieval-based chatbots in the tourism information service. Two retrieval-based models are MLP-based single QA and multi QA and two generative-based models namely LLaMA 2 and GEMMA were evaluated using confusion matrix, BLUE score, correctness response and naturalness response. The study found that LLaMA 2 outperformed other models, with the highest response Accuracy of 0.89, naturalness of 0.75, and BLEU score of 0.33. GEMMA received the lowest score, suggesting that it has trouble coming up with precise and organic answers. The retrieval-based models showed strong accuracy but were less natural in their responses. The ease of dataset creation for generative models, which only requires narrative text, further positions LLaMA 2 as the most suitable option for improving user experience in Tegal tourism services.