JUITA : Jurnal Informatika
JUITA Vol. 13 Issue 1, March 2025

Assessment of Retrieval and Generative Chatbots in Tourism Information Service

Handayani, Sharfina Febbi (Unknown)
Dairoh, Dairoh (Unknown)
Af'idah, Dwi Intan (Unknown)



Article Info

Publish Date
18 Mar 2025

Abstract

Chatbots are essential for improving the customer experience on tourism websites, especially when it comes to arranging travel and offering precise information. The purpose of this study is to evaluate the effectiveness of generative and retrieval-based chatbots in the tourism information service. Two retrieval-based models are MLP-based single QA and multi QA and two generative-based models namely LLaMA 2 and GEMMA were evaluated using confusion matrix, BLUE score, correctness response and naturalness response. The study found that LLaMA 2 outperformed other models, with the highest response Accuracy of 0.89, naturalness of 0.75, and BLEU score of 0.33. GEMMA received the lowest score, suggesting that it has trouble coming up with precise and organic answers. The retrieval-based models showed strong accuracy but were less natural in their responses. The ease of dataset creation for generative models, which only requires narrative text, further positions LLaMA 2 as the most suitable option for improving user experience in Tegal tourism services.

Copyrights © 2025






Journal Info

Abbrev

JUITA

Publisher

Subject

Computer Science & IT

Description

UITA: Jurnal Informatika is a science journal and informatics field application that presents articles on thoughts and research of the latest developments. JUITA is a journal peer reviewed and open access. JUITA is published by the Informatics Engineering Study Program, Universitas Muhammadiyah ...