Brilliance: Research of Artificial Intelligence
Vol. 6 No. 4 (2026): Brilliance: Research of Artificial Intelligence, Article Research November 2026

Analyzing Chatbot Interaction Patterns and Customer Satisfaction Classification Using Text Data

Dhafin Rizky Trisandy (Universitas Sriwijaya, Indonesia)
Rizka Dhini Kurnia (Universitas Sriwijaya, Indonesia)
Purwita Sari (Universitas Sriwijaya, Indonesia)
Ari Wedhasmara (Universitas Sriwijaya, Indonesia)
Apriansyah Putra (Universitas Sriwijaya, Indonesia)



Article Info

Publish Date
01 Nov 2026

Abstract

This study aims to identify recurring customer interaction patterns in the chatbot service of CV Mawar and independently evaluate text-based customer satisfaction classification using the Flipkart customer support dataset. The primary dataset consisted of 137 chatbot interaction sessions from CV Mawar. After removing 10 sessions with empty transcripts, 127 sessions were manually coded and analyzed descriptively. Because reliable customer satisfaction labels were limited, a separate Flipkart dataset containing customer remarks and Customer Satisfaction (CSAT) scores was used for machine learning. Text classification was performed using Term Frequency–Inverse Document Frequency (TF-IDF) and a linear Support Vector Machine (SVM). The model was evaluated using accuracy, balanced accuracy, precision, recall, F1-score, and a confusion matrix. Delivery estimates and product information or usage were the most frequent categories in CV Mawar, each representing 19.69% of the coded sessions. Only three usable sessions contained genuine service ratings, all rated “Good,” while no usable written feedback was available. The Flipkart classification model achieved 79.62% accuracy, 81.92% balanced accuracy, and a macro F1-score of 78.31%, outperforming the majority-class baseline accuracy of 68.74%. The findings highlight common customer information needs in chatbot interactions and demonstrate the potential of text classification for analyzing customer feedback. However, the limited satisfaction ratings at CV Mawar and the use of an independent Flipkart dataset restrict conclusions about satisfaction among CV Mawar customers.

Copyrights © 2026






Journal Info

Abbrev

brilliance

Publisher

Subject

Decision Sciences, Operations Research & Management Mathematics Other

Description

Brilliance: Research of Artificial Intelligence is The Scientific Journal. Brilliance is published twice in one year, namely in February, May and November. Brilliance aims to promote research in the field of Informatics Engineering which focuses on publishing quality papers about the latest ...