Apriansyah Putra
Universitas Sriwijaya, Indonesia

Published : 2 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 2 Documents
Search

Analyzing Chatbot Interaction Patterns and Customer Satisfaction Classification Using Text Data Dhafin Rizky Trisandy; Rizka Dhini Kurnia; Purwita Sari; Ari Wedhasmara; Apriansyah Putra
Brilliance: Research of Artificial Intelligence Vol. 6 No. 4 (2026): Brilliance: Research of Artificial Intelligence, Article Research November 2026
Publisher : Yayasan Cita Cendekiawan Al Khwarizmi

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47709/brilliance.v6i4.9810

Abstract

This study aims to identify recurring customer interaction patterns in the chatbot service of CV Mawar and independently evaluate text-based customer satisfaction classification using the Flipkart customer support dataset. The primary dataset consisted of 137 chatbot interaction sessions from CV Mawar. After removing 10 sessions with empty transcripts, 127 sessions were manually coded and analyzed descriptively. Because reliable customer satisfaction labels were limited, a separate Flipkart dataset containing customer remarks and Customer Satisfaction (CSAT) scores was used for machine learning. Text classification was performed using Term Frequency–Inverse Document Frequency (TF-IDF) and a linear Support Vector Machine (SVM). The model was evaluated using accuracy, balanced accuracy, precision, recall, F1-score, and a confusion matrix. Delivery estimates and product information or usage were the most frequent categories in CV Mawar, each representing 19.69% of the coded sessions. Only three usable sessions contained genuine service ratings, all rated “Good,” while no usable written feedback was available. The Flipkart classification model achieved 79.62% accuracy, 81.92% balanced accuracy, and a macro F1-score of 78.31%, outperforming the majority-class baseline accuracy of 68.74%. The findings highlight common customer information needs in chatbot interactions and demonstrate the potential of text classification for analyzing customer feedback. However, the limited satisfaction ratings at CV Mawar and the use of an independent Flipkart dataset restrict conclusions about satisfaction among CV Mawar customers.
Analysis and Prediction of MyTelkomsel Complaints Using IndoBERTweet and TimesFM-3 for Service Improvement Prioritization Muhammad Syaikh Azka; Apriansyah Putra; Rahmat Izwan Heroza; Ken Ditha Tania
Brilliance: Research of Artificial Intelligence Vol. 6 No. 4 (2026): Brilliance: Research of Artificial Intelligence, Article Research November 2026
Publisher : Yayasan Cita Cendekiawan Al Khwarizmi

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47709/brilliance.v6i4.9879

Abstract

Application reviews contain useful feedback, but the growing volume of Indonesian-language reviews makes manual identification of complaints inefficient, and earlier MyTelkomsel studies stopped at sentiment classification. Objective: This study develops an integrated framework to identify the dominant technical complaints in MyTelkomsel reviews, forecast their trends, and prioritize service improvements. Methods: A total of 51,112 application-related Google Play reviews, published between November 2022 and September 2026, were kept after keyword-based relevance filtering. IndoBERTweet, fine-tuned on the SmSA dataset, performed sentiment analysis, and 44,937 negative reviews were grouped by BERTopic with K-Means into ten complaint categories. TimesFM-3 then produced zero-shot six-month complaint forecasting for each category, which was evaluated with a rolling-origin backtest and used to compute a priority score from forecast load and trend. Results: Login & Account Access, Loading/Performance Issues, and General Bugs/Errors accounted for 55.4% of the negative reviews. After two surge months were excluded, TimesFM-3 reached a weighted absolute percentage error of 54.65% and an 80% interval coverage of 84.42%. Loading/Performance Issues was the only high-priority category (score 74.5), and four categories were medium priority. Problems After Updates was the only medium-priority category with an increasing trend. Conclusion: The framework supports ranking and monitoring of complaint categories. Its moderate accuracy and wide intervals mean that the forecasts should guide priorities rather than predict exact monthly volumes.