<![CDATA[This study develops an automated sentiment analysis system for classifying reviews of the RSI Sunan Kudus mobile application, collected from Google Play Store, Google Maps, and YouTube (N = 1,428). Sentiment labels were automatically assigned using a domain-adapted Indonesian lexicon (87 positive / 100 negative terms), with label reliability validated against a human-annotated gold-standard subset (n = 100, two annotators; κ = 0.900, almost perfect agreement), yielding 85.71% classifier accuracy against adjudicated labels. After excluding neutral reviews, a binary Multinomial Naive Bayes classifier was evaluated via 5-fold stratified cross-validation, achieving 91.01 ± 1.13% accuracy and 89.20 ± 1.48% Macro F1-score. Class imbalance (ratio = 2.39:1) was addressed using SMOTE within each training fold in feature-vector space. An ablation study across six model-vectorizer combinations identified LinearSVC with Bag-of-Words as best-performing (Macro F1 = 96.03 ± 1.34%); a preprocessing ablation showed the six-stage normalisation pipeline did not meaningfully improve Macro F1 over simpler variants. Cross-platform transfer experiments revealed substantial generalisation gaps (e.g., Macro F1 dropping from 81.1% to 26.8% for a Google-Maps-trained model applied to YouTube), underscoring the value of multi-platform data collection. Findings indicate three primary service pain points: OTP/login failures, long waiting times, and application connectivity issues.]]>
Copyrights © 2026
