Comparison of Naive Bayes and Support Vector Machine in Sentiment Analysis of Siwaslu Application - Trio Saputro; Arief Hermawan (bit-Tech, 2025)
The 2024 General Election (PEMILU) in Indonesia introduced the SIWASLU application, which generated a large volume of unstructured user feedback on the Google Play Store. Efficiently analyzing this public sentiment is crucial for guiding rapid application enhancements, yet the sheer volume of raw data poses a significant challenge. The purpose of this study is to evaluate the performance of two classifications, Naive Bayes and Support Vector Machine (SVM), to identify the most effective model for sorting review sentiment into positive, neutral, and negative categories. This research offers novelty as it is applied to a comprehensive multi-class scheme, differing from previous research focused on binary classification and its evaluation of a hybrid feature approach for SVM. The methodology began with the collection of 3,632 reviews, followed by pre-processing and lexicon-based labeling. The naive bayes model was trained using CountVectorizer features, while the SVM model was trained using a combination of TF-IDF features, additional engineered features, and a weighting technique to handle imbalanced data. Evaluation results demonstrate that the SVM model was significantly superior, achieving 85% accuracy and a macro-average F1-score of 0.72, outperforming the NB model (78% accuracy and 0.60 F1-score). The superiority of SVM was evident in its ability to identify the minority class, achieving a 0.75 Recall score for neutral sentiment. Practically, the developed SVM model is robust enough to be integrated into a real-time monitoring dashboard for BAWASLU, providing an automated system to categorize public concerns and enable faster, data-driven improvements.