Comparison of the SMOTE Method on Naive Bayes and K-Nearest Neighbor for Sentiment Analysis of the MyPertamina Application - Rian Syaputra Ainun Naim Rian; Novianti Puspitasari; Ummul Hairah (Prodi Pendidikan Teknik Informatika Universitas Pendidikan Ganesha, 2026)
MyPertamina is a digital application developed by PT Pertamina as a means of non-cash payment and a verification tool in the distribution of fuel oil, as well as offering loyalty programs in the form of points, prizes, and electronic coupons to its users. The application has a rating of 3.3 on the Google Play Store and 2.1 on the App Store, with more than 10 million downloads. However, these ratings do not fully reflect user satisfaction or issues. This research aims to analyze the effectiveness of the Naïve Bayes and K-Nearest Neighbor algorithms, as well as the influence of the SMOTE technique, in classifying user sentiment toward the MyPertamina application. Review data were obtained through a scraping technique, yielding 1,500 entries, which were then cleaned and labeled as positive, negative, or neutral using TextBlob. The TF-IDF method was applied for weighting, and 10-fold cross-validation was used for data splitting. Two testing scenarios were conducted: with SMOTE to address data imbalance, and without SMOTE. The results show that Naïve Bayes with SMOTE achieved the best performance, with an accuracy of 82.07%, precision of 81.67%, recall of 81.10%, and an F1-score of 80.96%, significantly improving classification performance, particularly for the neutral class. Based on these findings, Naïve Bayes with SMOTE is proven to be the most effective method for classifying user sentiment on MyPertamina reviews, as it produces balanced and accurate performance across all classes.