KLIK: Kajian Ilmiah Informatika dan Komputer
Vol. 4 No. 5 (2024): April 2024

Performance Evaluation of Sentiment Classification Models: A Comparative Study of NBC, SVM, and DT with SMOTE

Yerik Afrianto Singgalen (Universitas Katolik Indonesia Atma Jaya, Jakarta)



Article Info

Publish Date
30 Apr 2024

Abstract

This research explores the performance of sentiment classification models, namely Naive Bayes Classifier (NBC), Decision Tree (DT), and Support Vector Machine (SVM), using the CRISP-DM methodology in the context of digital content analysis and data mining. The analysis was conducted on a SMOTE dataset in Rapidminer, yielding significant performance metrics. The NBC model achieved an accuracy of 86.98% +/- 0.96%, precision of 100.00% +/- 0.00%, recall of 78.82% +/- 1.55%, and f-measure of 88.15% +/- 0.97%, with an AUC of 0.657 +/- 0.203. Similarly, the DT model exhibited an accuracy of 93.20% +/- 0.42%, precision of 90.87% +/- 0.64%, recall of 98.88% +/- 0.31%, and f-measure of 94.70% +/- 0.31%, with an AUC of 0.918 +/- 0.006. Furthermore, the SVM model demonstrated an accuracy of 96.80% +/- 0.65%, precision of 98.99% +/- 0.28%, recall of 95.77% +/- 1.03%, and f-measure of 97.35% +/- 0.55%, with an AUC of 0.994. These findings highlight the efficacy of these models in accurately classifying sentiments within digital content, suggesting their suitability for various data mining applications. Recommendations for future research include exploring ensemble methods, continuous model updating, alternative sampling techniques, feature engineering approaches, and collaboration with domain experts to enhance real-world applicability

Copyrights © 2024






Journal Info

Abbrev

klik

Publisher

Subject

Computer Science & IT

Description

Topik utama yang diterbitkan mencakup: 1. Teknik Informatika 2. Sistem Informasi 3. Sistem Pendukung Keputusan 4. Sistem Pakar 5. Kecerdasan Buatan 6. Manajemen Informasi 7. Data Mining 8. Big Data 9. Jaringan Komputer 10. Dan lain-lain (topik lainnya yang berhubungan dengan Teknologi Informati dan ...