Performance Evaluation of Sentiment Classification Models: A Comparative Study of NBC, SVM, and DT with SMOTE - Singgalen, Yerik Afrianto (STMIK Budi Darma, 2024)
This research explores the performance of sentiment classification models, namely Naive Bayes Classifier (NBC), Decision Tree (DT), and Support Vector Machine (SVM), using the CRISP-DM methodology in the context of digital content analysis and data mining. The analysis was conducted on a SMOTE dataset in Rapidminer, yielding significant performance metrics. The NBC model achieved an accuracy of 86.98% +/- 0.96%, precision of 100.00% +/- 0.00%, recall of 78.82% +/- 1.55%, and f-measure of 88.15% +/- 0.97%, with an AUC of 0.657 +/- 0.203. Similarly, the DT model exhibited an accuracy of 93.20% +/- 0.42%, precision of 90.87% +/- 0.64%, recall of 98.88% +/- 0.31%, and f-measure of 94.70% +/- 0.31%, with an AUC of 0.918 +/- 0.006. Furthermore, the SVM model demonstrated an accuracy of 96.80% +/- 0.65%, precision of 98.99% +/- 0.28%, recall of 95.77% +/- 1.03%, and f-measure of 97.35% +/- 0.55%, with an AUC of 0.994. These findings highlight the efficacy of these models in accurately classifying sentiments within digital content, suggesting their suitability for various data mining applications. Recommendations for future research include exploring ensemble methods, continuous model updating, alternative sampling techniques, feature engineering approaches, and collaboration with domain experts to enhance real-world applicability