An Evaluation of SMOTE Effectiveness in Handling Class Imbalance in Public Opinion Data on the MBG Program - Ramadhan, Nur Ghaniaviyanto; Khoirunnisa, Azka (Universitas Dharma Wacana, 2025)
The “Makan Bergizi Gratis” (MBG) Program is one of the strategic policies of the Government of Indonesia that reaps various opinions from the public, especially through social media. This study aims to classify public sentiment towards the MBG program with an ensemble learning-based machine learning approach, as well as evaluate the effectiveness of the SMOTE algorithm in dealing with class imbalance in opinion data. The dataset was collected from platform X (formerly Twitter) for the January–April 2025 period, totaling 4,374 tweets with label distributions: 1,783 positive, 1,634 negative, and 957 neutral. The preprocessing process includes data cleansing, normalization, stemming, and vectorization with TF-IDF. Five ensemble algorithms were used, namely Random Forest, AdaBoost, Bagging, Stacking, and Voting, tested in two scenarios: with and without the implementation of SMOTE. The results of the experiments showed that Random Forest provided the best and most consistent performance, with the F1-score increasing from 72.03% to 72.66% after the implementation of SMOTE. However, not all models benefit from SMOTE, such as Voting which experienced a drop in F1-score. These findings suggest that SMOTE is effective in increasing the sensitivity of the model to minority classes, but its success depends on the characteristics of the algorithm used. This study suggests the selective selection of balancing methods as well as the development of a more adaptive approach to handle unstructured opinion data.