Knowledge Engineering and Data Science


The Effect of Resampling on Classifier Performance: anEmpirical Study

Pujianto, Utomo (Unknown)
Akbar, Muhammad Iqbal (Unknown)
Lassela, Niendhitta Tamia (Unknown)
Sutaji, Deni (Unknown)



Article Info

Publish Date
01 Jul 2022

Abstract

An imbalanced class on a dataset is a common classification problem. The effect of using imbalanced class datasets can cause a decrease in the performance of the classifier. Resampling is one of the solutions to this problem. This study used 100 datasets from 3 websites: UCI Machine Learning, Kaggle, and OpenML. Each dataset will go through 3 processing stages: the resampling process, the classification process, and the significance testing process between performance evaluation values of the combination of classifier and the resampling using paired t-test. The resampling used in the process is Random Undersampling, Random Oversampling, and SMOTE. The classifier used in the classification process is Naïve Bayes Classifier, Decision Tree, and Neural Network. The classification results in accuracy, precision, recall, and fmeasure values are tested using paired t-tests to determine the significance of the classifier's performance from datasets that were not resampled and those that had applied the resampling. The paired t-test is also used to find a combination between the classifier and the resampling that gives significant results. This study obtained two results. The first result is that resampling on imbalanced class datasets can substantially affect the classifier's performance more than the classifier's performance from datasets that are not applied the resampling technique. The second result is that combining the Neural Network Algorithm without the resampling provides significance based on the accuracy value. Combining the Neural Network Algorithm with the SMOTE technique provides significant performance based on the amount of precision, recall, and f-measure. T

Copyrights © 2022






Journal Info

Abbrev

publication:keds

Publisher

Subject

Computer Science & IT Engineering

Description

The journal welcomes experimental and theoretical findings on data science and knowledge engineering along with their applications to real-life ...