Benchmarking Nine SMOTE-Balanced Classifiers Including Artificial Neural Network for CNC Predictive Maintenance - Didiek Trisatya; Priyo Haryoko (International Journal of Innovation in Mechanical Engineering and Advanced Materials, 2026)
Unplanned equipment failure in CNC manufacturing causes significant economic losses, driving demand for effective predictive maintenance (PdM). A critical research gap persists: existing studies on the AI4I 2020 Predictive Maintenance Dataset apply isolated classifiers under inconsistent preprocessing pipelines, preventing fair algorithmic comparison. No prior study has benchmarked nine diverse classifier families under a unified pipeline integrating SMOTE oversampling with domain-driven feature engineering. This study addresses that gap by systematically evaluating nine ML classifiers—Logistic Regression, K-Nearest Neighbors, Decision Tree, Random Forest, Gradient Boosting, AdaBoost, SVM (RBF kernel), Naive Bayes, and MLP Neural Network—on the AI4I 2020 dataset (10,000 records; 3.4% failure rate; 1:28 class imbalance). Two domain-engineered features were constructed: mechanical power (P = n × T × (π/30)) and thermal gradient (ΔT = T_process - T_air). Features were normalized; SMOTE was applied to training folds only; and 10-fold stratified cross-validation assessed six performance metrics. Three novel contributions are presented: (1) the first nine-classifier benchmark on AI4I 2020 under a unified SMOTE-and-feature-engineering pipeline enabling fair model comparison; (2) empirical demonstration that Average Precision is a more discriminating evaluation metric than AUC-ROC under severe 1:28 class imbalance; and (3) physical interpretation of feature importance linking dominant predictors to CNC failure mechanisms. Gradient Boosting achieved the best-balanced performance (F1-score: 0.6782, Accuracy: 97.20%, AUC-ROC: 0.9723); Random Forest attained the highest AUC-ROC (0.9772). Mechanical power (25.51%) and tool wear (23.91%) were dominant predictors, corresponding to tribological, fatigue loading, and thermal failure mechanisms. These findings support cost-effective condition-based maintenance strategies in industrial CNC environments.