Sistemasi: Jurnal Sistem Informasi
Vol 15, No 10 (2026): Sistemasi: Jurnal Sistem Informasi

Analyzing the Impact of SMOTE on Predictive Performance and Random Forest Feature Importance Stability in Financial Distress Prediction

Alan Kusuma Soesanto (Universitas Kristen Satya Wacana)
Charitas Fibriani (SCOPUS ID=57192643331) (Universitas Kristen Satya Wacana)



Article Info

Publish Date
31 Oct 2026

Abstract

Financial distress prediction is often challenged by class imbalance, which can cause classification models to become biased toward the majority class. The Synthetic Minority Over-sampling Technique (SMOTE) is widely used to improve minority-class detection; however, its impact on the consistency of feature importance as a basis for model interpretation remains relatively underexplored. This study examines the impact of SMOTE on predictive performance and feature importance stability in Random Forest models using the Taiwan corporate bankruptcy dataset, comprising 6,819 companies, of which 220 were bankrupt, and 39 financial ratios retained after feature selection. Three scenarios were compared: Random Forest without class imbalance treatment (RF), Random Forest with class weighting (RF + CW), and Random Forest with SMOTE applied within the cross-validation pipeline (RF + SMOTE). All three scenarios were optimized using the same hyperparameter search space through five-fold GridSearchCV, with the F2-score as the optimization metric. Differences in test-set performance were evaluated using paired bootstrap and McNemar's test, while feature importance stability was assessed using Spearman's rank correlation, Kendall's tau, and Jaccard similarity across 10 random seeds. Compared with RF, SMOTE significantly increased recall from 0.2045 to 0.6818 (95% CI for the difference: 0.341 to 0.636), but reduced precision from 0.8182 to 0.3659. Changes in ROC-AUC (from 0.9439 to 0.9551) and PR-AUC were not statistically significant. The predictive performance of RF + SMOTE also did not differ significantly from that of RF + CW. Feature importance rankings changed moderately after SMOTE was applied (Spearman's correlation: 0.644 ± 0.017), indicating greater variation than that observed across random seeds (0.970). Nevertheless, eight financial ratios, including Total Debt/Total Net Worth and Persistent EPS in the Last Four Seasons, consistently appeared among the top 10 features across all repetitions. These findings indicate that SMOTE primarily shifts the model's decision boundary toward greater minority-class detection rather than improving its discriminatory ability. Therefore, the interpretation of feature importance should prioritize financial ratios that remain stable across different modeling scenarios.

Copyrights © 2026






Journal Info

Abbrev

stmsi

Publisher

Subject

Computer Science & IT Electrical & Electronics Engineering

Description

Sistemasi adalah nama terbitan jurnal ilmiah dalam bidang ilmu sains komputer program studi Sistem Informasi Universitas Islam Indragiri, Tembilahan Riau. Jurnal Sistemasi Terbit 3x setahun yaitu bulan Januari, Mei dan September,Focus dan Scope Umum dari Sistemasi yaitu Bidang Sistem Informasi, ...