Comparison of Random Forest, Decision Tree, and XGBoost Models in Predicting Student Academic Success - Nurbaeti, Nurbaeti; Sulistiyaningsih, Neny; Rismayati, Ria (Journal of Artificial Intelligence and Software Engineering, 2025)
Students' academic success is influenced by various academic and non-academic factors. Machine learning (ML) offers an effective approach to predicting academic outcomes by analyzing complex data patterns. However, most previous studies are limited to graduation prediction and rarely incorporate non-academic features or multiple feature selection techniques. This study aims to compare the performance of three ML algorithms Random Forest, Decision Tree, and XGBoost in classifying students’ academic success using a dataset from the UCI Machine Learning Repository, consisting of 4424 records and 37 features. The data underwent cleaning, label transformation, and feature selection using PCA, SelectKBest, and Variance Threshold. Models were trained using a holdout method (80% training, 20% testing) and evaluated based on accuracy, precision, recall, and F1-score. The results show that Random Forest with Variance Threshold achieved the highest accuracy (0.77) and F1-score (0.84) on majority classes. XGBoost followed with 0.75 accuracy, while Decision Tree showed the lowest performance. All models struggled to classify the minority class, indicating challenges related to data imbalance. This research highlights the importance of algorithm choice and effective feature selection in academic classification tasks. It also emphasizes the need for data balancing strategies to reduce class bias. The findings can help educational institutions design data-driven interventions to improve learning outcomes and reduce dropout rates.