A Comparative Analysis of Machine Learning Approaches for Diabetes Prediction: Stability of Hold-Out Testing versus Cross-Validation - Ade Putra Prima Suhendri; Meidy Fajar Wahyu; Amin Hidayat; Lely Panca Andriyanto (Jurnal Publikasi Ilmu Komputer Terapan, 2026)
Automated medical diagnostic systems often face operational challenges when dealing with clinical datasets that contain noise, biological zero-value anomalies, and class imbalance issues. This study offers an empirical benchmark of six supervised machine learning algorithms—Random Forest (RF), Decision Tree (DT), Gradient Boosting (GB), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Gaussian Naïve Bayes (GNB)—evaluated directly on the un-imputed Pima Indians Diabetes dataset. We systematically compare the performance differences between a single 80:20 hold-out test split and a 10-fold stratified cross-validation, assessing accuracy, stability, and discrimination across various thresholds. Results show that while the hold-out test favors Random Forest (92.86%) and Decision Tree (92.21%) in accuracy, stratified cross-validation reveals that ensemble models like Gradient Boosting (88.94% CV accuracy) and Random Forest (88.55% CV accuracy) provide better operational stability. Additionally, threshold-independent metrics indicate that Gradient Boosting and Random Forest share top ROC-AUC scores (0.98), with Gradient Boosting leading in Precision-Recall AUC (0.97). Lower stability is observed in linear and probabilistic models such as SVM (81.17% accuracy, 0.84 ROC-AUC) and Gaussian Naïve Bayes on raw features. These results emphasize that relying solely on single hold-out evaluations may overestimate how well classifiers generalize, with ensemble methods emerging as the most robust approach for unconstrained clinical risk assessment.