<![CDATA[Recent machine learning studies on the SEER Breast Cancer dataset report 95–98% accuracy (2022–2025), but their methods are rarely examined. Because the dataset is imbalanced (84.7% Alive), accuracy alone is a poor discrimination metric — a trivial always-Alive classifier scores near 85% — so this audit quantifies leakage inflation on ROC-AUC, computed identically across 12 models. We measure how much performance stems from target leakage via Survival Months, which encodes retrospective survival time unavailable at prediction. Undetected, such leakage could lead clinicians to overtrust a model's risk estimates, risking inappropriate care decisions. We train 12 models under three conditions: a) naïve, with Survival Months and a non-stratified split; b) leakage-free with stratified evaluation; and c) fixed-horizon (36- and 60-month) binary mortality, excluding censored cases. Under Condition A, the best ROC-AUC is 0.889 (accuracy 0.899). Removing leakage yields a consistent AUC ceiling of 0.74–0.77 across all 12 models — an inflation of +0.07 to +0.20 AUC points. SHAP shows Survival Months dominates feature importance by 3.0–4.3× under leakage; imbalance-handling methods (BorderlineSMOTE, class weighting) raise minority-class F1 but lower mean Brier Skill Score to −0.585 and −0.597. The contribution is the audit methodology itself — a reusable protocol for detecting leakage-driven inflation in registry-based clinical prediction.]]>
Copyrights © 2026
