Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
Vani Maharani Nasution
Universitas Jakarta Internasional
Computer Science & IT Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Quantifying Target Leakage in SEER Breast Cancer Mortality Prediction Sestri Novia Rizki; Vani Maharani Nasution; Eko Amri Jaya; Verra Budhi Lestari; Elfina Maulid
Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) Vol 10 No 5 (2026): October 2026 (In progress)
Publisher : Ikatan Ahli Informatika Indonesia (IAII)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.29207/resti.v10i5.8105

Abstract

Recent machine learning studies on the SEER Breast Cancer dataset report 95–98% accuracy (2022–2025), but their methods are rarely examined. Because the dataset is imbalanced (84.7% Alive), accuracy alone is a poor discrimination metric — a trivial always-Alive classifier scores near 85% — so this audit quantifies leakage inflation on ROC-AUC, computed identically across 12 models. We measure how much performance stems from target leakage via Survival Months, which encodes retrospective survival time unavailable at prediction. Undetected, such leakage could lead clinicians to overtrust a model's risk estimates, risking inappropriate care decisions. We train 12 models under three conditions: a) naïve, with Survival Months and a non-stratified split; b) leakage-free with stratified evaluation; and c) fixed-horizon (36- and 60-month) binary mortality, excluding censored cases. Under Condition A, the best ROC-AUC is 0.889 (accuracy 0.899). Removing leakage yields a consistent AUC ceiling of 0.74–0.77 across all 12 models — an inflation of +0.07 to +0.20 AUC points. SHAP shows Survival Months dominates feature importance by 3.0–4.3× under leakage; imbalance-handling methods (BorderlineSMOTE, class weighting) raise minority-class F1 but lower mean Brier Skill Score to −0.585 and −0.597. The contribution is the audit methodology itself — a reusable protocol for detecting leakage-driven inflation in registry-based clinical prediction.
Co-Authors Eko Amri Jaya Elfina Maulid Sestri Novia Rizki Verra Budhi Lestari