Match outcome prediction in multiplayer online battle arena games is often reported with high accuracy, yet many reported figures rely on information that exists only after a match, and small samples inflate performance. This case study asks what can be predicted for a single player and how much apparent accuracy is optimism. We used 739 ranked All Pick Dota 2 matches (375 wins) of one player, retrieved through the OpenDota API, in three scenarios: pre-match win or loss classification (A), post-match explanatory classification (B), and pre-match regression of gold per minute (GPM) and KDA (C). Validation combined repeated cross-validation on 541 earlier matches for model selection, a single out-of-sample confirmation on 198 later matches, and walk-forward validation. Pre-match models reached AUC of 0.471 to 0.520 in development, 0.572 (95% CI 0.490 to 0.646) in confirmation, and 0.519 to 0.529 in walk-forward validation, close to chance; an AUC of 0.615 from an earlier exploration of 199 matches proved optimistic. The post-match model reached AUC of 0.921 but explains rather than predicts outcomes. Hero choice explained 26 to 30% of GPM variance but little of KDA. Chronological validation is needed to avoid leakage.
Copyrights © 2026