Evaluating the Impact of Model Complexity on the Accuracy of ID3 and Modified ID3: A Case Study of the Max_Depth Parameter - Asrianda, Asrianda; Mawengkang, Herman; Sihombing, Poltak; K. M. Nasution , Mahyuddin (Jurnal Teknik Informatika (Jutif), 2025)
The complexity of decision tree structures has a direct impact on the generalization capability of classification algorithms. This study investigates and evaluates the performance of the classical ID3 algorithm and its modified version in the context of tree depth. The primary objective is to identify the optimal accuracy point and assess the algorithms' robustness against overfitting. Experiments were conducted across tree depths ranging from 1 to 20, with accuracy used as the main evaluation metric. The results indicate that both algorithms achieved peak performance at depth 3, followed by a notable decline. While the classical ID3 algorithm exhibited a gradual decrease in accuracy, the modified ID3 showed a sharp drop and performance stagnation between depths 11 and 20. These findings suggest that the modified ID3 algorithm enhances sensitivity in selecting informative attributes but also increases the risk of overfitting in the absence of structural regularization mechanisms. Therefore, the study recommends the implementation of regularization strategies such as pruning and cross-validation to mitigate performance degradation caused by model complexity. This research not only contributes to the theoretical understanding of how tree depth influences classification performance but also offers practical insights for developing adaptive, stable, and accurate decision tree-based classification systems.