Roads are critical infrastructure that frequently experience damage, directly impacting transportation safety and efficiency. Manual road damage inspection is time-consuming and resource-intensive, highlighting the need for automated, image-based approaches. This study compares two Convolutional Neural Network (CNN) architectures—MobileNetV2 with transfer learning and a custom-built CNN—for classifying road surface damage severity. The dataset consists of 1,800 road surface images evenly distributed across three categories: good, minor damage, and severe damage. All images were normalized, augmented, and resized, followed by evaluation using 5-Fold Cross-Validation to ensure robust performance. Experimental results show that MobileNetV2 achieved an accuracy of 98%, outperforming the custom CNN, which achieved 89%. These findings demonstrate the effectiveness of transfer learning in improving classification accuracy with limited data and highlight the potential of MobileNetV2 for efficient, real-time road damage detection systems that can be integrated into intelligent infrastructure monitoring solutions.