Fatigue Detection Based on Facial Expressions Using ResNet50 with Grad-CAM Visualization - Wike Septiana Vidya Utami; Ervin Yohannes (Journal of Informatics and Computer Science (JINACS), 2026)
Abstract—Fatigue is a condition that can affect a person's performance and concentration, making an automatic system necessary for its rapid and accurate detection. This study applies a Convolutional Neural Network (CNN) architecture based on the pretrained ResNet50 model to classify facial images into two categories, fatigued and non-fatigued, using the Driver Drowsiness Dataset, which consists of approximately 41,700 images. The model was trained and validated using five data-split scenarios, namely 50:50, 60:40, 70:30, 80:20, and 90:10, with varying learning rates, batch sizes, and numbers of epochs. Performance was evaluated using accuracy, precision, recall, and F1-score metrics. The results show that the best performance of the ResNet50 model was obtained with a 60:40 data split, a learning rate of 0.0001, a batch size of 16, and 30 epochs, achieving an accuracy of 99.22%, a precision of 99.67%, a recall of 99.86%, and an F1-score of 99.26%. Visual analysis using Grad-CAM showed that the model focused attention on the eyes, eyelids, cheeks, and forehead during the classification decision, improving the interpretability of the predictions. The ResNet50 model was also implemented in a camera-based system to provide real-time fatigue predictions. These results indicate that ResNet50 is effective and has the potential to be applied in a practical and efficient facial-image-based fatigue detection system. Keywords—ResNet50, Convolutional Neural Network (CNN), Fatigue Detection, Grad-CAM, Facial Image Classification.