Integrated Cnn Based Facial Emotion Detection And Camera Based PPG Heart Rate Monitoring - Panggabean, Erwin; Simanjorang, R. Mahdelena; Apriani , Wira; Nuraisana , Nuraisana; Sipahutar, Hartati Palentina; Siagian, Tesalonika Pesta (Journal of Computer Networks, Architecture and High Performance Computing, 2025)
Human emotion detection and heart rate estimation are two important aspects in developing a more responsive and adaptive human-computer interaction system. This study proposes a real-time video-based system that is able to detect facial emotions and estimate the user's heart rate simultaneously. The Convolutional Neural Network (CNN) method is used to classify facial expressions into several emotion categories such as happy, sad, angry, afraid, and neutral. Meanwhile, heart rate estimation is carried out using a non-contact Photoplethysmography (PPG) approach, which utilizes variations in color intensity in the user's facial area from video recordings to calculate the pulse rate. This system is developed using a standard webcam camera without additional medical devices, allowing for practical and economical implementation. The test results show that the system is able to recognize facial expressions with good accuracy, and estimate heart rate with an average error rate that is still within the tolerance limit of non-medical applications. By integrating computer vision technology and biometric signals, this study contributes to the development of a passive, real-time, and easily accessible emotion and health monitoring system.