Implementation of MobileNetV2 and OpenCV in a Real-Time Sign Language Recognition System - Marcellinus Ronan Narendra; Radius Tanone (Soegijapranata Catholic University, 2026)
The main means of communication for the deaf is sign language, although communication hurdles are frequently caused by a lack of public comprehension. The goal of this project is to use the OpenCV library and MobileNetV2 architecture to create a real-time American Sign Language (ASL) recognition system. The ASL Alphabet from Kaggle, which has over 87,000 photos in 29 classes, is the dataset that was used. Convolutional Neural Networks (CNNs) based on MobileNetV2 with picture preprocessing and data augmentation were used to train the model. OpenCV was then used to combine the CNN with a camera for real-time implementation.The evaluation results indicate an average F1-score of 83.6, recall of 83.8, and precision of 83.5. The system is responsive for direct interaction because it can operate at 18–22 frames per second (FPS) on a typical laptop. The system still has issues with complicated backdrops, low light levels, and gesture similarities between some letters. Overall, this study demonstrates the efficacy of MobileNetV2 and OpenCV in developing a lightweight, efficient real-time sign language recognition system that facilitates inclusive communication for individuals with hearing impairments.