<![CDATA[Sign language is the primary communication tool for individuals with speech disabilities. Information in sign language is conveyed through unique hand gestures and movement patterns. However, this uniqueness often becomes a barrier for those unfamiliar with it, making it challenging for people with speech disabilities to interact with individuals who do not understand sign language, whether in their surroundings, public spaces, or facilities. This study aims to produce a portable sign language translation device that is easy to carry, can be used independently without assistance, and translates sign language into voice, allowing users to communicate directly, as if they were speaking. The translation device uses CNN technology processed on a Raspberry Pi Zero 2 W to recognize hand gestures and movement patterns. A Keyframe-based sequence classification method is applied to identify keyframes from video recordings of SIBI (Indonesian Sign Language System) gestures. Of the 10 SIBI words, six key frames were identified as input for the classification model, utilizing a TimeDistributed architecture with MobileNetV2 for feature extraction, followed by 1D convolution. The model was optimized for efficiency and performance to enhance inference time and classification accuracy. Reducing the number of frames from six to four provided the best balance, with the fastest inference time of 0.5 seconds and validation accuracy of 96.76% using Depthwise Separable Convolution. Real-time testing showed an average accuracy of 98.31%, with the highest F1-score reaching 100% for the words "Ini" and "Bagaimana", while the lowest F1-score of 83.1% was achieved for the word "11" and 83.6% for the word "Mana". The device consumes 53.81 Wh and is capable of operating for20 hours in a usage scenario of 8 hours active use and 12 hours in idle mode before the next charging.Indeks Terms— Keyframe-Based Sequence Classification, Hand Gesture Classification, Portable Sign Language Translator Device, MobileNetV2.]]>
Copyrights © 2024
