Journal of Innovation and Community Engagement
Vol. 7 No. 2 (2026)

Implementation of Hand Movement Tracking for Real-Time Sign Language Translation

Bernardus Anggodho Aryudhawan Hadi (Faculty of Industrial Technology, Universitas Atma Jaya, Yogyakarta, Indonesia)
Wilfridus Bambang Triadi Handaya (Faculty of Industrial Technology, Universitas Atma Jaya, Yogyakarta, Indonesia)
Suyoto Suyoto (Faculty of Industrial Technology, Universitas Atma Jaya, Yogyakarta, Indonesia)



Article Info

Publish Date
28 May 2026

Abstract

This study developed a real-time American Sign Language (ASL) sign language identification and interpretation system based on deep learning. The system used two data sources: the ASL alphabet dataset for individual character recognition and the WLASL dataset for vocabulary recognition. The WLASL dataset was chosen as the benchmark for evaluating complex word gestures because it encompasses a wide range of users and extensive movement dynamics. Data processing involved extracting hand-gesture and body-posture markers using MediaPipe, followed by preprocessing and augmentation. Two learning architectures were implemented: a Feedforward Neural Network for alphabet classification and a BiLSTM integrated with an Attention Mechanism for vocabulary recognition. The system was evaluated using accuracy, precision, recall, F1 Score, and K-fold cross-validation. The results demonstrated promising performance: 99% accuracy for alphabet recognition and 78% for vocabulary recognition, with the Attention Mechanism contributing substantially to vocabulary recognition. The system operates in real time at 15-20 FPS and is efficient on mid-range devices, potentially becoming an inclusive communication alternative for the sign language community.

Copyrights © 2026






Journal Info

Abbrev

ice

Publisher

Subject

Arts Humanities Engineering Social Sciences

Description

Sebagai jurnal yang dapat menjadi sarana inovasi bagi para seluruh masyarakat di tingkat Nasional maupun Internasional. Jurnal yang menjadi pilihan bagi seluruh lapisan masyarakat riset yang ingin menerapkan penemuannya dan mensosialisasikannya sehingga bermanfaat untuk banyak orang. Tujuan dari ...